[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:05:14.425717 19967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.127.254:36733
I20260709 13:05:14.427434 19967 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:05:14.428635 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:14.447255 19973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:14.447396 19974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:14.448151 19976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:14.450173 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:15.675246 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:15.675611 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:15.675771 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602315675755 us; error 0 us; skew 500 ppm
I20260709 13:05:15.676596 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:15.684185 19967 webserver.cc:533] Webserver started at http://127.19.127.254:38207/ using document root <none> and password file <none>
I20260709 13:05:15.685101 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:15.685303 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:15.685731 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:15.691833 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/master-0-root/instance:
uuid: "c00e2376160448cdad4a98426ee71b42"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-s3n6"
I20260709 13:05:15.700337 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:05:15.705961 19982 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:15.707520 19967 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:05:15.707964 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/master-0-root
uuid: "c00e2376160448cdad4a98426ee71b42"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-s3n6"
I20260709 13:05:15.708351 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:15.776398 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:15.778115 19967 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:05:15.778648 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:15.858060 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.254:36733
I20260709 13:05:15.858150 20033 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.254:36733 every 8 connection(s)
I20260709 13:05:15.866536 20034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:15.889180 20034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42: Bootstrap starting.
I20260709 13:05:15.895530 20034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.897492 20034 log.cc:826] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42: Log is configured to *not* fsync() on all Append() calls
I20260709 13:05:15.902724 20034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42: No bootstrap required, opened a new log
I20260709 13:05:15.924677 20034 raft_consensus.cc:359] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00e2376160448cdad4a98426ee71b42" member_type: VOTER }
I20260709 13:05:15.925271 20034 raft_consensus.cc:385] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.925479 20034 raft_consensus.cc:740] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c00e2376160448cdad4a98426ee71b42, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.926519 20034 consensus_queue.cc:260] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00e2376160448cdad4a98426ee71b42" member_type: VOTER }
I20260709 13:05:15.927058 20034 raft_consensus.cc:399] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:15.927400 20034 raft_consensus.cc:493] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:15.927722 20034 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.931993 20034 raft_consensus.cc:515] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00e2376160448cdad4a98426ee71b42" member_type: VOTER }
I20260709 13:05:15.932756 20034 leader_election.cc:304] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c00e2376160448cdad4a98426ee71b42; no voters: 
I20260709 13:05:15.934484 20034 leader_election.cc:290] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:15.934899 20037 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:15.937227 20037 raft_consensus.cc:697] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 1 LEADER]: Becoming Leader. State: Replica: c00e2376160448cdad4a98426ee71b42, State: Running, Role: LEADER
I20260709 13:05:15.938362 20037 consensus_queue.cc:237] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00e2376160448cdad4a98426ee71b42" member_type: VOTER }
I20260709 13:05:15.939474 20034 sys_catalog.cc:565] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:15.950897 20038 sys_catalog.cc:455] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c00e2376160448cdad4a98426ee71b42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00e2376160448cdad4a98426ee71b42" member_type: VOTER } }
I20260709 13:05:15.951139 20039 sys_catalog.cc:455] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c00e2376160448cdad4a98426ee71b42. Latest consensus state: current_term: 1 leader_uuid: "c00e2376160448cdad4a98426ee71b42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00e2376160448cdad4a98426ee71b42" member_type: VOTER } }
I20260709 13:05:15.952021 20038 sys_catalog.cc:458] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:15.952042 20039 sys_catalog.cc:458] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:15.955024 20047 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:15.967865 20047 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:15.970412 19967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:15.985338 20047 catalog_manager.cc:1383] Generated new cluster ID: 82054ae0c3be451e940e6b34667036d7
I20260709 13:05:15.985692 20047 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:16.012403 20047 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:16.014221 20047 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:16.041707 20047 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42: Generated new TSK 0
I20260709 13:05:16.043037 20047 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:16.102341 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:16.112834 20056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:16.114053 20057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:16.118301 20059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:16.119230 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:16.120222 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:16.120443 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:16.120651 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602316120618 us; error 0 us; skew 500 ppm
I20260709 13:05:16.121269 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:16.126540 19967 webserver.cc:533] Webserver started at http://127.19.127.193:45913/ using document root <none> and password file <none>
I20260709 13:05:16.127262 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:16.127925 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:16.128259 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:16.129472 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-0-root/instance:
uuid: "c32770b892e048f8807813329e5920dd"
format_stamp: "Formatted at 2026-07-09 13:05:16 on dist-test-slave-s3n6"
I20260709 13:05:16.136389 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:05:16.142010 20064 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.143069 19967 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:05:16.143471 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "c32770b892e048f8807813329e5920dd"
format_stamp: "Formatted at 2026-07-09 13:05:16 on dist-test-slave-s3n6"
I20260709 13:05:16.143934 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:16.169878 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:16.171797 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:16.174682 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:16.180691 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:16.181056 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.181397 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:16.181658 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.278254 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:33283
I20260709 13:05:16.278635 20126 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:33283 every 8 connection(s)
I20260709 13:05:16.287109 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:16.297837 20131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:16.310644 20132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:16.311838 19967 server_base.cc:1061] running on GCE node
W20260709 13:05:16.314146 20134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:16.315682 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:16.316006 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:16.316268 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602316316240 us; error 0 us; skew 500 ppm
I20260709 13:05:16.316936 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:16.319331 20127 heartbeater.cc:344] Connected to a master server at 127.19.127.254:36733
I20260709 13:05:16.319934 20127 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:16.320652 19967 webserver.cc:533] Webserver started at http://127.19.127.194:45369/ using document root <none> and password file <none>
I20260709 13:05:16.321463 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:16.321539 20127 heartbeater.cc:507] Master 127.19.127.254:36733 requested a full tablet report, sending...
I20260709 13:05:16.321807 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:16.322235 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:16.323915 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-1-root/instance:
uuid: "a8ace9ca637044febe962414e307431b"
format_stamp: "Formatted at 2026-07-09 13:05:16 on dist-test-slave-s3n6"
I20260709 13:05:16.325840 19999 ts_manager.cc:194] Registered new tserver with Master: c32770b892e048f8807813329e5920dd (127.19.127.193:33283)
I20260709 13:05:16.329056 19999 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40226
I20260709 13:05:16.331049 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:05:16.336342 20139 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.337378 19967 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:05:16.337713 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-1-root
uuid: "a8ace9ca637044febe962414e307431b"
format_stamp: "Formatted at 2026-07-09 13:05:16 on dist-test-slave-s3n6"
I20260709 13:05:16.338056 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:16.357420 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:16.358884 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:16.361887 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:16.365298 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:16.365525 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.365892 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:16.366094 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.471873 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.194:32971
I20260709 13:05:16.472096 20201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.194:32971 every 8 connection(s)
I20260709 13:05:16.477926 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:16.486550 20205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:16.489495 20206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:16.492889 20202 heartbeater.cc:344] Connected to a master server at 127.19.127.254:36733
I20260709 13:05:16.493319 20202 heartbeater.cc:461] Registering TS with master...
W20260709 13:05:16.494325 20208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:16.494365 20202 heartbeater.cc:507] Master 127.19.127.254:36733 requested a full tablet report, sending...
I20260709 13:05:16.497104 19999 ts_manager.cc:194] Registered new tserver with Master: a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:16.498472 19999 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40230
I20260709 13:05:16.499610 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:16.501457 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:16.501709 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:16.501905 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602316501866 us; error 0 us; skew 500 ppm
I20260709 13:05:16.502497 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:16.505708 19967 webserver.cc:533] Webserver started at http://127.19.127.195:40029/ using document root <none> and password file <none>
I20260709 13:05:16.506235 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:16.506434 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:16.506733 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:16.507863 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-2-root/instance:
uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
format_stamp: "Formatted at 2026-07-09 13:05:16 on dist-test-slave-s3n6"
I20260709 13:05:16.513393 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:16.517328 20213 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.518350 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:05:16.518718 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-2-root
uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
format_stamp: "Formatted at 2026-07-09 13:05:16 on dist-test-slave-s3n6"
I20260709 13:05:16.519078 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602314247669-19967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:16.547086 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:16.548445 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:16.550335 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:16.553683 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:16.554104 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.554499 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:16.554759 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.667487 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.195:33539
I20260709 13:05:16.667646 20275 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.195:33539 every 8 connection(s)
I20260709 13:05:16.687354 20276 heartbeater.cc:344] Connected to a master server at 127.19.127.254:36733
I20260709 13:05:16.687783 20276 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:16.688740 20276 heartbeater.cc:507] Master 127.19.127.254:36733 requested a full tablet report, sending...
I20260709 13:05:16.691059 19999 ts_manager.cc:194] Registered new tserver with Master: dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:16.692093 19967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015353473s
I20260709 13:05:16.692714 19999 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40240
I20260709 13:05:16.750787 19999 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40244:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:16.755031 19999 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:16.860550 20167 tablet_service.cc:1511] Processing CreateTablet for tablet b7696ef2f3854b77b90748592284f8c2 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:16.861138 20165 tablet_service.cc:1511] Processing CreateTablet for tablet bbd6cee70351442cb52711055c4ed04b (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:16.862473 20167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7696ef2f3854b77b90748592284f8c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.863234 20165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd6cee70351442cb52711055c4ed04b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.864671 20166 tablet_service.cc:1511] Processing CreateTablet for tablet b344b473053b4c238def4924ee4f9e39 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:16.866106 20166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b344b473053b4c238def4924ee4f9e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.872758 20164 tablet_service.cc:1511] Processing CreateTablet for tablet 8cd6e6d5cb5040cd848633cbc13f3672 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:16.874398 20164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cd6e6d5cb5040cd848633cbc13f3672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.874756 20091 tablet_service.cc:1511] Processing CreateTablet for tablet b344b473053b4c238def4924ee4f9e39 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:16.879356 20091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b344b473053b4c238def4924ee4f9e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.883721 20092 tablet_service.cc:1511] Processing CreateTablet for tablet b7696ef2f3854b77b90748592284f8c2 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:16.883950 20089 tablet_service.cc:1511] Processing CreateTablet for tablet 8cd6e6d5cb5040cd848633cbc13f3672 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:16.885789 20090 tablet_service.cc:1511] Processing CreateTablet for tablet bbd6cee70351442cb52711055c4ed04b (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:16.888516 20089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cd6e6d5cb5040cd848633cbc13f3672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.892813 20092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7696ef2f3854b77b90748592284f8c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.896605 20088 tablet_service.cc:1511] Processing CreateTablet for tablet 971df458df52428c80d3ae474c9719f6 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:16.899441 20090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd6cee70351442cb52711055c4ed04b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.906343 20088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 971df458df52428c80d3ae474c9719f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.907495 20163 tablet_service.cc:1511] Processing CreateTablet for tablet 971df458df52428c80d3ae474c9719f6 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:16.910048 20163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 971df458df52428c80d3ae474c9719f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.929816 20162 tablet_service.cc:1511] Processing CreateTablet for tablet 1b177a5f2c744fbd984baea329a28b82 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:16.933638 20087 tablet_service.cc:1511] Processing CreateTablet for tablet 1b177a5f2c744fbd984baea329a28b82 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:16.938239 20087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b177a5f2c744fbd984baea329a28b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.945398 20162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b177a5f2c744fbd984baea329a28b82. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:16.956296 20046 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:16.971025 20086 tablet_service.cc:1511] Processing CreateTablet for tablet 5af63f521d7948adbdbde1ca3c59de87 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:16.972600 20086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af63f521d7948adbdbde1ca3c59de87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.977136 20085 tablet_service.cc:1511] Processing CreateTablet for tablet 505e5550d19f4cc586e97ec0a240e353 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:16.978682 20085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505e5550d19f4cc586e97ec0a240e353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.981238 20161 tablet_service.cc:1511] Processing CreateTablet for tablet 5af63f521d7948adbdbde1ca3c59de87 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:16.982759 20161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af63f521d7948adbdbde1ca3c59de87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.983213 20084 tablet_service.cc:1511] Processing CreateTablet for tablet 6716983c2b104bdd8e141a5247695682 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:16.984699 20084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6716983c2b104bdd8e141a5247695682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.985848 20160 tablet_service.cc:1511] Processing CreateTablet for tablet 505e5550d19f4cc586e97ec0a240e353 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:16.987438 20160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505e5550d19f4cc586e97ec0a240e353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.995523 20159 tablet_service.cc:1511] Processing CreateTablet for tablet 6716983c2b104bdd8e141a5247695682 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:16.996932 20159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6716983c2b104bdd8e141a5247695682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.073150 20294 tablet_bootstrap.cc:492] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd: Bootstrap starting.
I20260709 13:05:17.071337 20239 tablet_service.cc:1511] Processing CreateTablet for tablet bbd6cee70351442cb52711055c4ed04b (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:17.081216 20240 tablet_service.cc:1511] Processing CreateTablet for tablet b344b473053b4c238def4924ee4f9e39 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:17.086177 20295 tablet_bootstrap.cc:492] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b: Bootstrap starting.
I20260709 13:05:17.088994 20234 tablet_service.cc:1511] Processing CreateTablet for tablet 505e5550d19f4cc586e97ec0a240e353 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:17.089268 20239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd6cee70351442cb52711055c4ed04b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.078241 20241 tablet_service.cc:1511] Processing CreateTablet for tablet b7696ef2f3854b77b90748592284f8c2 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:17.079689 20236 tablet_service.cc:1511] Processing CreateTablet for tablet 1b177a5f2c744fbd984baea329a28b82 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:17.087185 20238 tablet_service.cc:1511] Processing CreateTablet for tablet 8cd6e6d5cb5040cd848633cbc13f3672 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:17.079704 20237 tablet_service.cc:1511] Processing CreateTablet for tablet 971df458df52428c80d3ae474c9719f6 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:17.089764 20233 tablet_service.cc:1511] Processing CreateTablet for tablet 6716983c2b104bdd8e141a5247695682 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:17.085108 20235 tablet_service.cc:1511] Processing CreateTablet for tablet 5af63f521d7948adbdbde1ca3c59de87 (DEFAULT_TABLE table=test-workload [id=14a3d07b335f457994afa127d1acd837]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:17.098354 20240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b344b473053b4c238def4924ee4f9e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.104247 20234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505e5550d19f4cc586e97ec0a240e353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.108398 20237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 971df458df52428c80d3ae474c9719f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.123051 20241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7696ef2f3854b77b90748592284f8c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.126327 20233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6716983c2b104bdd8e141a5247695682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.130132 20235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af63f521d7948adbdbde1ca3c59de87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.143508 20236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b177a5f2c744fbd984baea329a28b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.167757 20238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cd6e6d5cb5040cd848633cbc13f3672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.184571 20295 tablet_bootstrap.cc:654] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.192759 20294 tablet_bootstrap.cc:654] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.206468 20296 tablet_bootstrap.cc:492] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8: Bootstrap starting.
I20260709 13:05:17.213730 20294 tablet_bootstrap.cc:492] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd: No bootstrap required, opened a new log
I20260709 13:05:17.214643 20294 ts_tablet_manager.cc:1403] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd: Time spent bootstrapping tablet: real 0.166s	user 0.005s	sys 0.014s
I20260709 13:05:17.220346 20294 raft_consensus.cc:359] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.231549 20294 raft_consensus.cc:385] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.231899 20294 raft_consensus.cc:740] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.232806 20294 consensus_queue.cc:260] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.239480 20296 tablet_bootstrap.cc:654] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.240172 20127 heartbeater.cc:499] Master 127.19.127.254:36733 was elected leader, sending a full tablet report...
I20260709 13:05:17.242257 20295 tablet_bootstrap.cc:492] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b: No bootstrap required, opened a new log
I20260709 13:05:17.242715 20295 ts_tablet_manager.cc:1403] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b: Time spent bootstrapping tablet: real 0.182s	user 0.010s	sys 0.007s
I20260709 13:05:17.245893 20295 raft_consensus.cc:359] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.246652 20295 raft_consensus.cc:385] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.246968 20295 raft_consensus.cc:740] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.247321 20294 ts_tablet_manager.cc:1434] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.001s
I20260709 13:05:17.247928 20295 consensus_queue.cc:260] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.248811 20294 tablet_bootstrap.cc:492] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd: Bootstrap starting.
I20260709 13:05:17.262396 20294 tablet_bootstrap.cc:654] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.271081 20296 tablet_bootstrap.cc:492] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8: No bootstrap required, opened a new log
I20260709 13:05:17.271699 20296 ts_tablet_manager.cc:1403] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.027s
I20260709 13:05:17.272496 20294 tablet_bootstrap.cc:492] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd: No bootstrap required, opened a new log
I20260709 13:05:17.273012 20294 ts_tablet_manager.cc:1403] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.001s
I20260709 13:05:17.274955 20296 raft_consensus.cc:359] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.275820 20296 raft_consensus.cc:385] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.276161 20296 raft_consensus.cc:740] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.276292 20294 raft_consensus.cc:359] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.277070 20294 raft_consensus.cc:385] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.277447 20294 raft_consensus.cc:740] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.277168 20296 consensus_queue.cc:260] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.278456 20294 consensus_queue.cc:260] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.280642 20295 ts_tablet_manager.cc:1434] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b: Time spent starting tablet: real 0.038s	user 0.016s	sys 0.023s
I20260709 13:05:17.281406 20294 ts_tablet_manager.cc:1434] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:17.281500 20202 heartbeater.cc:499] Master 127.19.127.254:36733 was elected leader, sending a full tablet report...
I20260709 13:05:17.282179 20295 tablet_bootstrap.cc:492] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b: Bootstrap starting.
I20260709 13:05:17.282421 20294 tablet_bootstrap.cc:492] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd: Bootstrap starting.
W20260709 13:05:17.288874 20128 tablet.cc:2406] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:17.291047 20295 tablet_bootstrap.cc:654] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.292853 20276 heartbeater.cc:499] Master 127.19.127.254:36733 was elected leader, sending a full tablet report...
I20260709 13:05:17.293093 20294 tablet_bootstrap.cc:654] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.294984 20296 ts_tablet_manager.cc:1434] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.003s
I20260709 13:05:17.296321 20296 tablet_bootstrap.cc:492] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8: Bootstrap starting.
I20260709 13:05:17.306133 20296 tablet_bootstrap.cc:654] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.313899 20294 tablet_bootstrap.cc:492] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd: No bootstrap required, opened a new log
I20260709 13:05:17.314415 20294 ts_tablet_manager.cc:1403] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.017s
I20260709 13:05:17.318259 20294 raft_consensus.cc:359] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.319375 20294 raft_consensus.cc:385] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.319835 20294 raft_consensus.cc:740] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.320957 20294 consensus_queue.cc:260] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.323779 20294 ts_tablet_manager.cc:1434] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.005s
I20260709 13:05:17.323879 20296 tablet_bootstrap.cc:492] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8: No bootstrap required, opened a new log
I20260709 13:05:17.324424 20296 ts_tablet_manager.cc:1403] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.011s
I20260709 13:05:17.324796 20294 tablet_bootstrap.cc:492] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd: Bootstrap starting.
I20260709 13:05:17.328037 20296 raft_consensus.cc:359] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.328887 20296 raft_consensus.cc:385] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.329223 20296 raft_consensus.cc:740] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.331353 20301 raft_consensus.cc:493] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.331179 20296 consensus_queue.cc:260] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.332515 20294 tablet_bootstrap.cc:654] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.332038 20301 raft_consensus.cc:515] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.333468 20296 ts_tablet_manager.cc:1434] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:05:17.334622 20296 tablet_bootstrap.cc:492] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8: Bootstrap starting.
I20260709 13:05:17.336469 20301 leader_election.cc:290] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.345173 20294 tablet_bootstrap.cc:492] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd: No bootstrap required, opened a new log
I20260709 13:05:17.345718 20294 ts_tablet_manager.cc:1403] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 13:05:17.349403 20294 raft_consensus.cc:359] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.350245 20294 raft_consensus.cc:385] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.350567 20294 raft_consensus.cc:740] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.351480 20294 consensus_queue.cc:260] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.354907 20294 ts_tablet_manager.cc:1434] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:05:17.356082 20294 tablet_bootstrap.cc:492] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd: Bootstrap starting.
I20260709 13:05:17.357719 20296 tablet_bootstrap.cc:654] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.360507 20295 tablet_bootstrap.cc:492] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b: No bootstrap required, opened a new log
I20260709 13:05:17.361083 20295 ts_tablet_manager.cc:1403] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b: Time spent bootstrapping tablet: real 0.079s	user 0.028s	sys 0.027s
I20260709 13:05:17.364357 20294 tablet_bootstrap.cc:654] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.364223 20295 raft_consensus.cc:359] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.365159 20295 raft_consensus.cc:385] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.365545 20295 raft_consensus.cc:740] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.366436 20295 consensus_queue.cc:260] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.368770 20295 ts_tablet_manager.cc:1434] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:17.369810 20295 tablet_bootstrap.cc:492] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b: Bootstrap starting.
I20260709 13:05:17.376513 20295 tablet_bootstrap.cc:654] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.378782 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd" is_pre_election: true
I20260709 13:05:17.384115 20102 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 0.
I20260709 13:05:17.380647 20296 tablet_bootstrap.cc:492] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8: No bootstrap required, opened a new log
I20260709 13:05:17.381661 20251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" is_pre_election: true
I20260709 13:05:17.385921 20141 leader_election.cc:304] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: 
I20260709 13:05:17.386534 20251 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 0.
I20260709 13:05:17.382220 20294 tablet_bootstrap.cc:492] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd: No bootstrap required, opened a new log
I20260709 13:05:17.387418 20294 ts_tablet_manager.cc:1403] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.000s
I20260709 13:05:17.387722 20301 raft_consensus.cc:2804] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.388114 20301 raft_consensus.cc:493] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.388460 20301 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.390087 20294 raft_consensus.cc:359] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.390715 20294 raft_consensus.cc:385] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.391191 20294 raft_consensus.cc:740] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.392077 20294 consensus_queue.cc:260] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.391198 20296 ts_tablet_manager.cc:1403] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent bootstrapping tablet: real 0.057s	user 0.008s	sys 0.005s
I20260709 13:05:17.393939 20295 tablet_bootstrap.cc:492] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b: No bootstrap required, opened a new log
I20260709 13:05:17.394464 20295 ts_tablet_manager.cc:1403] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.001s
I20260709 13:05:17.396412 20296 raft_consensus.cc:359] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.396569 20301 raft_consensus.cc:515] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.397267 20296 raft_consensus.cc:385] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.397634 20296 raft_consensus.cc:740] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.398983 20298 raft_consensus.cc:493] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.399533 20298 raft_consensus.cc:515] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.399748 20301 leader_election.cc:290] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 1 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.402894 20298 leader_election.cc:290] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.404224 20251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:17.404888 20251 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.405048 20294 ts_tablet_manager.cc:1434] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:17.406364 20294 tablet_bootstrap.cc:492] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd: Bootstrap starting.
I20260709 13:05:17.408730 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:17.409340 20102 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.409159 20295 raft_consensus.cc:359] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.409904 20295 raft_consensus.cc:385] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.410209 20295 raft_consensus.cc:740] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.411096 20295 consensus_queue.cc:260] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.412273 20296 consensus_queue.cc:260] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.414659 20296 ts_tablet_manager.cc:1434] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:05:17.415808 20296 tablet_bootstrap.cc:492] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8: Bootstrap starting.
I20260709 13:05:17.417001 20102 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 1.
I20260709 13:05:17.422231 20296 tablet_bootstrap.cc:654] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.425873 20294 tablet_bootstrap.cc:654] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.438490 20141 leader_election.cc:304] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: 
I20260709 13:05:17.444293 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" is_pre_election: true
I20260709 13:05:17.445078 20250 raft_consensus.cc:2468] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 0.
I20260709 13:05:17.446542 20066 leader_election.cc:304] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:17.447533 20313 raft_consensus.cc:2804] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.447959 20313 raft_consensus.cc:493] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.448330 20313 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.451361 20251 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 1.
I20260709 13:05:17.452898 20303 raft_consensus.cc:493] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.454027 20295 ts_tablet_manager.cc:1434] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b: Time spent starting tablet: real 0.059s	user 0.004s	sys 0.003s
I20260709 13:05:17.453612 20303 raft_consensus.cc:515] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.455081 20295 tablet_bootstrap.cc:492] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b: Bootstrap starting.
I20260709 13:05:17.455804 20313 raft_consensus.cc:515] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.458133 20313 leader_election.cc:290] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 election: Requested vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.459873 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:17.460521 20250 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.462669 20303 leader_election.cc:290] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:17.472600 20295 tablet_bootstrap.cc:654] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.476123 20301 raft_consensus.cc:2804] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.476689 20301 raft_consensus.cc:697] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 1 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:17.479969 20250 raft_consensus.cc:2468] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 1.
I20260709 13:05:17.481173 20066 leader_election.cc:304] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:17.481827 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.482164 20313 raft_consensus.cc:2804] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.482827 20313 raft_consensus.cc:697] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Becoming Leader. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:17.483990 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:17.483770 20313 consensus_queue.cc:237] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
W20260709 13:05:17.485606 20065 leader_election.cc:343] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ace9ca637044febe962414e307431b (127.19.127.194:32971): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:17.486682 20065 leader_election.cc:343] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a8ace9ca637044febe962414e307431b (127.19.127.194:32971): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.486588 20301 consensus_queue.cc:237] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.490728 20298 raft_consensus.cc:493] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.491273 20298 raft_consensus.cc:515] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.493541 20298 leader_election.cc:290] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.495328 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6716983c2b104bdd8e141a5247695682" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.495996 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.496670 20177 raft_consensus.cc:2468] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 0.
W20260709 13:05:17.496857 20065 leader_election.cc:343] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ace9ca637044febe962414e307431b (127.19.127.194:32971): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.498092 20214 leader_election.cc:304] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:17.498420 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6716983c2b104bdd8e141a5247695682" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" is_pre_election: true
I20260709 13:05:17.499125 20303 raft_consensus.cc:2804] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.499588 20303 raft_consensus.cc:493] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:17.499989 20066 leader_election.cc:343] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.500077 20303 raft_consensus.cc:3060] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.500488 20066 leader_election.cc:304] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c32770b892e048f8807813329e5920dd; no voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:17.508050 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd" is_pre_election: true
I20260709 13:05:17.509150 20102 raft_consensus.cc:2468] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 0.
I20260709 13:05:17.512163 20296 tablet_bootstrap.cc:492] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8: No bootstrap required, opened a new log
I20260709 13:05:17.512099 20298 raft_consensus.cc:2749] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.512892 20296 ts_tablet_manager.cc:1403] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent bootstrapping tablet: real 0.097s	user 0.020s	sys 0.037s
I20260709 13:05:17.516901 20296 raft_consensus.cc:359] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.517700 20296 raft_consensus.cc:385] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.518066 20296 raft_consensus.cc:740] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.517247 19999 catalog_manager.cc:5697] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 0 to 1, leader changed from <none> to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 1 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.519482 20294 tablet_bootstrap.cc:492] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd: No bootstrap required, opened a new log
I20260709 13:05:17.519187 20296 consensus_queue.cc:260] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.520130 20294 ts_tablet_manager.cc:1403] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd: Time spent bootstrapping tablet: real 0.114s	user 0.024s	sys 0.033s
I20260709 13:05:17.517225 20303 raft_consensus.cc:515] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.523110 20303 leader_election.cc:290] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:17.524925 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:17.525564 20177 raft_consensus.cc:3060] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.526772 20294 raft_consensus.cc:359] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.527570 20294 raft_consensus.cc:385] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.527884 20294 raft_consensus.cc:740] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.528414 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:17.528999 20102 raft_consensus.cc:3060] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.528690 20294 consensus_queue.cc:260] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.531011 20294 ts_tablet_manager.cc:1434] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:05:17.532009 20294 tablet_bootstrap.cc:492] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd: Bootstrap starting.
I20260709 13:05:17.534492 20295 tablet_bootstrap.cc:492] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b: No bootstrap required, opened a new log
I20260709 13:05:17.534981 20295 ts_tablet_manager.cc:1403] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b: Time spent bootstrapping tablet: real 0.080s	user 0.013s	sys 0.002s
I20260709 13:05:17.540307 20102 raft_consensus.cc:2468] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 1.
I20260709 13:05:17.542272 20215 leader_election.cc:304] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:17.543220 20303 raft_consensus.cc:2804] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.543952 20303 raft_consensus.cc:697] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Becoming Leader. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:17.545100 20303 consensus_queue.cc:237] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.541838 20294 tablet_bootstrap.cc:654] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.541672 20177 raft_consensus.cc:2468] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 1.
I20260709 13:05:17.548024 19996 catalog_manager.cc:5697] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd reported cstate change: term changed from 0 to 1, leader changed from <none> to c32770b892e048f8807813329e5920dd (127.19.127.193). New cstate: current_term: 1 leader_uuid: "c32770b892e048f8807813329e5920dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.552045 20313 raft_consensus.cc:493] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.552881 20313 raft_consensus.cc:515] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.557003 20313 leader_election.cc:290] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.558959 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" is_pre_election: true
W20260709 13:05:17.560737 20066 leader_election.cc:343] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.562897 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.563643 20177 raft_consensus.cc:2468] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 0.
I20260709 13:05:17.563768 20296 ts_tablet_manager.cc:1434] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent starting tablet: real 0.050s	user 0.014s	sys 0.013s
I20260709 13:05:17.565223 20296 tablet_bootstrap.cc:492] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8: Bootstrap starting.
I20260709 13:05:17.565028 20065 leader_election.cc:304] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:17.566421 20313 raft_consensus.cc:2804] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.566855 20313 raft_consensus.cc:493] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.567260 20313 raft_consensus.cc:3060] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.563771 20295 raft_consensus.cc:359] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.568869 20295 raft_consensus.cc:385] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.569185 20295 raft_consensus.cc:740] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.569998 20295 consensus_queue.cc:260] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.572315 20295 ts_tablet_manager.cc:1434] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.001s
I20260709 13:05:17.573109 20296 tablet_bootstrap.cc:654] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.573426 20295 tablet_bootstrap.cc:492] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b: Bootstrap starting.
I20260709 13:05:17.574574 20313 raft_consensus.cc:515] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.591364 20313 leader_election.cc:290] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 election: Requested vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.592176 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:17.592783 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:17.592967 20177 raft_consensus.cc:3060] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:17.594264 20066 leader_election.cc:343] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.596010 20296 tablet_bootstrap.cc:492] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8: No bootstrap required, opened a new log
I20260709 13:05:17.596536 20296 ts_tablet_manager.cc:1403] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.001s
I20260709 13:05:17.598958 20294 tablet_bootstrap.cc:492] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd: No bootstrap required, opened a new log
I20260709 13:05:17.599483 20294 ts_tablet_manager.cc:1403] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.009s
I20260709 13:05:17.602855 20294 raft_consensus.cc:359] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.603657 20294 raft_consensus.cc:385] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.603684 20177 raft_consensus.cc:2468] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 1.
I20260709 13:05:17.604051 20294 raft_consensus.cc:740] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.605562 20065 leader_election.cc:304] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:17.605970 20294 consensus_queue.cc:260] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.606855 20313 raft_consensus.cc:2804] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.615250 20295 tablet_bootstrap.cc:654] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.615056 20296 raft_consensus.cc:359] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.613775 19997 catalog_manager.cc:5697] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.619717 20296 raft_consensus.cc:385] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.620223 20296 raft_consensus.cc:740] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.622357 20298 raft_consensus.cc:493] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.622860 20298 raft_consensus.cc:515] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.626036 20298 leader_election.cc:290] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.627184 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.627525 20294 ts_tablet_manager.cc:1434] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:05:17.627308 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" is_pre_election: true
I20260709 13:05:17.628557 20294 tablet_bootstrap.cc:492] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd: Bootstrap starting.
W20260709 13:05:17.628852 20065 leader_election.cc:343] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ace9ca637044febe962414e307431b (127.19.127.194:32971): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:17.629498 20066 leader_election.cc:343] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.629550 20296 consensus_queue.cc:260] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.630023 20066 leader_election.cc:304] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c32770b892e048f8807813329e5920dd; no voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:17.631223 20298 raft_consensus.cc:2749] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.637818 20296 ts_tablet_manager.cc:1434] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.001s
I20260709 13:05:17.638862 20296 tablet_bootstrap.cc:492] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8: Bootstrap starting.
I20260709 13:05:17.639946 20313 raft_consensus.cc:697] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Becoming Leader. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:17.646740 20313 consensus_queue.cc:237] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.658600 20296 tablet_bootstrap.cc:654] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.658618 20295 tablet_bootstrap.cc:492] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b: No bootstrap required, opened a new log
I20260709 13:05:17.658943 20294 tablet_bootstrap.cc:654] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.659685 20295 ts_tablet_manager.cc:1403] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b: Time spent bootstrapping tablet: real 0.087s	user 0.019s	sys 0.003s
I20260709 13:05:17.667049 20296 tablet_bootstrap.cc:492] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8: No bootstrap required, opened a new log
I20260709 13:05:17.667575 20296 ts_tablet_manager.cc:1403] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.003s
I20260709 13:05:17.671728 20295 raft_consensus.cc:359] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.672760 20295 raft_consensus.cc:385] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.673142 20295 raft_consensus.cc:740] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.674093 20295 consensus_queue.cc:260] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.674693 20296 raft_consensus.cc:359] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.675462 20296 raft_consensus.cc:385] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.675796 20296 raft_consensus.cc:740] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.676725 20296 consensus_queue.cc:260] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.677518 20295 ts_tablet_manager.cc:1434] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.004s
I20260709 13:05:17.678939 20295 tablet_bootstrap.cc:492] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b: Bootstrap starting.
I20260709 13:05:17.682538 20296 ts_tablet_manager.cc:1434] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:05:17.683751 20296 tablet_bootstrap.cc:492] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8: Bootstrap starting.
I20260709 13:05:17.688321 20295 tablet_bootstrap.cc:654] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.694758 20294 tablet_bootstrap.cc:492] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd: No bootstrap required, opened a new log
I20260709 13:05:17.695263 20294 ts_tablet_manager.cc:1403] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.011s
I20260709 13:05:17.696934 20296 tablet_bootstrap.cc:654] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.698498 20294 raft_consensus.cc:359] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.699229 20294 raft_consensus.cc:385] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.699550 20294 raft_consensus.cc:740] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.707535 20313 raft_consensus.cc:493] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.708110 20313 raft_consensus.cc:515] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.708724 19996 catalog_manager.cc:5697] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd reported cstate change: term changed from 0 to 1, leader changed from <none> to c32770b892e048f8807813329e5920dd (127.19.127.193). New cstate: current_term: 1 leader_uuid: "c32770b892e048f8807813329e5920dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.710279 20313 leader_election.cc:290] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.710958 20303 raft_consensus.cc:493] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.711337 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.715068 20177 raft_consensus.cc:2468] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 0.
I20260709 13:05:17.714973 20303 raft_consensus.cc:515] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.716493 20065 leader_election.cc:304] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: 
I20260709 13:05:17.717152 20303 leader_election.cc:290] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:17.717470 20333 raft_consensus.cc:2804] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.717839 20333 raft_consensus.cc:493] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.718180 20333 raft_consensus.cc:3060] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.714010 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" is_pre_election: true
I20260709 13:05:17.710776 20294 consensus_queue.cc:260] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.723471 20294 ts_tablet_manager.cc:1434] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 13:05:17.723310 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd" is_pre_election: true
I20260709 13:05:17.723943 20102 raft_consensus.cc:2468] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 0.
I20260709 13:05:17.724303 20294 tablet_bootstrap.cc:492] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd: Bootstrap starting.
W20260709 13:05:17.725822 20066 leader_election.cc:343] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.726918 20215 leader_election.cc:304] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:17.727859 20303 raft_consensus.cc:2804] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.728229 20303 raft_consensus.cc:493] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.728212 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.728705 20303 raft_consensus.cc:3060] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:17.729676 20214 leader_election.cc:343] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ace9ca637044febe962414e307431b (127.19.127.194:32971): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.733831 20326 raft_consensus.cc:493] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.734364 20326 raft_consensus.cc:515] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.736840 20326 leader_election.cc:290] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:17.738940 20294 tablet_bootstrap.cc:654] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.739580 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd" is_pre_election: true
I20260709 13:05:17.740243 20102 raft_consensus.cc:2468] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 0.
I20260709 13:05:17.735138 20303 raft_consensus.cc:515] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.741981 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.742740 20303 leader_election.cc:290] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:17.743479 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b"
W20260709 13:05:17.746430 20214 leader_election.cc:343] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a8ace9ca637044febe962414e307431b (127.19.127.194:32971): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:17.747431 20214 leader_election.cc:343] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8ace9ca637044febe962414e307431b (127.19.127.194:32971): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.749735 20215 leader_election.cc:304] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: a8ace9ca637044febe962414e307431b
I20260709 13:05:17.751005 20303 raft_consensus.cc:2804] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.751636 20303 raft_consensus.cc:493] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.752077 20303 raft_consensus.cc:3060] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.751910 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:17.753005 20102 raft_consensus.cc:3060] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.758134 20333 raft_consensus.cc:515] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.762666 20102 raft_consensus.cc:2468] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 1.
I20260709 13:05:17.766194 20215 leader_election.cc:304] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: a8ace9ca637044febe962414e307431b
I20260709 13:05:17.766523 20294 tablet_bootstrap.cc:492] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd: No bootstrap required, opened a new log
I20260709 13:05:17.767789 20326 raft_consensus.cc:2804] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.767820 20294 ts_tablet_manager.cc:1403] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.003s
I20260709 13:05:17.775486 20294 raft_consensus.cc:359] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.776503 20294 raft_consensus.cc:385] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.776878 20294 raft_consensus.cc:740] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.777815 20294 consensus_queue.cc:260] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.786240 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:17.786924 20177 raft_consensus.cc:3060] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.788230 20333 leader_election.cc:290] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 election: Requested vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:17.790611 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:17.796866 20296 tablet_bootstrap.cc:492] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8: No bootstrap required, opened a new log
W20260709 13:05:17.794862 20066 leader_election.cc:343] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.797397 20296 ts_tablet_manager.cc:1403] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent bootstrapping tablet: real 0.114s	user 0.034s	sys 0.056s
I20260709 13:05:17.799574 20303 raft_consensus.cc:515] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.800726 20296 raft_consensus.cc:359] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.801470 20296 raft_consensus.cc:385] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.801826 20296 raft_consensus.cc:740] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.801898 20303 leader_election.cc:290] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:17.802600 20296 consensus_queue.cc:260] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.803077 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b"
W20260709 13:05:17.804873 20214 leader_election.cc:343] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a8ace9ca637044febe962414e307431b (127.19.127.194:32971): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.805024 20296 ts_tablet_manager.cc:1434] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:17.805687 20294 ts_tablet_manager.cc:1434] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 13:05:17.806143 20296 tablet_bootstrap.cc:492] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8: Bootstrap starting.
I20260709 13:05:17.806361 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:17.806844 20102 raft_consensus.cc:3060] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.796139 20326 raft_consensus.cc:697] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Becoming Leader. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:17.810956 20326 consensus_queue.cc:237] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.802649 20295 tablet_bootstrap.cc:492] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b: No bootstrap required, opened a new log
I20260709 13:05:17.814793 20177 raft_consensus.cc:2468] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 1.
I20260709 13:05:17.815238 20102 raft_consensus.cc:2468] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 1.
I20260709 13:05:17.815716 20295 ts_tablet_manager.cc:1403] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.022s
I20260709 13:05:17.820971 20065 leader_election.cc:304] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:17.822160 20333 raft_consensus.cc:2804] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.822628 20333 raft_consensus.cc:697] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Becoming Leader. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:17.823645 20295 raft_consensus.cc:359] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.824400 20295 raft_consensus.cc:385] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.824292 20215 leader_election.cc:304] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: a8ace9ca637044febe962414e307431b
I20260709 13:05:17.824733 20295 raft_consensus.cc:740] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.825536 20295 consensus_queue.cc:260] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.825702 20333 consensus_queue.cc:237] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.828250 20295 ts_tablet_manager.cc:1434] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:05:17.829325 20295 tablet_bootstrap.cc:492] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b: Bootstrap starting.
I20260709 13:05:17.819640 20296 tablet_bootstrap.cc:654] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.829514 20303 raft_consensus.cc:2804] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.838662 20303 raft_consensus.cc:697] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Becoming Leader. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:17.844002 20303 consensus_queue.cc:237] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.839179 20295 tablet_bootstrap.cc:654] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.844201 19996 catalog_manager.cc:5697] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.861153 20296 tablet_bootstrap.cc:492] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8: No bootstrap required, opened a new log
I20260709 13:05:17.861752 20296 ts_tablet_manager.cc:1403] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.018s
I20260709 13:05:17.864753 20296 raft_consensus.cc:359] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.865422 20296 raft_consensus.cc:385] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.865696 20296 raft_consensus.cc:740] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.869264 20296 consensus_queue.cc:260] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.871688 20296 ts_tablet_manager.cc:1434] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:05:17.872711 20296 tablet_bootstrap.cc:492] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8: Bootstrap starting.
I20260709 13:05:17.873850 19999 catalog_manager.cc:5697] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd reported cstate change: term changed from 0 to 1, leader changed from <none> to c32770b892e048f8807813329e5920dd (127.19.127.193). New cstate: current_term: 1 leader_uuid: "c32770b892e048f8807813329e5920dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.878932 20296 tablet_bootstrap.cc:654] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.880896 20295 tablet_bootstrap.cc:492] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b: No bootstrap required, opened a new log
I20260709 13:05:17.884513 20295 ts_tablet_manager.cc:1403] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.006s
I20260709 13:05:17.880872 19996 catalog_manager.cc:5697] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.885327 20296 tablet_bootstrap.cc:492] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8: No bootstrap required, opened a new log
I20260709 13:05:17.885788 20296 ts_tablet_manager.cc:1403] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 13:05:17.889861 20296 raft_consensus.cc:359] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.890575 20296 raft_consensus.cc:385] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.890836 20296 raft_consensus.cc:740] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.891467 20296 consensus_queue.cc:260] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.892184 20301 consensus_queue.cc:1048] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:17.894176 20296 ts_tablet_manager.cc:1434] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:17.887856 20295 raft_consensus.cc:359] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.895792 20295 raft_consensus.cc:385] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.896148 20295 raft_consensus.cc:740] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.897063 20295 consensus_queue.cc:260] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.899906 20295 ts_tablet_manager.cc:1434] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 13:05:17.901491 20295 tablet_bootstrap.cc:492] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b: Bootstrap starting.
I20260709 13:05:17.908978 20295 tablet_bootstrap.cc:654] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.918641 20295 tablet_bootstrap.cc:492] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b: No bootstrap required, opened a new log
I20260709 13:05:17.919139 20295 ts_tablet_manager.cc:1403] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 13:05:17.923056 20295 raft_consensus.cc:359] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.924049 20295 raft_consensus.cc:385] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.924765 20295 raft_consensus.cc:740] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.925387 20295 consensus_queue.cc:260] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.926271 20322 consensus_queue.cc:1048] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:17.957195 20303 raft_consensus.cc:493] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.957827 20303 raft_consensus.cc:515] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.961294 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd" is_pre_election: true
I20260709 13:05:17.962040 20101 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 0.
W20260709 13:05:17.962244 20046 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:17.963184 20303 leader_election.cc:290] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:17.963449 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.963376 20215 leader_election.cc:304] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:17.964159 20177 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 0.
I20260709 13:05:17.964361 20303 raft_consensus.cc:2804] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.964725 20303 raft_consensus.cc:493] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.965121 20303 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.966662 20326 raft_consensus.cc:493] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.967029 20326 raft_consensus.cc:515] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.969157 20326 leader_election.cc:290] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:17.970361 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6716983c2b104bdd8e141a5247695682" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:17.971060 20177 raft_consensus.cc:2468] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 0.
I20260709 13:05:17.970363 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6716983c2b104bdd8e141a5247695682" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd" is_pre_election: true
I20260709 13:05:17.973040 20295 ts_tablet_manager.cc:1434] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b: Time spent starting tablet: real 0.054s	user 0.028s	sys 0.008s
I20260709 13:05:17.973024 20214 leader_election.cc:304] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:17.973556 20101 raft_consensus.cc:2468] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 0.
I20260709 13:05:17.974148 20295 tablet_bootstrap.cc:492] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b: Bootstrap starting.
I20260709 13:05:17.974396 20326 raft_consensus.cc:2804] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.974783 20326 raft_consensus.cc:493] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.975118 20326 raft_consensus.cc:3060] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.980014 20295 tablet_bootstrap.cc:654] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.989343 20303 raft_consensus.cc:515] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.991034 20295 tablet_bootstrap.cc:492] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b: No bootstrap required, opened a new log
I20260709 13:05:17.991531 20295 ts_tablet_manager.cc:1403] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 13:05:17.992554 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:17.993168 20102 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.993065 20177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:17.993654 20177 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.994395 20295 raft_consensus.cc:359] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.995054 20295 raft_consensus.cc:385] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.995859 20295 raft_consensus.cc:740] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.996583 20295 consensus_queue.cc:260] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:17.999670 20295 ts_tablet_manager.cc:1434] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:05:18.001648 20177 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 1.
I20260709 13:05:17.999532 20102 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 1.
I20260709 13:05:18.003146 20214 leader_election.cc:304] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:18.004204 20342 raft_consensus.cc:2804] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:18.005385 20342 raft_consensus.cc:697] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Becoming Leader. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:18.006275 20342 consensus_queue.cc:237] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.007572 20177 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.004279 20303 leader_election.cc:290] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:17.986631 20326 raft_consensus.cc:515] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.017634 20333 consensus_queue.cc:1048] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.019481 20326 leader_election.cc:290] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:18.019802 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6716983c2b104bdd8e141a5247695682" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:18.020448 20176 raft_consensus.cc:3060] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.020162 20326 consensus_queue.cc:1048] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.021024 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6716983c2b104bdd8e141a5247695682" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:18.021935 20102 raft_consensus.cc:3060] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.031683 20102 raft_consensus.cc:2468] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 1.
I20260709 13:05:18.032886 20176 raft_consensus.cc:2468] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 1.
I20260709 13:05:18.033867 20215 leader_election.cc:304] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:18.032014 19999 catalog_manager.cc:5697] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:18.035517 20342 raft_consensus.cc:2804] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:18.036171 20342 raft_consensus.cc:697] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Becoming Leader. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:18.037037 20342 consensus_queue.cc:237] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.041759 20251 raft_consensus.cc:3060] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.053345 20313 consensus_queue.cc:1048] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.052202 20333 consensus_queue.cc:1048] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:18.062094 20357 consensus_queue.cc:1048] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.091161 19997 catalog_manager.cc:5697] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:18.095613 20353 raft_consensus.cc:493] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.096165 20353 raft_consensus.cc:515] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.098313 20353 leader_election.cc:290] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:18.099339 20313 consensus_queue.cc:1048] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.101449 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd" is_pre_election: true
I20260709 13:05:18.102367 20101 raft_consensus.cc:2393] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a8ace9ca637044febe962414e307431b in current term 1: Already voted for candidate dbe8bd74fd5d4c82960860fe361e08c8 in this term.
I20260709 13:05:18.102151 20249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" is_pre_election: true
I20260709 13:05:18.103957 20141 leader_election.cc:304] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8ace9ca637044febe962414e307431b; no voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:18.105065 20353 raft_consensus.cc:3060] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.112411 20353 raft_consensus.cc:2749] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.160974 20326 raft_consensus.cc:493] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.161751 20326 raft_consensus.cc:515] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.163565 20326 leader_election.cc:290] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:18.164441 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32770b892e048f8807813329e5920dd" is_pre_election: true
I20260709 13:05:18.164592 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ace9ca637044febe962414e307431b" is_pre_election: true
I20260709 13:05:18.165239 20176 raft_consensus.cc:2393] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe8bd74fd5d4c82960860fe361e08c8 in current term 1: Already voted for candidate c32770b892e048f8807813329e5920dd in this term.
I20260709 13:05:18.166659 20214 leader_election.cc:304] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe8bd74fd5d4c82960860fe361e08c8; no voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd
I20260709 13:05:18.168028 20326 raft_consensus.cc:3060] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.175485 20326 raft_consensus.cc:2749] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.229779 20298 consensus_queue.cc:1048] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.241950 20365 consensus_queue.cc:1048] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.254484 20342 consensus_queue.cc:1048] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.270326 20176 raft_consensus.cc:3060] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.278570 20303 consensus_queue.cc:1048] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.301690 20250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
mode: GRACEFUL
new_leader_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:51408
I20260709 13:05:18.302451 20250 raft_consensus.cc:606] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Received request to transfer leadership to TS c32770b892e048f8807813329e5920dd
I20260709 13:05:18.308658 20250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
mode: GRACEFUL
new_leader_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:51408
I20260709 13:05:18.309365 20250 raft_consensus.cc:606] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Received request to transfer leadership to TS c32770b892e048f8807813329e5920dd
I20260709 13:05:18.326243 20176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b"
dest_uuid: "a8ace9ca637044febe962414e307431b"
mode: GRACEFUL
new_leader_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:54426
I20260709 13:05:18.326820 20176 raft_consensus.cc:606] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 1 LEADER]: Received request to transfer leadership to TS c32770b892e048f8807813329e5920dd
I20260709 13:05:18.336149 20250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "971df458df52428c80d3ae474c9719f6"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
mode: GRACEFUL
new_leader_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:51408
I20260709 13:05:18.336728 20250 raft_consensus.cc:606] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Received request to transfer leadership to TS a8ace9ca637044febe962414e307431b
I20260709 13:05:18.342331 20250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
mode: GRACEFUL
new_leader_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:51408
I20260709 13:05:18.342947 20250 raft_consensus.cc:606] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Received request to transfer leadership to TS a8ace9ca637044febe962414e307431b
I20260709 13:05:18.357534 20101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87"
dest_uuid: "c32770b892e048f8807813329e5920dd"
mode: GRACEFUL
new_leader_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:56594
I20260709 13:05:18.358176 20101 raft_consensus.cc:606] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Received request to transfer leadership to TS a8ace9ca637044febe962414e307431b
I20260709 13:05:18.364439 20101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353"
dest_uuid: "c32770b892e048f8807813329e5920dd"
mode: GRACEFUL
new_leader_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:56594
I20260709 13:05:18.365084 20101 raft_consensus.cc:606] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Received request to transfer leadership to TS a8ace9ca637044febe962414e307431b
I20260709 13:05:18.414325 20303 consensus_queue.cc:1048] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.415053 20326 consensus_queue.cc:1048] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.430076 20342 raft_consensus.cc:993] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8: : Instructing follower a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:18.430540 20342 raft_consensus.cc:1081] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Signalling peer a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:18.433056 20175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "971df458df52428c80d3ae474c9719f6"
dest_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:54412
I20260709 13:05:18.433662 20175 raft_consensus.cc:493] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.434027 20175 raft_consensus.cc:3060] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.439108 20175 raft_consensus.cc:515] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.442459 20175 leader_election.cc:290] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:18.443679 20176 raft_consensus.cc:1240] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Rejecting Update request from peer dbe8bd74fd5d4c82960860fe361e08c8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:18.443997 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:18.444628 20250 raft_consensus.cc:3055] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.444984 20250 raft_consensus.cc:740] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:18.445837 20250 consensus_queue.cc:260] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.447268 20250 raft_consensus.cc:3060] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.449297 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:18.450487 20101 raft_consensus.cc:3060] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.457904 20101 raft_consensus.cc:2468] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:18.458361 20250 raft_consensus.cc:2468] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:18.459225 20141 leader_election.cc:304] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: 
I20260709 13:05:18.460043 20353 raft_consensus.cc:2804] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.460518 20353 raft_consensus.cc:697] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:18.461325 20353 consensus_queue.cc:237] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.468833 19997 catalog_manager.cc:5697] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 1 to 2, leader changed from dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195) to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.537300 20377 raft_consensus.cc:993] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b: : Instructing follower c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:18.537838 20301 raft_consensus.cc:1081] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 1 LEADER]: Signalling peer c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:18.539808 20101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b"
dest_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:56574
I20260709 13:05:18.540488 20101 raft_consensus.cc:493] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.540911 20101 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.545504 20101 raft_consensus.cc:515] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.547703 20101 leader_election.cc:290] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 2 election: Requested vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:18.548640 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:18.548889 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:18.549304 20176 raft_consensus.cc:3055] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.550011 20176 raft_consensus.cc:740] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:18.549532 20250 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.550850 20176 consensus_queue.cc:260] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.552693 20176 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.558378 20250 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 2.
I20260709 13:05:18.559849 20066 leader_election.cc:304] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:18.560776 20298 raft_consensus.cc:2804] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.560711 20326 consensus_queue.cc:1048] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.560516 20176 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 2.
I20260709 13:05:18.561408 20298 raft_consensus.cc:697] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Becoming Leader. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:18.565174 20298 consensus_queue.cc:237] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.579527 19997 catalog_manager.cc:5697] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd reported cstate change: term changed from 1 to 2, leader changed from a8ace9ca637044febe962414e307431b (127.19.127.194) to c32770b892e048f8807813329e5920dd (127.19.127.193). New cstate: current_term: 2 leader_uuid: "c32770b892e048f8807813329e5920dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.586185 20357 consensus_queue.cc:1048] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.595510 20326 raft_consensus.cc:993] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8: : Instructing follower c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:18.596040 20357 raft_consensus.cc:1081] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Signalling peer c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:18.597704 20101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2"
dest_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:56586
I20260709 13:05:18.598284 20101 raft_consensus.cc:493] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.598721 20101 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.605413 20101 raft_consensus.cc:515] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.607386 20101 leader_election.cc:290] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 2 election: Requested vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:18.608407 20175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:18.608588 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:18.609262 20250 raft_consensus.cc:3055] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.609266 20175 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.609804 20250 raft_consensus.cc:740] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:18.610868 20250 consensus_queue.cc:260] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.612373 20250 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.616945 20175 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 2.
I20260709 13:05:18.618294 20065 leader_election.cc:304] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: 
I20260709 13:05:18.619469 20250 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 2.
I20260709 13:05:18.619403 20298 raft_consensus.cc:2804] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.620373 20298 raft_consensus.cc:697] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Becoming Leader. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:18.621279 20298 consensus_queue.cc:237] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.629920 19997 catalog_manager.cc:5697] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd reported cstate change: term changed from 1 to 2, leader changed from dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195) to c32770b892e048f8807813329e5920dd (127.19.127.193). New cstate: current_term: 2 leader_uuid: "c32770b892e048f8807813329e5920dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.649361 20357 consensus_queue.cc:1048] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.659230 20342 consensus_queue.cc:1048] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.691174 20342 raft_consensus.cc:993] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8: : Instructing follower c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:18.691797 20357 raft_consensus.cc:1081] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Signalling peer c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:18.693560 20101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39"
dest_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:56586
I20260709 13:05:18.694271 20101 raft_consensus.cc:493] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.694671 20101 raft_consensus.cc:3060] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.696738 20313 raft_consensus.cc:993] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd: : Instructing follower a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:18.697182 20313 raft_consensus.cc:1081] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Signalling peer a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:18.698966 20175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87"
dest_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:54396
I20260709 13:05:18.699563 20175 raft_consensus.cc:493] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.699945 20175 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.702592 20101 raft_consensus.cc:515] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.705638 20101 leader_election.cc:290] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 2 election: Requested vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:18.705561 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:18.706132 20175 raft_consensus.cc:515] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.706223 20176 raft_consensus.cc:3060] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.707603 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:18.708240 20250 raft_consensus.cc:3055] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.708561 20250 raft_consensus.cc:740] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:18.709465 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:18.709358 20250 consensus_queue.cc:260] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.709998 20102 raft_consensus.cc:3055] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.710836 20102 raft_consensus.cc:740] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:18.711429 20250 raft_consensus.cc:3060] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.711643 20102 consensus_queue.cc:260] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.712754 20102 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.714309 20176 raft_consensus.cc:2468] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 2.
I20260709 13:05:18.715698 20065 leader_election.cc:304] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: 
I20260709 13:05:18.716061 20175 leader_election.cc:290] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:18.717362 20313 raft_consensus.cc:2804] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.717872 20249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:18.718041 20313 raft_consensus.cc:697] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Becoming Leader. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:18.718479 20249 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.718961 20313 consensus_queue.cc:237] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.720566 20250 raft_consensus.cc:2468] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 2.
I20260709 13:05:18.725780 20249 raft_consensus.cc:2468] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:18.726152 20102 raft_consensus.cc:2468] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:18.727325 20141 leader_election.cc:304] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:18.728183 20301 raft_consensus.cc:2804] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.728662 20301 raft_consensus.cc:697] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:18.729517 20301 consensus_queue.cc:237] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.729900 19999 catalog_manager.cc:5697] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd reported cstate change: term changed from 1 to 2, leader changed from dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195) to c32770b892e048f8807813329e5920dd (127.19.127.193). New cstate: current_term: 2 leader_uuid: "c32770b892e048f8807813329e5920dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.737390 19997 catalog_manager.cc:5697] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 1 to 2, leader changed from c32770b892e048f8807813329e5920dd (127.19.127.193) to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.838868 20357 raft_consensus.cc:993] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8: : Instructing follower a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:18.839359 20326 raft_consensus.cc:1081] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Signalling peer a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:18.840994 20175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82"
dest_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:54412
I20260709 13:05:18.841576 20175 raft_consensus.cc:493] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.841941 20175 raft_consensus.cc:3060] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.846997 20298 raft_consensus.cc:993] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd: : Instructing follower a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:18.847513 20313 raft_consensus.cc:1081] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Signalling peer a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:18.848657 20175 raft_consensus.cc:515] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.849323 20176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353"
dest_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:54396
I20260709 13:05:18.849916 20176 raft_consensus.cc:493] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.850307 20176 raft_consensus.cc:3060] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.851022 20175 leader_election.cc:290] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:18.852234 20249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:18.852794 20249 raft_consensus.cc:3055] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.853104 20249 raft_consensus.cc:740] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:18.853884 20249 consensus_queue.cc:260] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.855751 20249 raft_consensus.cc:3060] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.856588 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:18.857249 20102 raft_consensus.cc:3060] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.861876 20102 raft_consensus.cc:2468] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:18.862701 20249 raft_consensus.cc:2468] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:18.863786 20141 leader_election.cc:304] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:18.864637 20301 raft_consensus.cc:2804] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.865118 20301 raft_consensus.cc:697] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:18.865108 20176 raft_consensus.cc:515] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.866103 20301 consensus_queue.cc:237] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.867635 20176 leader_election.cc:290] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:18.868750 20102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:18.869402 20102 raft_consensus.cc:3055] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.869746 20102 raft_consensus.cc:740] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:18.870157 20249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:18.870796 20249 raft_consensus.cc:3060] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.870676 20102 consensus_queue.cc:260] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.874462 20102 raft_consensus.cc:3060] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.877076 19997 catalog_manager.cc:5697] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 1 to 2, leader changed from dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195) to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.878191 20249 raft_consensus.cc:2468] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:18.879704 20141 leader_election.cc:304] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:18.880548 20301 raft_consensus.cc:2804] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.881289 20301 raft_consensus.cc:697] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:18.881418 20102 raft_consensus.cc:2468] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:18.882685 20301 consensus_queue.cc:237] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:18.892288 19997 catalog_manager.cc:5697] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 1 to 2, leader changed from c32770b892e048f8807813329e5920dd (127.19.127.193) to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.986824 20102 raft_consensus.cc:1275] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:18.988371 20301 consensus_queue.cc:1048] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:18.999617 20249 raft_consensus.cc:1275] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.002170 20377 consensus_queue.cc:1048] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.080695 20176 raft_consensus.cc:1275] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.082361 20313 consensus_queue.cc:1048] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.092773 20249 raft_consensus.cc:1275] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.094594 20333 consensus_queue.cc:1048] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.166118 20249 raft_consensus.cc:1275] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.167071 20176 raft_consensus.cc:1275] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.167577 20377 consensus_queue.cc:1048] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.168466 20333 consensus_queue.cc:1048] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.177398 20102 raft_consensus.cc:1275] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.179726 20301 consensus_queue.cc:1048] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.184604 20249 raft_consensus.cc:1275] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.187477 20313 consensus_queue.cc:1048] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.224072 20176 raft_consensus.cc:1275] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.225457 20333 consensus_queue.cc:1048] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.227802 20249 raft_consensus.cc:1275] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.229969 20333 consensus_queue.cc:1048] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:19.252892 20249 raft_consensus.cc:1275] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.254235 20301 consensus_queue.cc:1048] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.264513 20102 raft_consensus.cc:1275] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.266238 20377 consensus_queue.cc:1048] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.295125 20102 raft_consensus.cc:1275] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.296988 20377 consensus_queue.cc:1048] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.309952 20249 raft_consensus.cc:1275] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.313063 20377 consensus_queue.cc:1048] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:21.401211 19967 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:05:21.402153 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:21.418800 20176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87"
dest_uuid: "a8ace9ca637044febe962414e307431b"
mode: GRACEFUL
new_leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
 from {username='slave'} at 127.0.0.1:48388
I20260709 13:05:21.419379 20176 raft_consensus.cc:606] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Received request to transfer leadership to TS dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:21.430789 20102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2"
dest_uuid: "c32770b892e048f8807813329e5920dd"
mode: GRACEFUL
new_leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
 from {username='slave'} at 127.0.0.1:53502
I20260709 13:05:21.431636 20102 raft_consensus.cc:606] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Received request to transfer leadership to TS dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:21.432852 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:21.433319 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:21.692488 20298 raft_consensus.cc:993] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd: : Instructing follower dbe8bd74fd5d4c82960860fe361e08c8 to start an election
I20260709 13:05:21.692919 20298 raft_consensus.cc:1081] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Signalling peer dbe8bd74fd5d4c82960860fe361e08c8 to start an election
I20260709 13:05:21.694458 20250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
 from {username='slave'} at 127.0.0.1:51398
I20260709 13:05:21.694947 20250 raft_consensus.cc:493] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:21.695226 20250 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:21.701066 20250 raft_consensus.cc:515] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:21.703016 20250 leader_election.cc:290] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 3 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:21.703794 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:21.704146 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:21.704413 20101 raft_consensus.cc:3055] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:21.704737 20176 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:21.704773 20101 raft_consensus.cc:740] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:21.705679 20101 consensus_queue.cc:260] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:21.706653 20101 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:21.710068 20176 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 3.
I20260709 13:05:21.711333 20214 leader_election.cc:304] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:21.711447 20101 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 3.
I20260709 13:05:21.712226 20391 raft_consensus.cc:2804] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:21.713317 20391 raft_consensus.cc:697] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Becoming Leader. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:21.714018 20391 consensus_queue.cc:237] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:21.724624 19997 catalog_manager.cc:5697] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 reported cstate change: term changed from 2 to 3, leader changed from c32770b892e048f8807813329e5920dd (127.19.127.193) to dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:21.812674 20353 raft_consensus.cc:993] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b: : Instructing follower dbe8bd74fd5d4c82960860fe361e08c8 to start an election
I20260709 13:05:21.813144 20353 raft_consensus.cc:1081] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Signalling peer dbe8bd74fd5d4c82960860fe361e08c8 to start an election
I20260709 13:05:21.814664 20250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
 from {username='slave'} at 127.0.0.1:51392
I20260709 13:05:21.815320 20250 raft_consensus.cc:493] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:21.815639 20250 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:21.821501 20250 raft_consensus.cc:515] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:21.823240 20250 leader_election.cc:290] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 3 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:21.824087 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:21.824352 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:21.824712 20101 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:21.824909 20176 raft_consensus.cc:3055] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:21.825261 20176 raft_consensus.cc:740] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:21.826120 20176 consensus_queue.cc:260] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:21.827538 20176 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:21.832109 20101 raft_consensus.cc:2468] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 3.
I20260709 13:05:21.833446 20215 leader_election.cc:304] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:21.834204 20391 raft_consensus.cc:2804] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:21.834204 20176 raft_consensus.cc:2468] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 3.
I20260709 13:05:21.834839 20391 raft_consensus.cc:697] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Becoming Leader. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:21.835716 20391 consensus_queue.cc:237] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:21.842904 19997 catalog_manager.cc:5697] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 reported cstate change: term changed from 2 to 3, leader changed from a8ace9ca637044febe962414e307431b (127.19.127.194) to dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:22.241043 20101 raft_consensus.cc:1275] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Refusing update from remote peer dbe8bd74fd5d4c82960860fe361e08c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:22.242334 20391 consensus_queue.cc:1048] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:22.256863 20176 raft_consensus.cc:1275] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Refusing update from remote peer dbe8bd74fd5d4c82960860fe361e08c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:22.258244 20391 consensus_queue.cc:1048] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:22.268194 20101 raft_consensus.cc:1275] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Refusing update from remote peer dbe8bd74fd5d4c82960860fe361e08c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:22.269640 20391 consensus_queue.cc:1048] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:22.281584 20176 raft_consensus.cc:1275] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Refusing update from remote peer dbe8bd74fd5d4c82960860fe361e08c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:22.283329 20391 consensus_queue.cc:1048] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:22.434270 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:22.445119 20250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
mode: GRACEFUL
new_leader_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:51408
I20260709 13:05:22.445709 20250 raft_consensus.cc:606] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Received request to transfer leadership to TS a8ace9ca637044febe962414e307431b
I20260709 13:05:22.452267 20101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39"
dest_uuid: "c32770b892e048f8807813329e5920dd"
mode: GRACEFUL
new_leader_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:56594
I20260709 13:05:22.452818 20101 raft_consensus.cc:606] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Received request to transfer leadership to TS a8ace9ca637044febe962414e307431b
I20260709 13:05:22.459398 20101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b"
dest_uuid: "c32770b892e048f8807813329e5920dd"
mode: GRACEFUL
new_leader_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:56594
I20260709 13:05:22.459885 20101 raft_consensus.cc:606] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Received request to transfer leadership to TS a8ace9ca637044febe962414e307431b
I20260709 13:05:22.465366 20101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672"
dest_uuid: "c32770b892e048f8807813329e5920dd"
mode: GRACEFUL
new_leader_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:56594
I20260709 13:05:22.465847 20101 raft_consensus.cc:606] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Received request to transfer leadership to TS a8ace9ca637044febe962414e307431b
I20260709 13:05:22.477263 20250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
mode: GRACEFUL
new_leader_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:51408
I20260709 13:05:22.477764 20250 raft_consensus.cc:606] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Received request to transfer leadership to TS a8ace9ca637044febe962414e307431b
I20260709 13:05:22.533444 20298 raft_consensus.cc:993] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd: : Instructing follower a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.533854 20298 raft_consensus.cc:1081] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Signalling peer a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.535305 20176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b"
dest_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:54396
I20260709 13:05:22.535915 20176 raft_consensus.cc:493] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:22.536180 20176 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.540465 20176 raft_consensus.cc:515] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.541846 20176 leader_election.cc:290] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 3 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:22.542892 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:22.543352 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:22.543553 20101 raft_consensus.cc:3055] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:22.543888 20101 raft_consensus.cc:740] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:22.543898 20250 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.544870 20101 consensus_queue.cc:260] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.546212 20101 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.551301 20250 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 3.
I20260709 13:05:22.552704 20141 leader_election.cc:304] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:22.553205 20101 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 3.
I20260709 13:05:22.553879 20353 raft_consensus.cc:2804] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:22.555184 20353 raft_consensus.cc:697] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 3 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:22.556203 20353 consensus_queue.cc:237] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.562625 19998 catalog_manager.cc:5697] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 2 to 3, leader changed from c32770b892e048f8807813329e5920dd (127.19.127.193) to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 3 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.687227 20406 raft_consensus.cc:993] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8: : Instructing follower a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.687886 20391 raft_consensus.cc:1081] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Signalling peer a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.689415 20176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87"
dest_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:54412
I20260709 13:05:22.689966 20176 raft_consensus.cc:493] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:22.690333 20176 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:22.694746 20176 raft_consensus.cc:515] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.696302 20176 leader_election.cc:290] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 4 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:22.697451 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:22.698038 20101 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:22.697966 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af63f521d7948adbdbde1ca3c59de87" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:22.698834 20250 raft_consensus.cc:3055] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:22.699097 20250 raft_consensus.cc:740] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:22.699826 20250 consensus_queue.cc:260] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.700717 20250 raft_consensus.cc:3060] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:22.703188 20101 raft_consensus.cc:2468] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 4.
I20260709 13:05:22.704370 20141 leader_election.cc:304] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: 
I20260709 13:05:22.705215 20353 raft_consensus.cc:2804] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:22.705688 20353 raft_consensus.cc:697] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 4 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:22.705739 20250 raft_consensus.cc:2468] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 4.
I20260709 13:05:22.706722 20353 consensus_queue.cc:237] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.713558 19998 catalog_manager.cc:5697] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 3 to 4, leader changed from dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195) to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 4 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.723340 20298 raft_consensus.cc:993] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd: : Instructing follower a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.723827 20413 raft_consensus.cc:1081] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Signalling peer a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.725653 20176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672"
dest_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:54396
I20260709 13:05:22.726166 20176 raft_consensus.cc:493] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:22.726428 20176 raft_consensus.cc:3060] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.731326 20176 raft_consensus.cc:515] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.732933 20176 leader_election.cc:290] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:22.734004 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:22.733860 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:22.734572 20250 raft_consensus.cc:3060] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.734611 20101 raft_consensus.cc:3055] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:22.735157 20101 raft_consensus.cc:740] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:22.735998 20101 consensus_queue.cc:260] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.737214 20101 raft_consensus.cc:3060] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.742074 20250 raft_consensus.cc:2468] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:22.743396 20141 leader_election.cc:304] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:22.743866 20101 raft_consensus.cc:2468] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 2.
I20260709 13:05:22.744223 20353 raft_consensus.cc:2804] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:22.744845 20353 raft_consensus.cc:697] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:22.745605 20353 consensus_queue.cc:237] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.753103 19998 catalog_manager.cc:5697] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 1 to 2, leader changed from c32770b892e048f8807813329e5920dd (127.19.127.193) to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.879230 20406 raft_consensus.cc:993] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8: : Instructing follower a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.879733 20391 raft_consensus.cc:1081] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Signalling peer a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.881909 20176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2"
dest_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:54412
I20260709 13:05:22.882531 20176 raft_consensus.cc:493] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:22.882946 20176 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:22.887627 20176 raft_consensus.cc:515] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.889381 20176 leader_election.cc:290] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 4 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:22.890291 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:22.890632 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7696ef2f3854b77b90748592284f8c2" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:22.890951 20101 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:22.891168 20250 raft_consensus.cc:3055] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:22.891510 20250 raft_consensus.cc:740] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:22.892439 20250 consensus_queue.cc:260] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.893728 20250 raft_consensus.cc:3060] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:22.898629 20101 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 4.
I20260709 13:05:22.900336 20141 leader_election.cc:304] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, c32770b892e048f8807813329e5920dd; no voters: 
I20260709 13:05:22.901857 20353 raft_consensus.cc:2804] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:22.901880 20250 raft_consensus.cc:2468] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 4.
I20260709 13:05:22.902456 20353 raft_consensus.cc:697] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 4 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:22.903652 20353 consensus_queue.cc:237] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.911103 19998 catalog_manager.cc:5697] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 3 to 4, leader changed from dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195) to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 4 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.967478 20298 raft_consensus.cc:993] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd: : Instructing follower a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.968040 20298 raft_consensus.cc:1081] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Signalling peer a8ace9ca637044febe962414e307431b to start an election
I20260709 13:05:22.969871 20176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39"
dest_uuid: "a8ace9ca637044febe962414e307431b"
 from {username='slave'} at 127.0.0.1:54396
I20260709 13:05:22.970407 20176 raft_consensus.cc:493] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:22.970892 20176 raft_consensus.cc:3060] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.975126 20176 raft_consensus.cc:515] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.976661 20176 leader_election.cc:290] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 3 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:22.977728 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:22.978005 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344b473053b4c238def4924ee4f9e39" candidate_uuid: "a8ace9ca637044febe962414e307431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:22.978247 20101 raft_consensus.cc:3055] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:22.978554 20250 raft_consensus.cc:3060] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.978560 20101 raft_consensus.cc:740] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:22.979460 20101 consensus_queue.cc:260] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.980316 20101 raft_consensus.cc:3060] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.983234 20250 raft_consensus.cc:2468] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 3.
I20260709 13:05:22.984349 20141 leader_election.cc:304] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:22.984529 20101 raft_consensus.cc:2468] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ace9ca637044febe962414e307431b in term 3.
I20260709 13:05:22.985051 20353 raft_consensus.cc:2804] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:22.985810 20353 raft_consensus.cc:697] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 3 LEADER]: Becoming Leader. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:22.986554 20353 consensus_queue.cc:237] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:22.994213 19998 catalog_manager.cc:5697] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b reported cstate change: term changed from 2 to 3, leader changed from c32770b892e048f8807813329e5920dd (127.19.127.193) to a8ace9ca637044febe962414e307431b (127.19.127.194). New cstate: current_term: 3 leader_uuid: "a8ace9ca637044febe962414e307431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.062238 20101 raft_consensus.cc:1275] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:23.063695 20414 consensus_queue.cc:1048] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:23.075477 20250 raft_consensus.cc:1275] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:23.077977 20353 consensus_queue.cc:1048] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:23.095788 20101 raft_consensus.cc:1275] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:23.096982 20353 consensus_queue.cc:1048] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:23.104918 20250 raft_consensus.cc:1275] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:23.107479 20414 consensus_queue.cc:1048] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:23.137588 20101 raft_consensus.cc:1275] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:23.138968 20414 consensus_queue.cc:1048] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:23.148239 20250 raft_consensus.cc:1275] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:23.149616 20414 consensus_queue.cc:1048] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:23.405444 20250 raft_consensus.cc:1275] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:23.407025 20414 consensus_queue.cc:1048] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:23.416129 20101 raft_consensus.cc:1275] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:23.418956 20353 consensus_queue.cc:1048] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:23.457604 20250 raft_consensus.cc:1275] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:23.458865 20423 consensus_queue.cc:1048] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:23.467952 20101 raft_consensus.cc:1275] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Refusing update from remote peer a8ace9ca637044febe962414e307431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:23.469880 20414 consensus_queue.cc:1048] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:25.485909 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:25.494606 20176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82"
dest_uuid: "a8ace9ca637044febe962414e307431b"
mode: GRACEFUL
new_leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
 from {username='slave'} at 127.0.0.1:48388
I20260709 13:05:25.495270 20176 raft_consensus.cc:606] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Received request to transfer leadership to TS dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:25.498068 20176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353"
dest_uuid: "a8ace9ca637044febe962414e307431b"
mode: GRACEFUL
new_leader_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:48388
I20260709 13:05:25.498533 20176 raft_consensus.cc:606] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Received request to transfer leadership to TS c32770b892e048f8807813329e5920dd
I20260709 13:05:25.500890 20176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672"
dest_uuid: "a8ace9ca637044febe962414e307431b"
mode: GRACEFUL
new_leader_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:48388
I20260709 13:05:25.501282 20176 raft_consensus.cc:606] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Received request to transfer leadership to TS c32770b892e048f8807813329e5920dd
I20260709 13:05:25.503648 20176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "971df458df52428c80d3ae474c9719f6"
dest_uuid: "a8ace9ca637044febe962414e307431b"
mode: GRACEFUL
new_leader_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:48388
I20260709 13:05:25.504050 20176 raft_consensus.cc:606] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Received request to transfer leadership to TS c32770b892e048f8807813329e5920dd
I20260709 13:05:25.506489 20176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b"
dest_uuid: "a8ace9ca637044febe962414e307431b"
mode: GRACEFUL
new_leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
 from {username='slave'} at 127.0.0.1:48388
I20260709 13:05:25.506891 20176 raft_consensus.cc:606] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 3 LEADER]: Received request to transfer leadership to TS dbe8bd74fd5d4c82960860fe361e08c8
I20260709 13:05:25.508203 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:25.508980 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:25.536970 20434 raft_consensus.cc:993] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b: : Instructing follower c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:25.537429 20434 raft_consensus.cc:1081] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Signalling peer c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:25.538926 20101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "971df458df52428c80d3ae474c9719f6"
dest_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:56574
I20260709 13:05:25.539516 20101 raft_consensus.cc:493] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:25.539840 20101 raft_consensus.cc:3060] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.545001 20101 raft_consensus.cc:515] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.546927 20101 leader_election.cc:290] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 3 election: Requested vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:25.547979 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:25.548213 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "971df458df52428c80d3ae474c9719f6" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:25.548776 20176 raft_consensus.cc:3055] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:25.548825 20250 raft_consensus.cc:3060] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.549180 20176 raft_consensus.cc:740] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:25.550182 20176 consensus_queue.cc:260] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.551546 20176 raft_consensus.cc:3060] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.556388 20250 raft_consensus.cc:2468] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 3.
I20260709 13:05:25.557930 20066 leader_election.cc:304] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:25.558573 20176 raft_consensus.cc:2468] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 3.
I20260709 13:05:25.560382 20435 raft_consensus.cc:2804] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:25.561437 20435 raft_consensus.cc:697] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 3 LEADER]: Becoming Leader. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:25.562376 20435 consensus_queue.cc:237] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.570278 20414 raft_consensus.cc:993] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b: : Instructing follower c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:25.570899 20434 raft_consensus.cc:1081] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Signalling peer c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:25.570257 19997 catalog_manager.cc:5697] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd reported cstate change: term changed from 2 to 3, leader changed from a8ace9ca637044febe962414e307431b (127.19.127.194) to c32770b892e048f8807813329e5920dd (127.19.127.193). New cstate: current_term: 3 leader_uuid: "c32770b892e048f8807813329e5920dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.572791 20101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353"
dest_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:56574
I20260709 13:05:25.573410 20101 raft_consensus.cc:493] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:25.573796 20101 raft_consensus.cc:3060] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.579260 20101 raft_consensus.cc:515] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.581076 20101 leader_election.cc:290] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 3 election: Requested vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:25.581779 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:25.582151 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505e5550d19f4cc586e97ec0a240e353" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:25.582402 20176 raft_consensus.cc:3055] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:25.582723 20250 raft_consensus.cc:3060] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.582767 20176 raft_consensus.cc:740] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:25.584028 20176 consensus_queue.cc:260] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.585619 20176 raft_consensus.cc:3060] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.587598 20250 raft_consensus.cc:2468] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 3.
I20260709 13:05:25.588790 20066 leader_election.cc:304] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:25.589689 20435 raft_consensus.cc:2804] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:25.590137 20435 raft_consensus.cc:697] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 3 LEADER]: Becoming Leader. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:25.591144 20435 consensus_queue.cc:237] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.593132 20176 raft_consensus.cc:2468] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 3.
I20260709 13:05:25.599174 19997 catalog_manager.cc:5697] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd reported cstate change: term changed from 2 to 3, leader changed from a8ace9ca637044febe962414e307431b (127.19.127.194) to c32770b892e048f8807813329e5920dd (127.19.127.193). New cstate: current_term: 3 leader_uuid: "c32770b892e048f8807813329e5920dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.618417 20414 raft_consensus.cc:993] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b: : Instructing follower dbe8bd74fd5d4c82960860fe361e08c8 to start an election
I20260709 13:05:25.618904 20414 raft_consensus.cc:1081] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 3 LEADER]: Signalling peer dbe8bd74fd5d4c82960860fe361e08c8 to start an election
I20260709 13:05:25.620541 20250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
 from {username='slave'} at 127.0.0.1:51392
I20260709 13:05:25.621065 20250 raft_consensus.cc:493] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:25.621337 20250 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:25.625411 20250 raft_consensus.cc:515] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.626854 20250 leader_election.cc:290] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 4 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:25.628149 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:25.628149 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd6cee70351442cb52711055c4ed04b" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:25.628954 20176 raft_consensus.cc:3055] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:25.628954 20101 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:25.629413 20176 raft_consensus.cc:740] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:25.630349 20176 consensus_queue.cc:260] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.631551 20176 raft_consensus.cc:3060] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:25.635710 20101 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 4.
I20260709 13:05:25.637174 20215 leader_election.cc:304] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:25.638145 20176 raft_consensus.cc:2468] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 4.
I20260709 13:05:25.638198 20391 raft_consensus.cc:2804] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:25.639554 20391 raft_consensus.cc:697] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 LEADER]: Becoming Leader. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:25.640515 20391 consensus_queue.cc:237] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.647735 19997 catalog_manager.cc:5697] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 reported cstate change: term changed from 3 to 4, leader changed from a8ace9ca637044febe962414e307431b (127.19.127.194) to dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195). New cstate: current_term: 4 leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.762040 20434 raft_consensus.cc:993] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b: : Instructing follower dbe8bd74fd5d4c82960860fe361e08c8 to start an election
I20260709 13:05:25.762526 20434 raft_consensus.cc:1081] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Signalling peer dbe8bd74fd5d4c82960860fe361e08c8 to start an election
I20260709 13:05:25.764061 20250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82"
dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
 from {username='slave'} at 127.0.0.1:51392
I20260709 13:05:25.764619 20250 raft_consensus.cc:493] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:25.764886 20250 raft_consensus.cc:3060] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.770476 20250 raft_consensus.cc:515] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.773154 20250 leader_election.cc:290] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 3 election: Requested vote from peers c32770b892e048f8807813329e5920dd (127.19.127.193:33283), a8ace9ca637044febe962414e307431b (127.19.127.194:32971)
I20260709 13:05:25.774317 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:25.774920 20176 raft_consensus.cc:3055] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:25.775235 20176 raft_consensus.cc:740] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:25.775990 20176 consensus_queue.cc:260] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.777242 20176 raft_consensus.cc:3060] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.779822 20101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b177a5f2c744fbd984baea329a28b82" candidate_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c32770b892e048f8807813329e5920dd"
I20260709 13:05:25.780364 20101 raft_consensus.cc:3060] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.784046 20176 raft_consensus.cc:2468] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 3.
I20260709 13:05:25.785229 20214 leader_election.cc:304] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ace9ca637044febe962414e307431b, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:25.786190 20391 raft_consensus.cc:2804] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:25.786674 20101 raft_consensus.cc:2468] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe8bd74fd5d4c82960860fe361e08c8 in term 3.
I20260709 13:05:25.786731 20391 raft_consensus.cc:697] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Becoming Leader. State: Replica: dbe8bd74fd5d4c82960860fe361e08c8, State: Running, Role: LEADER
I20260709 13:05:25.787931 20391 consensus_queue.cc:237] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.794478 19997 catalog_manager.cc:5697] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 reported cstate change: term changed from 2 to 3, leader changed from a8ace9ca637044febe962414e307431b (127.19.127.194) to dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.823344 20414 raft_consensus.cc:993] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b: : Instructing follower c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:25.823818 20414 raft_consensus.cc:1081] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Signalling peer c32770b892e048f8807813329e5920dd to start an election
I20260709 13:05:25.825268 20101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672"
dest_uuid: "c32770b892e048f8807813329e5920dd"
 from {username='slave'} at 127.0.0.1:56574
I20260709 13:05:25.825785 20101 raft_consensus.cc:493] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:25.826078 20101 raft_consensus.cc:3060] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.830416 20101 raft_consensus.cc:515] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.832791 20101 leader_election.cc:290] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 3 election: Requested vote from peers a8ace9ca637044febe962414e307431b (127.19.127.194:32971), dbe8bd74fd5d4c82960860fe361e08c8 (127.19.127.195:33539)
I20260709 13:05:25.833809 20250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dbe8bd74fd5d4c82960860fe361e08c8"
I20260709 13:05:25.833694 20176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cd6e6d5cb5040cd848633cbc13f3672" candidate_uuid: "c32770b892e048f8807813329e5920dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a8ace9ca637044febe962414e307431b"
I20260709 13:05:25.834334 20250 raft_consensus.cc:3060] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.834383 20176 raft_consensus.cc:3055] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:25.834824 20176 raft_consensus.cc:740] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a8ace9ca637044febe962414e307431b, State: Running, Role: LEADER
I20260709 13:05:25.835661 20176 consensus_queue.cc:260] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.836783 20176 raft_consensus.cc:3060] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.840956 20250 raft_consensus.cc:2468] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 3.
I20260709 13:05:25.842024 20066 leader_election.cc:304] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32770b892e048f8807813329e5920dd, dbe8bd74fd5d4c82960860fe361e08c8; no voters: 
I20260709 13:05:25.842892 20435 raft_consensus.cc:2804] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:25.843137 20176 raft_consensus.cc:2468] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32770b892e048f8807813329e5920dd in term 3.
I20260709 13:05:25.843451 20435 raft_consensus.cc:697] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 3 LEADER]: Becoming Leader. State: Replica: c32770b892e048f8807813329e5920dd, State: Running, Role: LEADER
I20260709 13:05:25.844376 20435 consensus_queue.cc:237] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } }
I20260709 13:05:25.851222 19997 catalog_manager.cc:5697] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd reported cstate change: term changed from 2 to 3, leader changed from a8ace9ca637044febe962414e307431b (127.19.127.194) to c32770b892e048f8807813329e5920dd (127.19.127.193). New cstate: current_term: 3 leader_uuid: "c32770b892e048f8807813329e5920dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.958041 20250 raft_consensus.cc:1275] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:25.959318 20435 consensus_queue.cc:1048] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:25.970599 20176 raft_consensus.cc:1275] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:25.972700 20450 consensus_queue.cc:1048] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:26.046271 20176 raft_consensus.cc:1275] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:26.047717 20450 consensus_queue.cc:1048] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:26.050961 20250 raft_consensus.cc:1275] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:26.052626 20450 consensus_queue.cc:1048] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:26.190797 20101 raft_consensus.cc:1275] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Refusing update from remote peer dbe8bd74fd5d4c82960860fe361e08c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:26.192209 20391 consensus_queue.cc:1048] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:26.202332 20176 raft_consensus.cc:1275] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Refusing update from remote peer dbe8bd74fd5d4c82960860fe361e08c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:26.203866 20391 consensus_queue.cc:1048] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:26.258383 20101 raft_consensus.cc:1275] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Refusing update from remote peer dbe8bd74fd5d4c82960860fe361e08c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:26.260030 20456 consensus_queue.cc:1048] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32770b892e048f8807813329e5920dd" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:26.271101 20176 raft_consensus.cc:1275] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Refusing update from remote peer dbe8bd74fd5d4c82960860fe361e08c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:26.273008 20443 consensus_queue.cc:1048] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:26.371011 20176 raft_consensus.cc:1275] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:26.372488 20450 consensus_queue.cc:1048] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ace9ca637044febe962414e307431b" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:26.381134 20250 raft_consensus.cc:1275] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Refusing update from remote peer c32770b892e048f8807813329e5920dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:26.382668 20435 consensus_queue.cc:1048] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe8bd74fd5d4c82960860fe361e08c8" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:26.510198 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:26.515769 19967 tablet_server.cc:179] TabletServer@127.19.127.193:0 shutting down...
I20260709 13:05:26.544034 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:26.544804 19967 tablet_replica.cc:333] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd: stopping tablet replica
I20260709 13:05:26.545423 19967 raft_consensus.cc:2243] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:26.546269 19967 raft_consensus.cc:2272] T 971df458df52428c80d3ae474c9719f6 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.548872 19967 tablet_replica.cc:333] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd: stopping tablet replica
I20260709 13:05:26.549437 19967 raft_consensus.cc:2243] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.549908 19967 raft_consensus.cc:2272] T b7696ef2f3854b77b90748592284f8c2 P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.551652 19967 tablet_replica.cc:333] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd: stopping tablet replica
I20260709 13:05:26.552177 19967 raft_consensus.cc:2243] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.552578 19967 raft_consensus.cc:2272] T 1b177a5f2c744fbd984baea329a28b82 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.554929 19967 tablet_replica.cc:333] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd: stopping tablet replica
I20260709 13:05:26.555516 19967 raft_consensus.cc:2243] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:26.556212 19967 raft_consensus.cc:2272] T 505e5550d19f4cc586e97ec0a240e353 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.558362 19967 tablet_replica.cc:333] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd: stopping tablet replica
I20260709 13:05:26.558846 19967 raft_consensus.cc:2243] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.559258 19967 raft_consensus.cc:2272] T b344b473053b4c238def4924ee4f9e39 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.560940 19967 tablet_replica.cc:333] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd: stopping tablet replica
I20260709 13:05:26.561446 19967 raft_consensus.cc:2243] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:26.562162 19967 raft_consensus.cc:2272] T 8cd6e6d5cb5040cd848633cbc13f3672 P c32770b892e048f8807813329e5920dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.564155 19967 tablet_replica.cc:333] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd: stopping tablet replica
I20260709 13:05:26.564649 19967 raft_consensus.cc:2243] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.565135 19967 raft_consensus.cc:2272] T 5af63f521d7948adbdbde1ca3c59de87 P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.566741 19967 tablet_replica.cc:333] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd: stopping tablet replica
I20260709 13:05:26.567195 19967 raft_consensus.cc:2243] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.567631 19967 raft_consensus.cc:2272] T 6716983c2b104bdd8e141a5247695682 P c32770b892e048f8807813329e5920dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.569167 19967 tablet_replica.cc:333] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd: stopping tablet replica
I20260709 13:05:26.569593 19967 raft_consensus.cc:2243] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.569989 19967 raft_consensus.cc:2272] T bbd6cee70351442cb52711055c4ed04b P c32770b892e048f8807813329e5920dd [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:26.677415 20215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.127.193:33283: connect: Connection refused (error 111)
I20260709 13:05:26.681325 19967 tablet_server.cc:196] TabletServer@127.19.127.193:0 shutdown complete.
W20260709 13:05:26.681758 20141 consensus_peers.cc:597] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b -> Peer c32770b892e048f8807813329e5920dd (127.19.127.193:33283): Couldn't send request to peer c32770b892e048f8807813329e5920dd. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:33283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:26.681934 20215 consensus_peers.cc:597] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 -> Peer c32770b892e048f8807813329e5920dd (127.19.127.193:33283): Couldn't send request to peer c32770b892e048f8807813329e5920dd. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:33283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.702286 19967 tablet_server.cc:179] TabletServer@127.19.127.194:0 shutting down...
W20260709 13:05:26.708701 20214 consensus_peers.cc:597] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 -> Peer a8ace9ca637044febe962414e307431b (127.19.127.194:32971): Couldn't send request to peer a8ace9ca637044febe962414e307431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:26.724609 20215 consensus_peers.cc:597] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 -> Peer c32770b892e048f8807813329e5920dd (127.19.127.193:33283): Couldn't send request to peer c32770b892e048f8807813329e5920dd. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:33283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.737007 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:26.737653 19967 tablet_replica.cc:333] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b: stopping tablet replica
I20260709 13:05:26.738221 19967 raft_consensus.cc:2243] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.738672 19967 raft_consensus.cc:2272] T 971df458df52428c80d3ae474c9719f6 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.741698 19967 tablet_replica.cc:333] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b: stopping tablet replica
I20260709 13:05:26.742251 19967 raft_consensus.cc:2243] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.742664 19967 raft_consensus.cc:2272] T 6716983c2b104bdd8e141a5247695682 P a8ace9ca637044febe962414e307431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.744608 19967 tablet_replica.cc:333] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b: stopping tablet replica
I20260709 13:05:26.745149 19967 raft_consensus.cc:2243] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:05:26.745939 19967 raft_consensus.cc:2272] T 5af63f521d7948adbdbde1ca3c59de87 P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.747843 19967 tablet_replica.cc:333] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b: stopping tablet replica
I20260709 13:05:26.748356 19967 raft_consensus.cc:2243] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.748746 19967 raft_consensus.cc:2272] T 505e5550d19f4cc586e97ec0a240e353 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.751264 19967 tablet_replica.cc:333] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b: stopping tablet replica
I20260709 13:05:26.751894 19967 raft_consensus.cc:2243] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.752328 19967 raft_consensus.cc:2272] T 8cd6e6d5cb5040cd848633cbc13f3672 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.754407 19967 tablet_replica.cc:333] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b: stopping tablet replica
I20260709 13:05:26.754974 19967 raft_consensus.cc:2243] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:26.755811 19967 raft_consensus.cc:2272] T b344b473053b4c238def4924ee4f9e39 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.757568 19967 tablet_replica.cc:333] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b: stopping tablet replica
I20260709 13:05:26.758064 19967 raft_consensus.cc:2243] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.758486 19967 raft_consensus.cc:2272] T 1b177a5f2c744fbd984baea329a28b82 P a8ace9ca637044febe962414e307431b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.760555 19967 tablet_replica.cc:333] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b: stopping tablet replica
I20260709 13:05:26.761039 19967 raft_consensus.cc:2243] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:05:26.761781 19967 raft_consensus.cc:2272] T b7696ef2f3854b77b90748592284f8c2 P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.763414 19967 tablet_replica.cc:333] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b: stopping tablet replica
I20260709 13:05:26.763849 19967 raft_consensus.cc:2243] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.764248 19967 raft_consensus.cc:2272] T bbd6cee70351442cb52711055c4ed04b P a8ace9ca637044febe962414e307431b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.796877 19967 tablet_server.cc:196] TabletServer@127.19.127.194:0 shutdown complete.
I20260709 13:05:26.816479 19967 tablet_server.cc:179] TabletServer@127.19.127.195:0 shutting down...
I20260709 13:05:26.842803 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:26.843737 19967 tablet_replica.cc:333] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8: stopping tablet replica
I20260709 13:05:26.844446 19967 raft_consensus.cc:2243] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.844907 19967 raft_consensus.cc:2272] T 505e5550d19f4cc586e97ec0a240e353 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.847602 19967 tablet_replica.cc:333] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8: stopping tablet replica
I20260709 13:05:26.848204 19967 raft_consensus.cc:2243] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.848654 19967 raft_consensus.cc:2272] T b7696ef2f3854b77b90748592284f8c2 P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.850749 19967 tablet_replica.cc:333] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8: stopping tablet replica
I20260709 13:05:26.851322 19967 raft_consensus.cc:2243] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:26.852031 19967 raft_consensus.cc:2272] T 1b177a5f2c744fbd984baea329a28b82 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.854480 19967 tablet_replica.cc:333] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8: stopping tablet replica
W20260709 13:05:26.854790 20215 consensus_peers.cc:597] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 -> Peer c32770b892e048f8807813329e5920dd (127.19.127.193:33283): Couldn't send request to peer c32770b892e048f8807813329e5920dd. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:33283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.855167 19967 raft_consensus.cc:2243] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:26.855991 19967 raft_consensus.cc:2272] T 6716983c2b104bdd8e141a5247695682 P dbe8bd74fd5d4c82960860fe361e08c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.857757 19967 tablet_replica.cc:333] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8: stopping tablet replica
I20260709 13:05:26.858302 19967 raft_consensus.cc:2243] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.858722 19967 raft_consensus.cc:2272] T 8cd6e6d5cb5040cd848633cbc13f3672 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.860954 19967 tablet_replica.cc:333] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8: stopping tablet replica
I20260709 13:05:26.861426 19967 raft_consensus.cc:2243] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.861819 19967 raft_consensus.cc:2272] T b344b473053b4c238def4924ee4f9e39 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.863399 19967 tablet_replica.cc:333] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8: stopping tablet replica
I20260709 13:05:26.863822 19967 raft_consensus.cc:2243] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.864171 19967 raft_consensus.cc:2272] T 5af63f521d7948adbdbde1ca3c59de87 P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.865721 19967 tablet_replica.cc:333] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8: stopping tablet replica
I20260709 13:05:26.866111 19967 raft_consensus.cc:2243] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:26.866465 19967 raft_consensus.cc:2272] T 971df458df52428c80d3ae474c9719f6 P dbe8bd74fd5d4c82960860fe361e08c8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.868474 19967 tablet_replica.cc:333] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8: stopping tablet replica
I20260709 13:05:26.868889 19967 raft_consensus.cc:2243] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:05:26.869552 19967 raft_consensus.cc:2272] T bbd6cee70351442cb52711055c4ed04b P dbe8bd74fd5d4c82960860fe361e08c8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:26.896718 19967 tablet_server.cc:196] TabletServer@127.19.127.195:0 shutdown complete.
I20260709 13:05:26.913522 19967 master.cc:562] Master@127.19.127.254:36733 shutting down...
I20260709 13:05:27.017643 20054 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:05:27.067883 19967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.068452 19967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.068800 19967 tablet_replica.cc:333] T 00000000000000000000000000000000 P c00e2376160448cdad4a98426ee71b42: stopping tablet replica
I20260709 13:05:27.088539 19967 master.cc:584] Master@127.19.127.254:36733 shutdown complete.
a8ace9ca637044febe962414e307431b  4
c32770b892e048f8807813329e5920dd  4
dbe8bd74fd5d4c82960860fe361e08c8  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12710 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:05:27.133247 19967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.127.254:33007
I20260709 13:05:27.134495 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:27.140861 20466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:27.140954 20467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:27.142020 20469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:27.143733 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:27.144568 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:27.144846 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:27.145058 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602327145022 us; error 0 us; skew 500 ppm
I20260709 13:05:27.145623 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:27.148295 19967 webserver.cc:533] Webserver started at http://127.19.127.254:45871/ using document root <none> and password file <none>
I20260709 13:05:27.148842 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:27.149044 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:27.149344 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:27.150486 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/master-0-root/instance:
uuid: "8cfa489b35414a3f80aa1415ec73685b"
format_stamp: "Formatted at 2026-07-09 13:05:27 on dist-test-slave-s3n6"
I20260709 13:05:27.155167 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:05:27.159015 20474 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.159855 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:05:27.160161 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/master-0-root
uuid: "8cfa489b35414a3f80aa1415ec73685b"
format_stamp: "Formatted at 2026-07-09 13:05:27 on dist-test-slave-s3n6"
I20260709 13:05:27.160457 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:27.174404 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:27.175776 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:27.214915 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.254:33007
I20260709 13:05:27.215013 20525 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.254:33007 every 8 connection(s)
I20260709 13:05:27.218927 20526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.228410 20526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b: Bootstrap starting.
I20260709 13:05:27.232947 20526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.238050 20526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b: No bootstrap required, opened a new log
I20260709 13:05:27.240223 20526 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfa489b35414a3f80aa1415ec73685b" member_type: VOTER }
I20260709 13:05:27.240605 20526 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.240801 20526 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfa489b35414a3f80aa1415ec73685b, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.241305 20526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfa489b35414a3f80aa1415ec73685b" member_type: VOTER }
I20260709 13:05:27.241700 20526 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:27.241888 20526 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:27.242095 20526 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.246188 20526 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfa489b35414a3f80aa1415ec73685b" member_type: VOTER }
I20260709 13:05:27.246724 20526 leader_election.cc:304] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cfa489b35414a3f80aa1415ec73685b; no voters: 
I20260709 13:05:27.247869 20526 leader_election.cc:290] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:27.248200 20529 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:27.249480 20529 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 1 LEADER]: Becoming Leader. State: Replica: 8cfa489b35414a3f80aa1415ec73685b, State: Running, Role: LEADER
I20260709 13:05:27.250195 20529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfa489b35414a3f80aa1415ec73685b" member_type: VOTER }
I20260709 13:05:27.250844 20526 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:27.254947 20530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cfa489b35414a3f80aa1415ec73685b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfa489b35414a3f80aa1415ec73685b" member_type: VOTER } }
I20260709 13:05:27.255739 20530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:27.255920 20531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cfa489b35414a3f80aa1415ec73685b. Latest consensus state: current_term: 1 leader_uuid: "8cfa489b35414a3f80aa1415ec73685b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfa489b35414a3f80aa1415ec73685b" member_type: VOTER } }
I20260709 13:05:27.257190 20531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:27.261021 20536 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:27.270565 20536 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:27.271700 19967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:27.280748 20536 catalog_manager.cc:1383] Generated new cluster ID: e3402fd3ad704fb99734990a7bb8eb83
I20260709 13:05:27.281304 20536 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:27.304638 20536 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:27.305987 20536 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:27.322393 20536 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b: Generated new TSK 0
I20260709 13:05:27.323242 20536 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:27.338282 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:27.344748 20547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:27.345237 20548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:27.347229 20550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:27.347803 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:27.348510 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:27.348698 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:27.348824 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602327348803 us; error 0 us; skew 500 ppm
I20260709 13:05:27.349283 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:27.351634 19967 webserver.cc:533] Webserver started at http://127.19.127.193:44809/ using document root <none> and password file <none>
I20260709 13:05:27.352252 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:27.352417 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:27.352620 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:27.353646 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-0-root/instance:
uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
format_stamp: "Formatted at 2026-07-09 13:05:27 on dist-test-slave-s3n6"
I20260709 13:05:27.358172 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 13:05:27.361537 20555 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.362481 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:05:27.362870 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
format_stamp: "Formatted at 2026-07-09 13:05:27 on dist-test-slave-s3n6"
I20260709 13:05:27.363178 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:27.391933 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:27.393275 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:27.394866 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:27.397233 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:27.397476 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.397739 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:27.397934 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.441568 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:33571
I20260709 13:05:27.441706 20617 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:33571 every 8 connection(s)
I20260709 13:05:27.446671 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:27.455545 20622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:27.455718 20623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:27.459045 20625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:27.459458 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:27.460425 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:27.460659 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:27.460845 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602327460827 us; error 0 us; skew 500 ppm
I20260709 13:05:27.461521 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:27.464569 19967 webserver.cc:533] Webserver started at http://127.19.127.194:39421/ using document root <none> and password file <none>
I20260709 13:05:27.465121 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:27.465343 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:27.465631 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:27.466536 20618 heartbeater.cc:344] Connected to a master server at 127.19.127.254:33007
I20260709 13:05:27.466972 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-1-root/instance:
uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
format_stamp: "Formatted at 2026-07-09 13:05:27 on dist-test-slave-s3n6"
I20260709 13:05:27.467072 20618 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:27.467959 20618 heartbeater.cc:507] Master 127.19.127.254:33007 requested a full tablet report, sending...
I20260709 13:05:27.470536 20491 ts_manager.cc:194] Registered new tserver with Master: a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571)
I20260709 13:05:27.472179 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:05:27.472539 20491 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50624
I20260709 13:05:27.475857 20630 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.476650 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:05:27.476969 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-1-root
uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
format_stamp: "Formatted at 2026-07-09 13:05:27 on dist-test-slave-s3n6"
I20260709 13:05:27.477270 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:27.525449 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:27.526938 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:27.528757 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:27.531673 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:27.531932 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.532197 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:27.532399 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.581974 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.194:42733
I20260709 13:05:27.582086 20692 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.194:42733 every 8 connection(s)
I20260709 13:05:27.587370 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:27.596575 20696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:27.598875 20697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:27.602905 20699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:27.603590 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:27.603567 20693 heartbeater.cc:344] Connected to a master server at 127.19.127.254:33007
I20260709 13:05:27.604271 20693 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:27.604772 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:27.604979 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:27.605085 20693 heartbeater.cc:507] Master 127.19.127.254:33007 requested a full tablet report, sending...
I20260709 13:05:27.605242 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602327605215 us; error 0 us; skew 500 ppm
I20260709 13:05:27.605904 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:27.607522 20491 ts_manager.cc:194] Registered new tserver with Master: 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:27.608639 19967 webserver.cc:533] Webserver started at http://127.19.127.195:38597/ using document root <none> and password file <none>
I20260709 13:05:27.609058 20491 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50632
I20260709 13:05:27.609155 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:27.609406 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:27.609710 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:27.610886 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-2-root/instance:
uuid: "956d2eb6cb0a422998647448946aae00"
format_stamp: "Formatted at 2026-07-09 13:05:27 on dist-test-slave-s3n6"
I20260709 13:05:27.615643 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:05:27.619246 20704 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.620535 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:05:27.620911 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-2-root
uuid: "956d2eb6cb0a422998647448946aae00"
format_stamp: "Formatted at 2026-07-09 13:05:27 on dist-test-slave-s3n6"
I20260709 13:05:27.621251 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602314247669-19967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:27.639935 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:27.641299 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:27.642946 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:27.645459 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:27.645711 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.645977 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:27.646198 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:27.694327 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.195:34945
I20260709 13:05:27.694450 20766 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.195:34945 every 8 connection(s)
I20260709 13:05:27.710124 20767 heartbeater.cc:344] Connected to a master server at 127.19.127.254:33007
I20260709 13:05:27.710644 20767 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:27.711481 20767 heartbeater.cc:507] Master 127.19.127.254:33007 requested a full tablet report, sending...
I20260709 13:05:27.713909 20491 ts_manager.cc:194] Registered new tserver with Master: 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:27.714788 19967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017233211s
I20260709 13:05:27.715386 20491 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50634
I20260709 13:05:27.743914 20491 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50642:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:27.746848 20491 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:27.812556 20658 tablet_service.cc:1511] Processing CreateTablet for tablet 17172d3dd9d0431ab73310e11af8a823 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:05:27.814030 20658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17172d3dd9d0431ab73310e11af8a823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.814643 20656 tablet_service.cc:1511] Processing CreateTablet for tablet 0c8ac689b5ca413993647d45b5fbfd17 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:05:27.816408 20657 tablet_service.cc:1511] Processing CreateTablet for tablet a9f33b8bce6a478cb6d1414801695d14 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:05:27.820453 20655 tablet_service.cc:1511] Processing CreateTablet for tablet fe2b9bdf278a40569029fb629711e517 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:05:27.821943 20655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe2b9bdf278a40569029fb629711e517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.825450 20654 tablet_service.cc:1511] Processing CreateTablet for tablet a75a764e8e6f4b20ac14649f9a94c543 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:05:27.829885 20653 tablet_service.cc:1511] Processing CreateTablet for tablet d8334cae58bb4bf8872f48c6311c273a (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:05:27.831456 20657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f33b8bce6a478cb6d1414801695d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.833042 20654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a75a764e8e6f4b20ac14649f9a94c543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.833848 20656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8ac689b5ca413993647d45b5fbfd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.834573 20653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8334cae58bb4bf8872f48c6311c273a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.845191 20652 tablet_service.cc:1511] Processing CreateTablet for tablet d20e3ddf20eb4be997bddfa2d06bb90a (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:05:27.847246 20652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d20e3ddf20eb4be997bddfa2d06bb90a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.854287 20651 tablet_service.cc:1511] Processing CreateTablet for tablet a71cd8bd222646d3995a43f62d226b28 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:05:27.856130 20651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71cd8bd222646d3995a43f62d226b28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.864074 20650 tablet_service.cc:1511] Processing CreateTablet for tablet 4b79b8b205834585a55c92a7a0eaf03e (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:05:27.865527 20650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b79b8b205834585a55c92a7a0eaf03e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.870496 20583 tablet_service.cc:1511] Processing CreateTablet for tablet 17172d3dd9d0431ab73310e11af8a823 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:05:27.872072 20583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17172d3dd9d0431ab73310e11af8a823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.871698 20580 tablet_service.cc:1511] Processing CreateTablet for tablet fe2b9bdf278a40569029fb629711e517 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:05:27.873165 20580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe2b9bdf278a40569029fb629711e517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.873092 20582 tablet_service.cc:1511] Processing CreateTablet for tablet a9f33b8bce6a478cb6d1414801695d14 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:05:27.874567 20582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f33b8bce6a478cb6d1414801695d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.876552 20581 tablet_service.cc:1511] Processing CreateTablet for tablet 0c8ac689b5ca413993647d45b5fbfd17 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:05:27.878116 20581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8ac689b5ca413993647d45b5fbfd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.880496 20579 tablet_service.cc:1511] Processing CreateTablet for tablet a75a764e8e6f4b20ac14649f9a94c543 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:05:27.881898 20579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a75a764e8e6f4b20ac14649f9a94c543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.884553 20578 tablet_service.cc:1511] Processing CreateTablet for tablet d8334cae58bb4bf8872f48c6311c273a (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:05:27.885947 20578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8334cae58bb4bf8872f48c6311c273a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.888506 20577 tablet_service.cc:1511] Processing CreateTablet for tablet d20e3ddf20eb4be997bddfa2d06bb90a (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:05:27.889906 20577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d20e3ddf20eb4be997bddfa2d06bb90a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.891410 20732 tablet_service.cc:1511] Processing CreateTablet for tablet 17172d3dd9d0431ab73310e11af8a823 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:05:27.892953 20732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17172d3dd9d0431ab73310e11af8a823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.896490 20576 tablet_service.cc:1511] Processing CreateTablet for tablet a71cd8bd222646d3995a43f62d226b28 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:05:27.897967 20576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71cd8bd222646d3995a43f62d226b28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.901139 20731 tablet_service.cc:1511] Processing CreateTablet for tablet a9f33b8bce6a478cb6d1414801695d14 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:05:27.902627 20731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f33b8bce6a478cb6d1414801695d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.902818 20574 tablet_service.cc:1511] Processing CreateTablet for tablet 0b972ae2bc69426aa33cd855b1ae4c1f (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:05:27.901139 20575 tablet_service.cc:1511] Processing CreateTablet for tablet 4b79b8b205834585a55c92a7a0eaf03e (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:05:27.906864 20574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b972ae2bc69426aa33cd855b1ae4c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.907624 20575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b79b8b205834585a55c92a7a0eaf03e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.907992 20781 tablet_bootstrap.cc:492] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:27.921340 20730 tablet_service.cc:1511] Processing CreateTablet for tablet 0c8ac689b5ca413993647d45b5fbfd17 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:05:27.922863 20730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8ac689b5ca413993647d45b5fbfd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.934386 20729 tablet_service.cc:1511] Processing CreateTablet for tablet fe2b9bdf278a40569029fb629711e517 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:05:27.934700 20727 tablet_service.cc:1511] Processing CreateTablet for tablet d8334cae58bb4bf8872f48c6311c273a (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:05:27.935989 20729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe2b9bdf278a40569029fb629711e517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.936661 20723 tablet_service.cc:1511] Processing CreateTablet for tablet 0b972ae2bc69426aa33cd855b1ae4c1f (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:05:27.937207 20728 tablet_service.cc:1511] Processing CreateTablet for tablet a75a764e8e6f4b20ac14649f9a94c543 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:05:27.938786 20723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b972ae2bc69426aa33cd855b1ae4c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.939402 20726 tablet_service.cc:1511] Processing CreateTablet for tablet d20e3ddf20eb4be997bddfa2d06bb90a (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:05:27.939952 20728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a75a764e8e6f4b20ac14649f9a94c543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.940680 20726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d20e3ddf20eb4be997bddfa2d06bb90a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.941479 20727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8334cae58bb4bf8872f48c6311c273a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.942808 20725 tablet_service.cc:1511] Processing CreateTablet for tablet a71cd8bd222646d3995a43f62d226b28 (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:05:27.935966 20656 tablet_service.cc:1511] Processing CreateTablet for tablet 0b972ae2bc69426aa33cd855b1ae4c1f (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:05:27.944180 20725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71cd8bd222646d3995a43f62d226b28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.944654 20656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b972ae2bc69426aa33cd855b1ae4c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.948295 20724 tablet_service.cc:1511] Processing CreateTablet for tablet 4b79b8b205834585a55c92a7a0eaf03e (DEFAULT_TABLE table=test-workload [id=27e9298154454aa09921d3fbc2bc190a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:05:27.949796 20724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b79b8b205834585a55c92a7a0eaf03e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:27.963217 20781 tablet_bootstrap.cc:654] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.999425 20782 tablet_bootstrap.cc:492] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.006220 20782 tablet_bootstrap.cc:654] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.038518 20783 tablet_bootstrap.cc:492] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.044801 20782 tablet_bootstrap.cc:492] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.045306 20782 ts_tablet_manager.cc:1403] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.046s	user 0.004s	sys 0.024s
I20260709 13:05:28.040334 20781 tablet_bootstrap.cc:492] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
I20260709 13:05:28.045900 20781 ts_tablet_manager.cc:1403] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.138s	user 0.015s	sys 0.008s
I20260709 13:05:28.048378 20782 raft_consensus.cc:359] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.049118 20782 raft_consensus.cc:385] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.049420 20782 raft_consensus.cc:740] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.049602 20781 raft_consensus.cc:359] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.050343 20781 raft_consensus.cc:385] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.050786 20781 raft_consensus.cc:740] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.050343 20782 consensus_queue.cc:260] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.051533 20781 consensus_queue.cc:260] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.054811 20783 tablet_bootstrap.cc:654] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.057265 20618 heartbeater.cc:499] Master 127.19.127.254:33007 was elected leader, sending a full tablet report...
I20260709 13:05:28.058183 20782 ts_tablet_manager.cc:1434] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.013s	user 0.012s	sys 0.001s
I20260709 13:05:28.059466 20782 tablet_bootstrap.cc:492] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.069999 20782 tablet_bootstrap.cc:654] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.081403 20693 heartbeater.cc:499] Master 127.19.127.254:33007 was elected leader, sending a full tablet report...
I20260709 13:05:28.081809 20781 ts_tablet_manager.cc:1434] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.022s
I20260709 13:05:28.082846 20781 tablet_bootstrap.cc:492] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:28.083534 20782 tablet_bootstrap.cc:492] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.084025 20782 ts_tablet_manager.cc:1403] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.013s
I20260709 13:05:28.086994 20782 raft_consensus.cc:359] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.087881 20782 raft_consensus.cc:385] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.088187 20782 raft_consensus.cc:740] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.089071 20782 consensus_queue.cc:260] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.090287 20781 tablet_bootstrap.cc:654] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.091617 20782 ts_tablet_manager.cc:1434] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:05:28.092661 20782 tablet_bootstrap.cc:492] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.101553 20782 tablet_bootstrap.cc:654] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.102651 20783 tablet_bootstrap.cc:492] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.103181 20783 ts_tablet_manager.cc:1403] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.017s
I20260709 13:05:28.106333 20781 tablet_bootstrap.cc:492] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
I20260709 13:05:28.106472 20783 raft_consensus.cc:359] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.107051 20781 ts_tablet_manager.cc:1403] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.010s
I20260709 13:05:28.107353 20783 raft_consensus.cc:385] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.107722 20783 raft_consensus.cc:740] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.108680 20783 consensus_queue.cc:260] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.111158 20781 raft_consensus.cc:359] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.112087 20781 raft_consensus.cc:385] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.112918 20767 heartbeater.cc:499] Master 127.19.127.254:33007 was elected leader, sending a full tablet report...
I20260709 13:05:28.113788 20783 ts_tablet_manager.cc:1434] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:05:28.112447 20781 raft_consensus.cc:740] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.114878 20783 tablet_bootstrap.cc:492] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.116675 20782 tablet_bootstrap.cc:492] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.117165 20782 ts_tablet_manager.cc:1403] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.002s
I20260709 13:05:28.120419 20782 raft_consensus.cc:359] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.114884 20781 consensus_queue.cc:260] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.121881 20783 tablet_bootstrap.cc:654] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.122225 20782 raft_consensus.cc:385] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.122622 20782 raft_consensus.cc:740] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.123958 20781 ts_tablet_manager.cc:1434] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:05:28.123662 20782 consensus_queue.cc:260] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.125078 20781 tablet_bootstrap.cc:492] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:28.126143 20782 ts_tablet_manager.cc:1434] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:05:28.127267 20782 tablet_bootstrap.cc:492] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.133569 20783 tablet_bootstrap.cc:492] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.134150 20783 ts_tablet_manager.cc:1403] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.011s
I20260709 13:05:28.135219 20782 tablet_bootstrap.cc:654] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.137302 20783 raft_consensus.cc:359] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.138079 20783 raft_consensus.cc:385] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.138396 20783 raft_consensus.cc:740] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.133589 20781 tablet_bootstrap.cc:654] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.142670 20783 consensus_queue.cc:260] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.145221 20783 ts_tablet_manager.cc:1434] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:05:28.145910 20782 tablet_bootstrap.cc:492] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.146175 20783 tablet_bootstrap.cc:492] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.146574 20782 ts_tablet_manager.cc:1403] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.003s
I20260709 13:05:28.150804 20782 raft_consensus.cc:359] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.152086 20782 raft_consensus.cc:385] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.152486 20782 raft_consensus.cc:740] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.153477 20783 tablet_bootstrap.cc:654] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.153563 20782 consensus_queue.cc:260] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.162002 20782 ts_tablet_manager.cc:1434] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.005s
I20260709 13:05:28.163434 20782 tablet_bootstrap.cc:492] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.166749 20783 tablet_bootstrap.cc:492] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.167392 20783 ts_tablet_manager.cc:1403] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.021s	user 0.003s	sys 0.017s
I20260709 13:05:28.169461 20781 tablet_bootstrap.cc:492] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
I20260709 13:05:28.169956 20781 ts_tablet_manager.cc:1403] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.023s
I20260709 13:05:28.170763 20782 tablet_bootstrap.cc:654] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.170645 20783 raft_consensus.cc:359] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.171456 20783 raft_consensus.cc:385] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.171819 20783 raft_consensus.cc:740] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.172760 20781 raft_consensus.cc:359] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.172600 20783 consensus_queue.cc:260] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.173424 20781 raft_consensus.cc:385] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.173729 20781 raft_consensus.cc:740] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.174453 20781 consensus_queue.cc:260] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.175379 20783 ts_tablet_manager.cc:1434] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:28.176614 20781 ts_tablet_manager.cc:1434] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:05:28.176687 20783 tablet_bootstrap.cc:492] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.179262 20781 tablet_bootstrap.cc:492] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:28.182791 20782 tablet_bootstrap.cc:492] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.183369 20782 ts_tablet_manager.cc:1403] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.003s
I20260709 13:05:28.184865 20783 tablet_bootstrap.cc:654] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.186470 20782 raft_consensus.cc:359] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.187072 20781 tablet_bootstrap.cc:654] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.187348 20782 raft_consensus.cc:385] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.187739 20782 raft_consensus.cc:740] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.188982 20782 consensus_queue.cc:260] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.191483 20787 raft_consensus.cc:493] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.192173 20787 raft_consensus.cc:515] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.195391 20782 ts_tablet_manager.cc:1434] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:05:28.202847 20782 tablet_bootstrap.cc:492] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.205935 20787 leader_election.cc:290] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.212621 20783 tablet_bootstrap.cc:492] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.213181 20783 ts_tablet_manager.cc:1403] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.007s
I20260709 13:05:28.214828 20782 tablet_bootstrap.cc:654] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.220029 20781 tablet_bootstrap.cc:492] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
I20260709 13:05:28.220552 20781 ts_tablet_manager.cc:1403] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.018s
I20260709 13:05:28.225754 20783 raft_consensus.cc:359] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.228503 20781 raft_consensus.cc:359] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.229516 20781 raft_consensus.cc:385] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.229900 20781 raft_consensus.cc:740] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.231261 20781 consensus_queue.cc:260] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.234567 20781 ts_tablet_manager.cc:1434] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.005s
I20260709 13:05:28.235958 20781 tablet_bootstrap.cc:492] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:28.226876 20783 raft_consensus.cc:385] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.243666 20783 raft_consensus.cc:740] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.244747 20783 consensus_queue.cc:260] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.248193 20783 ts_tablet_manager.cc:1434] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.001s
I20260709 13:05:28.250074 20783 tablet_bootstrap.cc:492] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.260205 20781 tablet_bootstrap.cc:654] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:28.265123 20539 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:28.266641 20782 tablet_bootstrap.cc:492] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.267239 20783 tablet_bootstrap.cc:654] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.267244 20782 ts_tablet_manager.cc:1403] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.065s	user 0.023s	sys 0.026s
I20260709 13:05:28.273866 20781 tablet_bootstrap.cc:492] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
I20260709 13:05:28.274519 20781 ts_tablet_manager.cc:1403] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.004s
I20260709 13:05:28.275184 20782 raft_consensus.cc:359] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.275975 20782 raft_consensus.cc:385] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.276297 20782 raft_consensus.cc:740] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.280187 20781 raft_consensus.cc:359] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.281263 20781 raft_consensus.cc:385] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.280889 20782 consensus_queue.cc:260] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.281628 20781 raft_consensus.cc:740] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.282158 20783 tablet_bootstrap.cc:492] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.282656 20783 ts_tablet_manager.cc:1403] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.003s
I20260709 13:05:28.282397 20781 consensus_queue.cc:260] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.285000 20781 ts_tablet_manager.cc:1434] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.003s
I20260709 13:05:28.285729 20783 raft_consensus.cc:359] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.286134 20782 ts_tablet_manager.cc:1434] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:05:28.286161 20781 tablet_bootstrap.cc:492] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:28.286554 20783 raft_consensus.cc:385] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.286918 20783 raft_consensus.cc:740] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.287407 20782 tablet_bootstrap.cc:492] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.289294 20783 consensus_queue.cc:260] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.295605 20781 tablet_bootstrap.cc:654] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.295684 20783 ts_tablet_manager.cc:1434] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:05:28.296938 20783 tablet_bootstrap.cc:492] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.298949 20782 tablet_bootstrap.cc:654] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.303709 20783 tablet_bootstrap.cc:654] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.317830 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.318601 20742 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.320888 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.321583 20593 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.323866 20781 tablet_bootstrap.cc:492] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
I20260709 13:05:28.324393 20781 ts_tablet_manager.cc:1403] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.008s
I20260709 13:05:28.321501 20634 leader_election.cc:304] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:28.325351 20782 tablet_bootstrap.cc:492] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.325892 20787 raft_consensus.cc:2804] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.326252 20787 raft_consensus.cc:493] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.326586 20787 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.326812 20782 ts_tablet_manager.cc:1403] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.009s
I20260709 13:05:28.325385 20783 tablet_bootstrap.cc:492] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.327605 20781 raft_consensus.cc:359] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.328498 20781 raft_consensus.cc:385] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.328991 20781 raft_consensus.cc:740] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.329937 20781 consensus_queue.cc:260] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.330154 20782 raft_consensus.cc:359] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.328537 20783 ts_tablet_manager.cc:1403] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.005s
I20260709 13:05:28.331231 20782 raft_consensus.cc:385] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.331692 20782 raft_consensus.cc:740] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.332419 20782 consensus_queue.cc:260] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.333895 20783 raft_consensus.cc:359] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.334497 20782 ts_tablet_manager.cc:1434] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 13:05:28.334595 20783 raft_consensus.cc:385] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.335009 20783 raft_consensus.cc:740] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.334909 20787 raft_consensus.cc:515] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.335608 20782 tablet_bootstrap.cc:492] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.335912 20783 consensus_queue.cc:260] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.338636 20783 ts_tablet_manager.cc:1434] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:28.339673 20783 tablet_bootstrap.cc:492] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.342051 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:28.342571 20782 tablet_bootstrap.cc:654] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.342707 20593 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.346422 20783 tablet_bootstrap.cc:654] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.347811 20781 ts_tablet_manager.cc:1434] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.004s
I20260709 13:05:28.352926 20781 tablet_bootstrap.cc:492] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:28.355757 20782 tablet_bootstrap.cc:492] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.356266 20782 ts_tablet_manager.cc:1403] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.005s
I20260709 13:05:28.358978 20782 raft_consensus.cc:359] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.359581 20781 tablet_bootstrap.cc:654] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.359786 20782 raft_consensus.cc:385] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.360128 20782 raft_consensus.cc:740] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.361251 20783 tablet_bootstrap.cc:492] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.360924 20782 consensus_queue.cc:260] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.361708 20783 ts_tablet_manager.cc:1403] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.000s
I20260709 13:05:28.363242 20782 ts_tablet_manager.cc:1434] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:28.367555 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.368237 20742 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.368762 20782 tablet_bootstrap.cc:492] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.370801 20783 raft_consensus.cc:359] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.375092 20783 raft_consensus.cc:385] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.375895 20783 raft_consensus.cc:740] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.376281 20742 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 1.
I20260709 13:05:28.365372 20593 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 1.
I20260709 13:05:28.367050 20786 raft_consensus.cc:493] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.377153 20783 consensus_queue.cc:260] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.377737 20786 raft_consensus.cc:515] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.377995 20634 leader_election.cc:304] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:28.379933 20783 ts_tablet_manager.cc:1434] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:05:28.381060 20783 tablet_bootstrap.cc:492] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.381664 20786 leader_election.cc:290] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.376281 20782 tablet_bootstrap.cc:654] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.382181 20786 raft_consensus.cc:493] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.382663 20786 raft_consensus.cc:515] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.384723 20786 leader_election.cc:290] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.366891 20781 tablet_bootstrap.cc:492] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
I20260709 13:05:28.393913 20781 ts_tablet_manager.cc:1403] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.004s
I20260709 13:05:28.394483 20783 tablet_bootstrap.cc:654] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.397248 20781 raft_consensus.cc:359] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.398187 20781 raft_consensus.cc:385] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.398505 20781 raft_consensus.cc:740] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.400817 20782 tablet_bootstrap.cc:492] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.401568 20782 ts_tablet_manager.cc:1403] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.010s
I20260709 13:05:28.365022 20787 leader_election.cc:290] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.374485 20808 raft_consensus.cc:493] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.404974 20787 raft_consensus.cc:493] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.404654 20808 raft_consensus.cc:515] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.405619 20787 raft_consensus.cc:515] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.407009 20782 raft_consensus.cc:359] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.407509 20808 leader_election.cc:290] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.404654 20818 raft_consensus.cc:2804] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.409454 20782 raft_consensus.cc:385] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.410260 20782 raft_consensus.cc:740] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.411618 20782 consensus_queue.cc:260] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.413466 20787 leader_election.cc:290] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.413933 20782 ts_tablet_manager.cc:1434] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:05:28.414445 20818 raft_consensus.cc:697] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Becoming Leader. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:28.416723 20818 consensus_queue.cc:237] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.421597 20782 tablet_bootstrap.cc:492] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62: Bootstrap starting.
I20260709 13:05:28.419770 20781 consensus_queue.cc:260] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.424207 20781 ts_tablet_manager.cc:1434] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.030s	user 0.002s	sys 0.005s
I20260709 13:05:28.431491 20786 raft_consensus.cc:493] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:28.430590 20634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:34945, user_credentials={real_user=slave}} blocked reactor thread for 52663.3us
I20260709 13:05:28.432574 20786 raft_consensus.cc:515] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.435583 20786 leader_election.cc:290] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.438817 20781 tablet_bootstrap.cc:492] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:28.442715 20782 tablet_bootstrap.cc:654] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.445091 20813 raft_consensus.cc:493] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.445405 20488 catalog_manager.cc:5697] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.448455 20783 tablet_bootstrap.cc:492] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.449257 20783 ts_tablet_manager.cc:1403] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.068s	user 0.023s	sys 0.004s
I20260709 13:05:28.452142 20792 raft_consensus.cc:493] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.452682 20792 raft_consensus.cc:515] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.445729 20813 raft_consensus.cc:515] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.455019 20783 raft_consensus.cc:359] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.456832 20783 raft_consensus.cc:385] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.457317 20783 raft_consensus.cc:740] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.460625 20792 leader_election.cc:290] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:28.461835 20813 leader_election.cc:290] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.469362 20783 consensus_queue.cc:260] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.482705 20783 ts_tablet_manager.cc:1434] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 13:05:28.483733 20783 tablet_bootstrap.cc:492] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.486828 20781 tablet_bootstrap.cc:654] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.487387 20812 raft_consensus.cc:493] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.487898 20812 raft_consensus.cc:515] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.488628 20741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.489317 20741 raft_consensus.cc:2468] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.489751 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.497468 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.498271 20593 raft_consensus.cc:2468] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.499503 20786 raft_consensus.cc:493] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.499996 20786 raft_consensus.cc:515] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.491865 20813 raft_consensus.cc:493] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.491708 20812 leader_election.cc:290] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.503726 20818 raft_consensus.cc:493] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.507548 20818 raft_consensus.cc:515] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.508332 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.508983 20593 raft_consensus.cc:2468] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.509732 20818 leader_election.cc:290] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.502126 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.510133 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.510538 20742 raft_consensus.cc:2468] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.510751 20592 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.518097 20634 leader_election.cc:304] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:28.506430 20812 raft_consensus.cc:493] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:28.519474 20634 leader_election.cc:343] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.519613 20812 raft_consensus.cc:515] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.520463 20634 leader_election.cc:304] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:28.522100 20812 leader_election.cc:290] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.503342 20813 raft_consensus.cc:515] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.526789 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.527501 20813 leader_election.cc:290] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.527503 20740 raft_consensus.cc:2468] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 0.
I20260709 13:05:28.530275 20741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8ac689b5ca413993647d45b5fbfd17" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.530953 20741 raft_consensus.cc:2468] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 0.
I20260709 13:05:28.531551 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.531502 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.532393 20786 leader_election.cc:290] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.532554 20592 raft_consensus.cc:2468] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.525406 20787 raft_consensus.cc:2804] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.533712 20787 raft_consensus.cc:493] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.534071 20787 raft_consensus.cc:3060] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.526199 20818 raft_consensus.cc:2804] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.522647 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2b9bdf278a40569029fb629711e517" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.539656 20813 raft_consensus.cc:493] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.539947 20742 raft_consensus.cc:2468] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 0.
I20260709 13:05:28.540172 20813 raft_consensus.cc:515] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.541281 20782 tablet_bootstrap.cc:492] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62: No bootstrap required, opened a new log
I20260709 13:05:28.541872 20782 ts_tablet_manager.cc:1403] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62: Time spent bootstrapping tablet: real 0.121s	user 0.023s	sys 0.023s
I20260709 13:05:28.542487 20813 leader_election.cc:290] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.544391 20634 leader_election.cc:304] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 956d2eb6cb0a422998647448946aae00
I20260709 13:05:28.546231 20813 raft_consensus.cc:2804] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.545804 20559 leader_election.cc:304] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.538213 20818 raft_consensus.cc:493] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.545799 20782 raft_consensus.cc:359] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.548884 20782 raft_consensus.cc:385] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.549477 20782 raft_consensus.cc:740] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.551559 20782 consensus_queue.cc:260] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.552932 20741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.553761 20782 ts_tablet_manager.cc:1434] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:05:28.554304 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.555356 20559 leader_election.cc:304] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.556321 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.556937 20740 raft_consensus.cc:2468] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 0.
I20260709 13:05:28.558467 20559 leader_election.cc:304] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
W20260709 13:05:28.559388 20559 leader_election.cc:343] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:28.560041 20559 leader_election.cc:343] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.560699 20559 leader_election.cc:304] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.561733 20812 raft_consensus.cc:2804] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.562101 20812 raft_consensus.cc:493] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.562433 20812 raft_consensus.cc:3060] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.551978 20786 raft_consensus.cc:2804] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.567979 20786 raft_consensus.cc:493] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.568331 20786 raft_consensus.cc:3060] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.548841 20818 raft_consensus.cc:3060] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.546802 20813 raft_consensus.cc:493] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.581687 20813 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.588192 20781 tablet_bootstrap.cc:492] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
I20260709 13:05:28.583707 20634 leader_election.cc:304] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.588869 20781 ts_tablet_manager.cc:1403] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.150s	user 0.011s	sys 0.037s
I20260709 13:05:28.589609 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.591051 20808 raft_consensus.cc:2804] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.591552 20808 raft_consensus.cc:493] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.592795 20808 raft_consensus.cc:3060] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.591519 20592 raft_consensus.cc:2468] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 0.
I20260709 13:05:28.595551 20708 leader_election.cc:304] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.596664 20792 raft_consensus.cc:2804] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.597029 20792 raft_consensus.cc:493] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.597338 20792 raft_consensus.cc:3060] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.595391 20827 raft_consensus.cc:2804] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.598790 20827 raft_consensus.cc:493] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.599164 20827 raft_consensus.cc:3060] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.602311 20783 tablet_bootstrap.cc:654] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.602142 20808 raft_consensus.cc:515] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.588189 20787 raft_consensus.cc:515] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.606428 20781 raft_consensus.cc:359] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.607040 20781 raft_consensus.cc:385] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.607333 20781 raft_consensus.cc:740] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.607892 20787 leader_election.cc:290] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.607990 20781 consensus_queue.cc:260] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.609644 20781 ts_tablet_manager.cc:1434] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:05:28.610380 20781 tablet_bootstrap.cc:492] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:28.610549 20827 raft_consensus.cc:515] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.610962 20818 raft_consensus.cc:515] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.614051 20827 leader_election.cc:290] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.615309 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8ac689b5ca413993647d45b5fbfd17" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.615358 20792 raft_consensus.cc:515] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.616374 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2b9bdf278a40569029fb629711e517" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.604527 20813 raft_consensus.cc:515] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.616993 20740 raft_consensus.cc:3060] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.617247 20781 tablet_bootstrap.cc:654] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.618559 20813 leader_election.cc:290] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.613371 20818 leader_election.cc:290] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.613919 20830 raft_consensus.cc:2804] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.619910 20830 raft_consensus.cc:493] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.621421 20786 raft_consensus.cc:515] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.620529 20830 raft_consensus.cc:3060] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.624852 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8ac689b5ca413993647d45b5fbfd17" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.625078 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.625219 20786 leader_election.cc:290] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.625576 20742 raft_consensus.cc:2468] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.625696 20593 raft_consensus.cc:2468] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
I20260709 13:05:28.605027 20812 raft_consensus.cc:515] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.628706 20592 raft_consensus.cc:2393] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9b8479b509c3477bb21a7f8eaa9fef73 in current term 1: Already voted for candidate a3a2ec2796c64f29a00145d5f75f7b62 in this term.
I20260709 13:05:28.628648 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.629410 20742 raft_consensus.cc:3060] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.631318 20741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8ac689b5ca413993647d45b5fbfd17" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.631917 20741 raft_consensus.cc:3060] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.633217 20808 leader_election.cc:290] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.634930 20739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:28.635720 20739 raft_consensus.cc:2468] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 0.
W20260709 13:05:28.637766 20634 leader_election.cc:343] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.639027 20634 leader_election.cc:304] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:28.640213 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:28.640511 20634 leader_election.cc:304] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:28.641175 20592 raft_consensus.cc:3060] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.641579 20808 raft_consensus.cc:2804] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.641793 20792 leader_election.cc:290] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:28.639892 20812 leader_election.cc:290] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.640780 20740 raft_consensus.cc:2468] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 1.
I20260709 13:05:28.645310 20818 raft_consensus.cc:2804] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.646229 20818 raft_consensus.cc:493] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.646648 20818 raft_consensus.cc:3060] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.642127 20808 raft_consensus.cc:493] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.646862 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:28.647778 20593 raft_consensus.cc:2393] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9b8479b509c3477bb21a7f8eaa9fef73 in current term 1: Already voted for candidate a3a2ec2796c64f29a00145d5f75f7b62 in this term.
I20260709 13:05:28.647265 20808 raft_consensus.cc:3060] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.648923 20741 raft_consensus.cc:2468] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 1.
I20260709 13:05:28.649899 20739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.652416 20667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:28.652396 20668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:28.653867 20667 raft_consensus.cc:2393] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 956d2eb6cb0a422998647448946aae00 in current term 1: Already voted for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in this term.
I20260709 13:05:28.655692 20559 leader_election.cc:304] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.654702 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:28.656570 20812 raft_consensus.cc:2804] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.657267 20812 raft_consensus.cc:697] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:28.658116 20812 consensus_queue.cc:237] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.659093 20559 leader_election.cc:304] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.660130 20827 raft_consensus.cc:2804] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.660239 20741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.658366 20818 raft_consensus.cc:515] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.658908 20592 raft_consensus.cc:2468] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 1.
I20260709 13:05:28.661110 20741 raft_consensus.cc:2393] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9b8479b509c3477bb21a7f8eaa9fef73 in current term 1: Already voted for candidate 956d2eb6cb0a422998647448946aae00 in this term.
I20260709 13:05:28.661341 20827 raft_consensus.cc:697] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:28.659093 20792 raft_consensus.cc:493] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.662281 20827 consensus_queue.cc:237] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.662606 20792 raft_consensus.cc:515] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.662338 20818 leader_election.cc:290] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.664297 20708 leader_election.cc:304] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:28.665800 20808 raft_consensus.cc:515] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.652936 20742 raft_consensus.cc:2468] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 1.
I20260709 13:05:28.667797 20667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b79b8b205834585a55c92a7a0eaf03e" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:28.670899 20808 leader_election.cc:290] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.670603 20792 leader_election.cc:290] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:28.678917 20792 raft_consensus.cc:2804] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:28.680331 20706 leader_election.cc:343] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.680546 20792 raft_consensus.cc:697] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Becoming Leader. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:28.682122 20559 leader_election.cc:304] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.679908 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:28.683219 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.683488 20592 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.683599 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:28.684234 20593 raft_consensus.cc:3060] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.685702 20830 raft_consensus.cc:515] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.690570 20668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:28.687505 20812 raft_consensus.cc:2804] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.689147 20667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2b9bdf278a40569029fb629711e517" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:28.696638 20593 raft_consensus.cc:2468] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 1.
I20260709 13:05:28.691843 20668 raft_consensus.cc:2393] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3a2ec2796c64f29a00145d5f75f7b62 in current term 1: Already voted for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in this term.
I20260709 13:05:28.693369 20781 tablet_bootstrap.cc:492] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
I20260709 13:05:28.691900 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.696818 20667 raft_consensus.cc:2468] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 0.
I20260709 13:05:28.698148 20781 ts_tablet_manager.cc:1403] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.088s	user 0.019s	sys 0.004s
I20260709 13:05:28.697139 20591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b79b8b205834585a55c92a7a0eaf03e" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.692536 20792 consensus_queue.cc:237] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.698705 20591 raft_consensus.cc:2468] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 0.
I20260709 13:05:28.700462 20812 raft_consensus.cc:697] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:28.701299 20708 leader_election.cc:304] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:28.701514 20783 tablet_bootstrap.cc:492] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.701495 20812 consensus_queue.cc:237] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.702402 20783 ts_tablet_manager.cc:1403] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.219s	user 0.015s	sys 0.004s
I20260709 13:05:28.703697 20592 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 1.
I20260709 13:05:28.702927 20489 catalog_manager.cc:5697] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.705526 20667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:28.711018 20667 raft_consensus.cc:2393] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3a2ec2796c64f29a00145d5f75f7b62 in current term 1: Already voted for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in this term.
I20260709 13:05:28.707319 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:28.710904 20668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8ac689b5ca413993647d45b5fbfd17" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:28.712288 20834 raft_consensus.cc:2804] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.712393 20668 raft_consensus.cc:2393] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3a2ec2796c64f29a00145d5f75f7b62 in current term 1: Already voted for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in this term.
I20260709 13:05:28.712744 20834 raft_consensus.cc:493] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.713141 20834 raft_consensus.cc:3060] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.716899 20668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:28.717296 20781 raft_consensus.cc:359] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.717773 20668 raft_consensus.cc:2393] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3a2ec2796c64f29a00145d5f75f7b62 in current term 1: Already voted for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in this term.
I20260709 13:05:28.717991 20781 raft_consensus.cc:385] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.718293 20781 raft_consensus.cc:740] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.720796 20783 raft_consensus.cc:359] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.721714 20783 raft_consensus.cc:385] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.722039 20783 raft_consensus.cc:740] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.719259 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.723635 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2b9bdf278a40569029fb629711e517" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:28.723788 20742 raft_consensus.cc:3060] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.724259 20666 raft_consensus.cc:3060] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.724136 20781 consensus_queue.cc:260] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.727346 20781 ts_tablet_manager.cc:1434] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.003s
I20260709 13:05:28.727586 20667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:28.728677 20667 raft_consensus.cc:2393] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3a2ec2796c64f29a00145d5f75f7b62 in current term 1: Already voted for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in this term.
I20260709 13:05:28.730154 20781 tablet_bootstrap.cc:492] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73: Bootstrap starting.
I20260709 13:05:28.719849 20830 leader_election.cc:290] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:28.739640 20781 tablet_bootstrap.cc:654] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.742439 20668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:28.743813 20634 leader_election.cc:304] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.744369 20667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8ac689b5ca413993647d45b5fbfd17" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:28.745141 20667 raft_consensus.cc:2393] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3a2ec2796c64f29a00145d5f75f7b62 in current term 1: Already voted for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in this term.
I20260709 13:05:28.745710 20591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:28.745944 20808 raft_consensus.cc:2804] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.746321 20591 raft_consensus.cc:3060] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.746421 20808 raft_consensus.cc:697] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Becoming Leader. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:28.747361 20808 consensus_queue.cc:237] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.751484 20783 consensus_queue.cc:260] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.733987 20834 raft_consensus.cc:515] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
W20260709 13:05:28.755072 20708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:33571, user_credentials={real_user=slave}} blocked reactor thread for 53893.5us
I20260709 13:05:28.757851 20783 ts_tablet_manager.cc:1434] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 13:05:28.758687 20783 tablet_bootstrap.cc:492] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00: Bootstrap starting.
I20260709 13:05:28.760068 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.760694 20740 raft_consensus.cc:3060] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.769731 20591 raft_consensus.cc:2468] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 1.
I20260709 13:05:28.770339 20740 raft_consensus.cc:2468] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 1.
I20260709 13:05:28.776850 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8ac689b5ca413993647d45b5fbfd17" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:28.777014 20666 raft_consensus.cc:2468] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 1.
I20260709 13:05:28.777668 20740 raft_consensus.cc:2393] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9b8479b509c3477bb21a7f8eaa9fef73 in current term 1: Already voted for candidate a3a2ec2796c64f29a00145d5f75f7b62 in this term.
I20260709 13:05:28.785809 20557 leader_election.cc:304] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3a2ec2796c64f29a00145d5f75f7b62; no voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:28.785373 20491 catalog_manager.cc:5697] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.788477 20812 raft_consensus.cc:2749] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.789806 20781 tablet_bootstrap.cc:492] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73: No bootstrap required, opened a new log
W20260709 13:05:28.788290 20557 leader_election.cc:343] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.790480 20781 ts_tablet_manager.cc:1403] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.000s
I20260709 13:05:28.796069 20557 leader_election.cc:304] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3a2ec2796c64f29a00145d5f75f7b62; no voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:28.797255 20812 raft_consensus.cc:2749] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.799367 20834 leader_election.cc:290] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
W20260709 13:05:28.799907 20557 leader_election.cc:343] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.801203 20742 raft_consensus.cc:2468] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 1.
I20260709 13:05:28.787930 20488 catalog_manager.cc:5697] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.801512 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b79b8b205834585a55c92a7a0eaf03e" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:28.787734 20591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8ac689b5ca413993647d45b5fbfd17" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:28.802168 20592 raft_consensus.cc:3060] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.802290 20559 leader_election.cc:304] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:28.803665 20812 raft_consensus.cc:2804] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.804178 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b79b8b205834585a55c92a7a0eaf03e" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:28.804198 20812 raft_consensus.cc:697] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:28.803874 20488 catalog_manager.cc:5697] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.804944 20666 raft_consensus.cc:3060] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.805335 20812 consensus_queue.cc:237] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.810297 20592 raft_consensus.cc:2468] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 1.
I20260709 13:05:28.811564 20708 leader_election.cc:304] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:28.812439 20834 raft_consensus.cc:2804] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.812953 20834 raft_consensus.cc:697] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Becoming Leader. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:28.813651 20781 raft_consensus.cc:359] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.813808 20834 consensus_queue.cc:237] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.785804 20634 leader_election.cc:304] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:28.795876 20489 catalog_manager.cc:5697] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 956d2eb6cb0a422998647448946aae00 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "956d2eb6cb0a422998647448946aae00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.815692 20808 raft_consensus.cc:2749] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.815215 20783 tablet_bootstrap.cc:654] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.817790 20836 raft_consensus.cc:493] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.814321 20781 raft_consensus.cc:385] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.818617 20634 leader_election.cc:304] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:28.819028 20781 raft_consensus.cc:740] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Initialized, Role: FOLLOWER
W20260709 13:05:28.820024 20634 leader_election.cc:343] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.818327 20836 raft_consensus.cc:515] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
W20260709 13:05:28.821713 20634 leader_election.cc:343] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.820468 20808 raft_consensus.cc:2749] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.824029 20666 raft_consensus.cc:2468] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 1.
I20260709 13:05:28.820211 20781 consensus_queue.cc:260] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.827600 20634 leader_election.cc:304] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:28.830796 20783 tablet_bootstrap.cc:492] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00: No bootstrap required, opened a new log
I20260709 13:05:28.831579 20783 ts_tablet_manager.cc:1403] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00: Time spent bootstrapping tablet: real 0.073s	user 0.014s	sys 0.004s
I20260709 13:05:28.836712 20781 ts_tablet_manager.cc:1434] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:05:28.839265 20634 leader_election.cc:304] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 956d2eb6cb0a422998647448946aae00
I20260709 13:05:28.839636 20813 raft_consensus.cc:2804] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.841017 20818 raft_consensus.cc:2804] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.842144 20818 raft_consensus.cc:697] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Becoming Leader. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:28.842481 20836 leader_election.cc:290] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:28.843502 20818 consensus_queue.cc:237] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.840955 20634 leader_election.cc:304] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:28.844223 20591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:28.846246 20591 raft_consensus.cc:2393] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 956d2eb6cb0a422998647448946aae00 in current term 1: Already voted for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in this term.
I20260709 13:05:28.848331 20787 raft_consensus.cc:2749] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.850291 20787 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:28.852798 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:28.856774 20706 leader_election.cc:304] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 956d2eb6cb0a422998647448946aae00; no voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:28.858047 20488 catalog_manager.cc:5697] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 956d2eb6cb0a422998647448946aae00 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "956d2eb6cb0a422998647448946aae00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.859630 20783 raft_consensus.cc:359] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.862262 20834 raft_consensus.cc:3060] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.875452 20783 raft_consensus.cc:385] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.875839 20783 raft_consensus.cc:740] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.876103 20489 catalog_manager.cc:5697] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.877833 20813 raft_consensus.cc:697] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Becoming Leader. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:28.878713 20813 consensus_queue.cc:237] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.884217 20783 consensus_queue.cc:260] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:28.885823 20834 raft_consensus.cc:2749] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.890362 20783 ts_tablet_manager.cc:1434] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 13:05:28.894520 20813 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:28.902653 20488 catalog_manager.cc:5697] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.904731 20488 catalog_manager.cc:5697] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:29.056301 20834 raft_consensus.cc:493] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.056818 20834 raft_consensus.cc:515] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.058717 20834 leader_election.cc:290] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:29.059718 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:29.059720 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" is_pre_election: true
I20260709 13:05:29.060494 20592 raft_consensus.cc:2393] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 956d2eb6cb0a422998647448946aae00 in current term 1: Already voted for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in this term.
I20260709 13:05:29.061450 20708 leader_election.cc:304] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 956d2eb6cb0a422998647448946aae00; no voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:29.062144 20834 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:29.067080 20834 raft_consensus.cc:2749] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:29.088060 20818 raft_consensus.cc:493] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.088598 20818 raft_consensus.cc:515] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.090930 20818 leader_election.cc:290] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:29.091645 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" is_pre_election: true
I20260709 13:05:29.091969 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d2eb6cb0a422998647448946aae00" is_pre_election: true
I20260709 13:05:29.092706 20742 raft_consensus.cc:2393] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9b8479b509c3477bb21a7f8eaa9fef73 in current term 1: Already voted for candidate a3a2ec2796c64f29a00145d5f75f7b62 in this term.
I20260709 13:05:29.093611 20634 leader_election.cc:304] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:29.094269 20818 raft_consensus.cc:3060] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:29.099179 20666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
mode: GRACEFUL
new_leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:55286
I20260709 13:05:29.099851 20666 raft_consensus.cc:606] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Received request to transfer leadership to TS a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:29.101128 20818 raft_consensus.cc:2749] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:29.105535 20666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
mode: GRACEFUL
new_leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:55286
I20260709 13:05:29.106027 20666 raft_consensus.cc:606] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Received request to transfer leadership to TS a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:29.116580 20666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
mode: GRACEFUL
new_leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:55286
I20260709 13:05:29.117082 20666 raft_consensus.cc:606] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Received request to transfer leadership to TS a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:29.130421 20592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
mode: GRACEFUL
new_leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:33266
I20260709 13:05:29.130923 20592 raft_consensus.cc:606] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Received request to transfer leadership to TS 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:29.137588 20812 consensus_queue.cc:1048] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.141314 20830 consensus_queue.cc:1048] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.148949 20740 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a"
dest_uuid: "956d2eb6cb0a422998647448946aae00"
mode: GRACEFUL
new_leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:35782
I20260709 13:05:29.149600 20740 raft_consensus.cc:606] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Received request to transfer leadership to TS 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:29.162462 20742 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b79b8b205834585a55c92a7a0eaf03e"
dest_uuid: "956d2eb6cb0a422998647448946aae00"
mode: GRACEFUL
new_leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:35782
I20260709 13:05:29.163115 20742 raft_consensus.cc:606] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Received request to transfer leadership to TS 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:29.169327 20592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
mode: GRACEFUL
new_leader_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:33266
I20260709 13:05:29.169972 20592 raft_consensus.cc:606] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Received request to transfer leadership to TS 956d2eb6cb0a422998647448946aae00
I20260709 13:05:29.189175 20830 consensus_queue.cc:1048] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.199548 20830 consensus_queue.cc:1048] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.207858 20812 consensus_queue.cc:1048] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.207896 20786 raft_consensus.cc:993] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62: : Instructing follower 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:29.208714 20830 raft_consensus.cc:1081] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Signalling peer 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:29.210404 20666 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:05:29.210969 20666 raft_consensus.cc:493] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.211377 20666 raft_consensus.cc:3060] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.221506 20827 consensus_queue.cc:1048] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:29.223495 20666 raft_consensus.cc:515] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.229840 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:29.230484 20592 raft_consensus.cc:3055] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:29.230799 20592 raft_consensus.cc:740] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:29.231624 20592 consensus_queue.cc:260] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.232733 20592 raft_consensus.cc:3060] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.237792 20666 leader_election.cc:290] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 2 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:29.239174 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:29.238086 20592 raft_consensus.cc:2468] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 2.
I20260709 13:05:29.239904 20742 raft_consensus.cc:3060] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.239317 20827 consensus_queue.cc:1048] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.241053 20634 leader_election.cc:304] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:29.241910 20818 raft_consensus.cc:2804] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:29.242381 20818 raft_consensus.cc:697] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Becoming Leader. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:29.243211 20818 consensus_queue.cc:237] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.248201 20742 raft_consensus.cc:2468] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 2.
I20260709 13:05:29.255765 20830 raft_consensus.cc:993] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62: : Instructing follower 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:29.256258 20827 raft_consensus.cc:1081] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Signalling peer 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:29.256371 20813 consensus_queue.cc:1048] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.256985 20491 catalog_manager.cc:5697] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 reported cstate change: term changed from 1 to 2, leader changed from a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193) to 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:29.258172 20742 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f"
dest_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:35774
I20260709 13:05:29.258733 20742 raft_consensus.cc:493] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.259111 20742 raft_consensus.cc:3060] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.262262 20830 consensus_queue.cc:1048] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.267436 20742 raft_consensus.cc:515] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.268539 20740 raft_consensus.cc:1240] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Rejecting Update request from peer a3a2ec2796c64f29a00145d5f75f7b62 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:29.272428 20830 consensus_queue.cc:1059] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:29.273500 20813 consensus_queue.cc:1048] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.273694 20830 raft_consensus.cc:3055] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:29.273880 20591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:29.274767 20830 raft_consensus.cc:740] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:29.275205 20742 leader_election.cc:290] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 2 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:29.276053 20667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b972ae2bc69426aa33cd855b1ae4c1f" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:29.279930 20830 consensus_queue.cc:260] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.282123 20830 raft_consensus.cc:3060] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.294104 20813 consensus_queue.cc:1048] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.299518 20813 raft_consensus.cc:993] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73: : Instructing follower a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:29.300985 20787 raft_consensus.cc:1081] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Signalling peer a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:29.307507 20844 consensus_queue.cc:1048] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:29.308233 20593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:33248
I20260709 13:05:29.308825 20593 raft_consensus.cc:493] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.309196 20593 raft_consensus.cc:3060] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.309767 20591 raft_consensus.cc:2468] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 2.
I20260709 13:05:29.312444 20708 leader_election.cc:304] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:29.315482 20834 raft_consensus.cc:2804] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:29.315996 20834 raft_consensus.cc:697] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 2 LEADER]: Becoming Leader. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:29.316881 20834 consensus_queue.cc:237] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.322470 20593 raft_consensus.cc:515] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.324167 20592 raft_consensus.cc:1240] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Rejecting Update request from peer 9b8479b509c3477bb21a7f8eaa9fef73 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:29.326838 20813 consensus_queue.cc:1059] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:29.327883 20813 raft_consensus.cc:3055] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:29.328208 20813 raft_consensus.cc:740] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:29.328378 20593 leader_election.cc:290] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 2 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:29.328413 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:29.329903 20741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f33b8bce6a478cb6d1414801695d14" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:29.329320 20813 consensus_queue.cc:260] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.334689 20813 raft_consensus.cc:3060] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.335382 20834 consensus_queue.cc:1048] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:29.341573 20834 consensus_queue.cc:1048] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.356742 20666 raft_consensus.cc:2468] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 2.
I20260709 13:05:29.358652 20557 leader_election.cc:304] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 956d2eb6cb0a422998647448946aae00
I20260709 13:05:29.359587 20830 raft_consensus.cc:2804] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:29.360111 20830 raft_consensus.cc:697] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:29.359985 20489 catalog_manager.cc:5697] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 reported cstate change: term changed from 1 to 2, leader changed from a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193) to 956d2eb6cb0a422998647448946aae00 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "956d2eb6cb0a422998647448946aae00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:29.361001 20866 raft_consensus.cc:993] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00: : Instructing follower 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:29.360987 20830 consensus_queue.cc:237] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.361637 20866 raft_consensus.cc:1081] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Signalling peer 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:29.367879 20666 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:55280
I20260709 13:05:29.368468 20666 raft_consensus.cc:493] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.368848 20666 raft_consensus.cc:3060] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.372220 20834 consensus_queue.cc:1048] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.376015 20666 raft_consensus.cc:515] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.380158 20666 leader_election.cc:290] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 2 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:29.380764 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:29.381366 20740 raft_consensus.cc:3055] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:29.379832 20489 catalog_manager.cc:5697] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 1 to 2, leader changed from 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194) to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:29.381702 20740 raft_consensus.cc:740] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:29.382650 20740 consensus_queue.cc:260] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.383670 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:29.383936 20740 raft_consensus.cc:3060] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.384266 20592 raft_consensus.cc:3060] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.395344 20866 consensus_queue.cc:1048] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:29.397763 20740 raft_consensus.cc:2468] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 2.
I20260709 13:05:29.399170 20634 leader_election.cc:304] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:29.400094 20813 raft_consensus.cc:2804] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:29.400573 20813 raft_consensus.cc:697] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Becoming Leader. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:29.401353 20813 consensus_queue.cc:237] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.399240 20592 raft_consensus.cc:2468] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 2.
I20260709 13:05:29.404875 20834 raft_consensus.cc:993] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00: : Instructing follower 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:29.405313 20836 raft_consensus.cc:1081] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Signalling peer 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:29.406817 20666 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b79b8b205834585a55c92a7a0eaf03e"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:55280
I20260709 13:05:29.407385 20666 raft_consensus.cc:493] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.407827 20666 raft_consensus.cc:3060] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.413313 20666 raft_consensus.cc:515] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.414908 20666 leader_election.cc:290] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 2 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:29.415944 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b79b8b205834585a55c92a7a0eaf03e" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:29.416566 20592 raft_consensus.cc:3060] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.416579 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b79b8b205834585a55c92a7a0eaf03e" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:29.417193 20740 raft_consensus.cc:3055] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:29.417464 20740 raft_consensus.cc:740] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:29.418174 20740 consensus_queue.cc:260] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.419134 20740 raft_consensus.cc:3060] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.419456 20489 catalog_manager.cc:5697] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 reported cstate change: term changed from 1 to 2, leader changed from 956d2eb6cb0a422998647448946aae00 (127.19.127.195) to 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:29.424803 20592 raft_consensus.cc:2468] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 2.
I20260709 13:05:29.425275 20740 raft_consensus.cc:2468] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 2.
I20260709 13:05:29.426389 20634 leader_election.cc:304] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:29.427152 20813 raft_consensus.cc:2804] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:29.427637 20813 raft_consensus.cc:697] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Becoming Leader. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:29.428279 20808 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.428478 20813 consensus_queue.cc:237] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.436597 20489 catalog_manager.cc:5697] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 reported cstate change: term changed from 1 to 2, leader changed from 956d2eb6cb0a422998647448946aae00 (127.19.127.195) to 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:29.441089 20844 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:29.451388 20844 raft_consensus.cc:993] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73: : Instructing follower a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:29.451913 20844 raft_consensus.cc:1081] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Signalling peer a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:29.453589 20592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:33248
I20260709 13:05:29.454073 20592 raft_consensus.cc:493] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.454391 20592 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.459262 20592 raft_consensus.cc:515] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.461735 20592 leader_election.cc:290] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 2 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:29.462083 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:29.462318 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:29.462704 20666 raft_consensus.cc:3055] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:29.462815 20742 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.463099 20666 raft_consensus.cc:740] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:29.464007 20666 consensus_queue.cc:260] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.465291 20666 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.469905 20742 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 2.
I20260709 13:05:29.471154 20559 leader_election.cc:304] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:29.471837 20666 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 2.
I20260709 13:05:29.471921 20830 raft_consensus.cc:2804] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:29.472522 20830 raft_consensus.cc:697] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:29.473315 20830 consensus_queue.cc:237] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.479406 20491 catalog_manager.cc:5697] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 1 to 2, leader changed from 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194) to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:29.492162 20808 raft_consensus.cc:993] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73: : Instructing follower a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:29.492631 20844 raft_consensus.cc:1081] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Signalling peer a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:29.494024 20592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:33248
I20260709 13:05:29.494518 20592 raft_consensus.cc:493] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:29.494843 20592 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.499517 20592 raft_consensus.cc:515] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.501274 20592 leader_election.cc:290] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 2 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:29.502416 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:29.502416 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:29.503121 20742 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.503194 20666 raft_consensus.cc:3055] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:29.503625 20666 raft_consensus.cc:740] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:29.504571 20666 consensus_queue.cc:260] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.505657 20666 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.508297 20742 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 2.
I20260709 13:05:29.509408 20559 leader_election.cc:304] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:29.510154 20830 raft_consensus.cc:2804] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:29.510659 20830 raft_consensus.cc:697] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:29.510834 20666 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 2.
I20260709 13:05:29.511770 20830 consensus_queue.cc:237] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:29.518163 20491 catalog_manager.cc:5697] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 1 to 2, leader changed from 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194) to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:29.668223 20742 raft_consensus.cc:1275] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.669595 20844 consensus_queue.cc:1048] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.677353 20592 raft_consensus.cc:1275] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.679260 20808 consensus_queue.cc:1048] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.732772 20592 raft_consensus.cc:1275] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.733986 20834 consensus_queue.cc:1048] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.742775 20666 raft_consensus.cc:3060] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.750732 20666 raft_consensus.cc:1275] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.752547 20834 consensus_queue.cc:1048] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.811655 20742 raft_consensus.cc:3060] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.817134 20742 raft_consensus.cc:1275] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.818571 20830 consensus_queue.cc:1048] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.828219 20666 raft_consensus.cc:1275] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.831627 20812 consensus_queue.cc:1048] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.894500 20742 raft_consensus.cc:1275] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.895802 20813 consensus_queue.cc:1048] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.903493 20592 raft_consensus.cc:1275] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.905019 20808 consensus_queue.cc:1048] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.915390 20742 raft_consensus.cc:1275] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.916795 20812 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.925555 20666 raft_consensus.cc:1275] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.927145 20874 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.976711 20592 raft_consensus.cc:1275] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.978123 20808 consensus_queue.cc:1048] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.980466 20742 raft_consensus.cc:1275] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.983613 20874 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:29.990262 20742 raft_consensus.cc:1275] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.992332 20813 consensus_queue.cc:1048] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:30.000615 20666 raft_consensus.cc:1275] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:30.002034 20874 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:32.203800 19967 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:05:32.204608 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:32.222055 20592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
mode: GRACEFUL
new_leader_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:54760
I20260709 13:05:32.222658 20592 raft_consensus.cc:606] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Received request to transfer leadership to TS 956d2eb6cb0a422998647448946aae00
I20260709 13:05:32.225168 20592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
mode: GRACEFUL
new_leader_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:54760
I20260709 13:05:32.225656 20592 raft_consensus.cc:606] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Received request to transfer leadership to TS 956d2eb6cb0a422998647448946aae00
I20260709 13:05:32.226594 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:32.227065 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:32.503803 20874 raft_consensus.cc:993] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62: : Instructing follower 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:32.504314 20879 raft_consensus.cc:1081] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Signalling peer 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:32.505939 20742 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823"
dest_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:35774
I20260709 13:05:32.506448 20742 raft_consensus.cc:493] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:32.506759 20742 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.511649 20742 raft_consensus.cc:515] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:32.513396 20742 leader_election.cc:290] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 3 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571)
I20260709 13:05:32.514274 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:32.514535 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:32.514856 20666 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.515050 20592 raft_consensus.cc:3055] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:32.515411 20592 raft_consensus.cc:740] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:32.516232 20592 consensus_queue.cc:260] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:32.517491 20592 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.521201 20666 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 3.
I20260709 13:05:32.522218 20706 leader_election.cc:304] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:32.522985 20878 raft_consensus.cc:2804] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:32.524026 20592 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 3.
I20260709 13:05:32.524137 20878 raft_consensus.cc:697] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Becoming Leader. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:32.525048 20878 consensus_queue.cc:237] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:32.531625 20489 catalog_manager.cc:5697] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 reported cstate change: term changed from 2 to 3, leader changed from a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193) to 956d2eb6cb0a422998647448946aae00 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "956d2eb6cb0a422998647448946aae00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:32.642642 20874 raft_consensus.cc:993] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62: : Instructing follower 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:32.643119 20879 raft_consensus.cc:1081] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Signalling peer 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:32.644569 20742 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543"
dest_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:35774
I20260709 13:05:32.645023 20742 raft_consensus.cc:493] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:32.645304 20742 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.650629 20742 raft_consensus.cc:515] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:32.652213 20742 leader_election.cc:290] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 3 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:32.653272 20592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:32.653329 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:32.653890 20592 raft_consensus.cc:3055] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:32.653964 20666 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.654183 20592 raft_consensus.cc:740] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:32.654927 20592 consensus_queue.cc:260] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:32.655838 20592 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.658687 20666 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 3.
I20260709 13:05:32.659828 20706 leader_election.cc:304] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:32.660403 20592 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 3.
I20260709 13:05:32.660597 20878 raft_consensus.cc:2804] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:32.661140 20878 raft_consensus.cc:697] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Becoming Leader. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:32.661871 20878 consensus_queue.cc:237] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:32.668745 20488 catalog_manager.cc:5697] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 reported cstate change: term changed from 2 to 3, leader changed from a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193) to 956d2eb6cb0a422998647448946aae00 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "956d2eb6cb0a422998647448946aae00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:33.089349 20592 raft_consensus.cc:1275] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:33.090720 20878 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:33.104152 20666 raft_consensus.cc:1275] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:33.105701 20666 raft_consensus.cc:1275] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:33.105924 20882 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:33.107164 20882 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:33.117178 20592 raft_consensus.cc:1275] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:33.119369 20878 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:33.228048 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:33.239356 20742 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823"
dest_uuid: "956d2eb6cb0a422998647448946aae00"
mode: GRACEFUL
new_leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:35782
I20260709 13:05:33.239943 20742 raft_consensus.cc:606] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Received request to transfer leadership to TS a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:33.258682 20742 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543"
dest_uuid: "956d2eb6cb0a422998647448946aae00"
mode: GRACEFUL
new_leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:35782
I20260709 13:05:33.259614 20742 raft_consensus.cc:606] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Received request to transfer leadership to TS a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:33.266093 20666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
mode: GRACEFUL
new_leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:55286
I20260709 13:05:33.266607 20666 raft_consensus.cc:606] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Received request to transfer leadership to TS a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:33.273525 20666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
mode: GRACEFUL
new_leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:55286
I20260709 13:05:33.274001 20666 raft_consensus.cc:606] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Received request to transfer leadership to TS a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:33.280531 20666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
mode: GRACEFUL
new_leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:55286
I20260709 13:05:33.280992 20666 raft_consensus.cc:606] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Received request to transfer leadership to TS a3a2ec2796c64f29a00145d5f75f7b62
I20260709 13:05:33.351711 20898 raft_consensus.cc:993] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73: : Instructing follower a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.352226 20808 raft_consensus.cc:1081] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Signalling peer a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.353915 20592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:33248
I20260709 13:05:33.354435 20592 raft_consensus.cc:493] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:33.354781 20592 raft_consensus.cc:3060] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.360369 20592 raft_consensus.cc:515] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.361793 20592 leader_election.cc:290] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 3 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:33.362733 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:33.362668 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d20e3ddf20eb4be997bddfa2d06bb90a" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:33.363341 20742 raft_consensus.cc:3060] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.363413 20666 raft_consensus.cc:3055] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:33.363816 20666 raft_consensus.cc:740] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:33.364523 20666 consensus_queue.cc:260] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.365645 20666 raft_consensus.cc:3060] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.369745 20742 raft_consensus.cc:2468] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 3.
I20260709 13:05:33.370944 20559 leader_election.cc:304] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:33.371723 20879 raft_consensus.cc:2804] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:33.371865 20666 raft_consensus.cc:2468] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 3.
I20260709 13:05:33.372937 20879 raft_consensus.cc:697] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:33.373706 20879 consensus_queue.cc:237] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.380283 20489 catalog_manager.cc:5697] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 2 to 3, leader changed from 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194) to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 3 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.559490 20891 raft_consensus.cc:993] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00: : Instructing follower a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.559974 20878 raft_consensus.cc:1081] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Signalling peer a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.561522 20592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:33256
I20260709 13:05:33.562000 20592 raft_consensus.cc:493] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:33.562336 20592 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:33.567790 20592 raft_consensus.cc:515] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.568151 20891 raft_consensus.cc:993] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00: : Instructing follower a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.568672 20891 raft_consensus.cc:1081] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Signalling peer a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.569734 20592 leader_election.cc:290] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 4 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:33.570622 20592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:33256
I20260709 13:05:33.570858 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:33.571144 20592 raft_consensus.cc:493] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:33.571427 20742 raft_consensus.cc:3055] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:33.571439 20898 raft_consensus.cc:993] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73: : Instructing follower a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.571537 20592 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:33.571877 20742 raft_consensus.cc:740] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:33.572135 20808 raft_consensus.cc:1081] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Signalling peer a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.571724 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:33.572753 20742 consensus_queue.cc:260] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.573257 20666 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:33.573777 20593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:33248
I20260709 13:05:33.573886 20742 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:33.574185 20593 raft_consensus.cc:493] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:33.574456 20593 raft_consensus.cc:3060] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.578425 20592 raft_consensus.cc:515] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.578969 20593 raft_consensus.cc:515] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.580629 20742 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 4.
I20260709 13:05:33.581414 20667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:33.582015 20667 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:33.582302 20559 leader_election.cc:304] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:33.583120 20879 raft_consensus.cc:2804] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:33.583629 20879 raft_consensus.cc:697] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:33.583820 20740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:33.584393 20740 raft_consensus.cc:3055] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:33.584231 20879 consensus_queue.cc:237] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.584718 20740 raft_consensus.cc:740] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:33.585549 20740 consensus_queue.cc:260] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.586642 20740 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:33.587672 20668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:33.588505 20668 raft_consensus.cc:3055] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:33.588830 20668 raft_consensus.cc:740] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:33.589761 20593 leader_election.cc:290] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 2 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:33.589612 20668 consensus_queue.cc:260] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.590751 20668 raft_consensus.cc:3060] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.591209 20489 catalog_manager.cc:5697] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 3 to 4, leader changed from 956d2eb6cb0a422998647448946aae00 (127.19.127.195) to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 4 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.593369 20667 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 4.
I20260709 13:05:33.594574 20557 leader_election.cc:304] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:33.595525 20879 raft_consensus.cc:2804] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:33.596016 20879 raft_consensus.cc:697] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:33.596822 20879 consensus_queue.cc:237] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.597935 20666 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 4.
I20260709 13:05:33.598165 20668 raft_consensus.cc:2468] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 2.
I20260709 13:05:33.599431 20557 leader_election.cc:304] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:33.590790 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:33.600337 20899 raft_consensus.cc:2804] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:33.600474 20742 raft_consensus.cc:3060] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.581554 20592 leader_election.cc:290] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 4 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:33.602579 20740 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 4.
I20260709 13:05:33.603371 20808 raft_consensus.cc:993] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73: : Instructing follower a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.603781 20808 raft_consensus.cc:1081] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Signalling peer a3a2ec2796c64f29a00145d5f75f7b62 to start an election
I20260709 13:05:33.604499 20899 raft_consensus.cc:697] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:33.605752 20593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
 from {username='slave'} at 127.0.0.1:33248
I20260709 13:05:33.605640 20899 consensus_queue.cc:237] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.606310 20593 raft_consensus.cc:493] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:33.606714 20593 raft_consensus.cc:3060] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.609413 20742 raft_consensus.cc:2468] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 2.
I20260709 13:05:33.612816 20593 raft_consensus.cc:515] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.614477 20593 leader_election.cc:290] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 3 election: Requested vote from peers 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:33.613902 20489 catalog_manager.cc:5697] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 3 to 4, leader changed from 956d2eb6cb0a422998647448946aae00 (127.19.127.195) to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 4 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.615320 20666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:33.615841 20666 raft_consensus.cc:3055] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:33.615690 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8334cae58bb4bf8872f48c6311c273a" candidate_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:33.616246 20666 raft_consensus.cc:740] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:33.615607 20489 catalog_manager.cc:5697] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 1 to 2, leader changed from 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194) to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.616396 20742 raft_consensus.cc:3060] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.616981 20666 consensus_queue.cc:260] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.617870 20666 raft_consensus.cc:3060] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.623510 20742 raft_consensus.cc:2468] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 3.
I20260709 13:05:33.624671 20666 raft_consensus.cc:2468] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2ec2796c64f29a00145d5f75f7b62 in term 3.
I20260709 13:05:33.625829 20557 leader_election.cc:304] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b8479b509c3477bb21a7f8eaa9fef73, a3a2ec2796c64f29a00145d5f75f7b62; no voters: 
I20260709 13:05:33.626654 20899 raft_consensus.cc:2804] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:33.627211 20899 raft_consensus.cc:697] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 LEADER]: Becoming Leader. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:33.628043 20899 consensus_queue.cc:237] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:33.634369 20489 catalog_manager.cc:5697] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 reported cstate change: term changed from 2 to 3, leader changed from 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194) to a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193). New cstate: current_term: 3 leader_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.795487 20742 raft_consensus.cc:1275] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:33.796905 20899 consensus_queue.cc:1048] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:33.807873 20666 raft_consensus.cc:1275] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:33.809892 20899 consensus_queue.cc:1048] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:34.030784 20666 raft_consensus.cc:1275] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:34.032073 20899 consensus_queue.cc:1048] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:34.041997 20742 raft_consensus.cc:1275] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:34.043881 20913 consensus_queue.cc:1048] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:34.049769 20740 raft_consensus.cc:1275] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:34.051162 20913 consensus_queue.cc:1048] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:34.053571 20666 raft_consensus.cc:1275] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 4 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:34.054949 20913 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:34.063357 20666 raft_consensus.cc:1275] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:34.066445 20740 raft_consensus.cc:1275] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 4 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:34.067626 20879 consensus_queue.cc:1048] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:34.068408 20899 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:34.104961 20668 raft_consensus.cc:1275] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 4 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:34.106158 20913 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:34.113126 20740 raft_consensus.cc:1275] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 4 FOLLOWER]: Refusing update from remote peer a3a2ec2796c64f29a00145d5f75f7b62: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:34.114571 20904 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:36.291038 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:36.299427 20593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
mode: GRACEFUL
new_leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:54760
I20260709 13:05:36.300186 20593 raft_consensus.cc:606] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Received request to transfer leadership to TS 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:36.303129 20593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
mode: GRACEFUL
new_leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:54760
I20260709 13:05:36.303764 20593 raft_consensus.cc:606] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Received request to transfer leadership to TS 9b8479b509c3477bb21a7f8eaa9fef73
I20260709 13:05:36.306443 20593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
mode: GRACEFUL
new_leader_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:54760
I20260709 13:05:36.306983 20593 raft_consensus.cc:606] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Received request to transfer leadership to TS 956d2eb6cb0a422998647448946aae00
I20260709 13:05:36.309813 20593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe2b9bdf278a40569029fb629711e517"
dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
mode: GRACEFUL
new_leader_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:54760
I20260709 13:05:36.310515 20593 raft_consensus.cc:606] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Received request to transfer leadership to TS 956d2eb6cb0a422998647448946aae00
I20260709 13:05:36.312002 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:05:36.312435 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:36.338965 20913 raft_consensus.cc:993] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62: : Instructing follower 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:36.339555 20904 raft_consensus.cc:1081] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Signalling peer 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:36.341243 20742 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543"
dest_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:35774
I20260709 13:05:36.341794 20742 raft_consensus.cc:493] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:36.342156 20742 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:36.347456 20742 raft_consensus.cc:515] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.349645 20742 leader_election.cc:290] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 5 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:36.351263 20668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:36.351168 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a75a764e8e6f4b20ac14649f9a94c543" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:36.352113 20668 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:36.352216 20593 raft_consensus.cc:3055] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:36.352600 20593 raft_consensus.cc:740] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:36.353471 20593 consensus_queue.cc:260] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.354496 20593 raft_consensus.cc:3060] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:36.358083 20668 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 5.
I20260709 13:05:36.359530 20706 leader_election.cc:304] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:36.360199 20593 raft_consensus.cc:2468] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 5.
I20260709 13:05:36.360512 20878 raft_consensus.cc:2804] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:36.361768 20878 raft_consensus.cc:697] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 5 LEADER]: Becoming Leader. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:36.362926 20878 consensus_queue.cc:237] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.370563 20489 catalog_manager.cc:5697] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 reported cstate change: term changed from 4 to 5, leader changed from a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193) to 956d2eb6cb0a422998647448946aae00 (127.19.127.195). New cstate: current_term: 5 leader_uuid: "956d2eb6cb0a422998647448946aae00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.584065 20913 raft_consensus.cc:993] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62: : Instructing follower 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:36.584530 20904 raft_consensus.cc:1081] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Signalling peer 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:36.586081 20668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:05:36.586575 20668 raft_consensus.cc:493] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:36.586874 20668 raft_consensus.cc:3060] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.592034 20668 raft_consensus.cc:515] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.593721 20668 leader_election.cc:290] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 3 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:36.594695 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:36.594833 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71cd8bd222646d3995a43f62d226b28" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:36.595362 20593 raft_consensus.cc:3055] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:36.595445 20742 raft_consensus.cc:3060] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.595752 20593 raft_consensus.cc:740] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:36.596565 20593 consensus_queue.cc:260] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.597527 20593 raft_consensus.cc:3060] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.600095 20742 raft_consensus.cc:2468] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 3.
I20260709 13:05:36.601354 20634 leader_election.cc:304] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:36.602174 20808 raft_consensus.cc:2804] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:36.602831 20593 raft_consensus.cc:2468] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 3.
I20260709 13:05:36.603459 20808 raft_consensus.cc:697] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 LEADER]: Becoming Leader. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:36.604441 20808 consensus_queue.cc:237] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.612350 20489 catalog_manager.cc:5697] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 reported cstate change: term changed from 2 to 3, leader changed from a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193) to 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194). New cstate: current_term: 3 leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.696942 20904 raft_consensus.cc:993] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62: : Instructing follower 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:36.697435 20913 raft_consensus.cc:1081] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Signalling peer 956d2eb6cb0a422998647448946aae00 to start an election
I20260709 13:05:36.698940 20742 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe2b9bdf278a40569029fb629711e517"
dest_uuid: "956d2eb6cb0a422998647448946aae00"
 from {username='slave'} at 127.0.0.1:35774
I20260709 13:05:36.699441 20742 raft_consensus.cc:493] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:36.699738 20742 raft_consensus.cc:3060] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.704003 20742 raft_consensus.cc:515] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.705410 20742 leader_election.cc:290] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 2 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733)
I20260709 13:05:36.706419 20668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2b9bdf278a40569029fb629711e517" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
I20260709 13:05:36.706481 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2b9bdf278a40569029fb629711e517" candidate_uuid: "956d2eb6cb0a422998647448946aae00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:36.707082 20668 raft_consensus.cc:3060] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.707135 20593 raft_consensus.cc:3055] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:36.707568 20593 raft_consensus.cc:740] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:36.708328 20593 consensus_queue.cc:260] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.709270 20593 raft_consensus.cc:3060] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.712489 20668 raft_consensus.cc:2468] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 2.
I20260709 13:05:36.713574 20706 leader_election.cc:304] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:36.714385 20878 raft_consensus.cc:2804] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:36.714473 20593 raft_consensus.cc:2468] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956d2eb6cb0a422998647448946aae00 in term 2.
I20260709 13:05:36.714910 20878 raft_consensus.cc:697] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 2 LEADER]: Becoming Leader. State: Replica: 956d2eb6cb0a422998647448946aae00, State: Running, Role: LEADER
I20260709 13:05:36.715909 20878 consensus_queue.cc:237] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.722414 20489 catalog_manager.cc:5697] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 reported cstate change: term changed from 1 to 2, leader changed from a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193) to 956d2eb6cb0a422998647448946aae00 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "956d2eb6cb0a422998647448946aae00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.757936 20904 raft_consensus.cc:993] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62: : Instructing follower 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:36.758389 20913 raft_consensus.cc:1081] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Signalling peer 9b8479b509c3477bb21a7f8eaa9fef73 to start an election
I20260709 13:05:36.759912 20668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823"
dest_uuid: "9b8479b509c3477bb21a7f8eaa9fef73"
 from {username='slave'} at 127.0.0.1:55264
I20260709 13:05:36.760437 20668 raft_consensus.cc:493] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:36.760754 20668 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:36.766222 20668 raft_consensus.cc:515] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.768036 20668 leader_election.cc:290] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 5 election: Requested vote from peers a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193:33571), 956d2eb6cb0a422998647448946aae00 (127.19.127.195:34945)
I20260709 13:05:36.768949 20742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "956d2eb6cb0a422998647448946aae00"
I20260709 13:05:36.768807 20593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17172d3dd9d0431ab73310e11af8a823" candidate_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a3a2ec2796c64f29a00145d5f75f7b62"
I20260709 13:05:36.769461 20742 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:36.769577 20593 raft_consensus.cc:3055] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:36.769989 20593 raft_consensus.cc:740] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3a2ec2796c64f29a00145d5f75f7b62, State: Running, Role: LEADER
I20260709 13:05:36.770740 20593 consensus_queue.cc:260] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.771715 20593 raft_consensus.cc:3060] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:36.774139 20742 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 5.
I20260709 13:05:36.775177 20634 leader_election.cc:304] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956d2eb6cb0a422998647448946aae00, 9b8479b509c3477bb21a7f8eaa9fef73; no voters: 
I20260709 13:05:36.775961 20808 raft_consensus.cc:2804] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:36.776392 20808 raft_consensus.cc:697] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 5 LEADER]: Becoming Leader. State: Replica: 9b8479b509c3477bb21a7f8eaa9fef73, State: Running, Role: LEADER
I20260709 13:05:36.776932 20593 raft_consensus.cc:2468] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8479b509c3477bb21a7f8eaa9fef73 in term 5.
I20260709 13:05:36.777452 20808 consensus_queue.cc:237] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } }
I20260709 13:05:36.785842 20489 catalog_manager.cc:5697] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 reported cstate change: term changed from 4 to 5, leader changed from a3a2ec2796c64f29a00145d5f75f7b62 (127.19.127.193) to 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194). New cstate: current_term: 5 leader_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.799808 20668 raft_consensus.cc:1275] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 5 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:05:36.800928 20878 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:36.811163 20593 raft_consensus.cc:1275] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 5 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:05:36.813894 20931 consensus_queue.cc:1048] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:37.032848 20742 raft_consensus.cc:1275] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:37.034022 20808 consensus_queue.cc:1048] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:37.043023 20593 raft_consensus.cc:1275] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:37.044515 20925 consensus_queue.cc:1048] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:37.197216 20593 raft_consensus.cc:1275] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 5 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:05:37.198518 20925 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:37.207652 20742 raft_consensus.cc:1275] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 5 FOLLOWER]: Refusing update from remote peer 9b8479b509c3477bb21a7f8eaa9fef73: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:05:37.209574 20808 consensus_queue.cc:1048] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d2eb6cb0a422998647448946aae00" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:37.279228 20668 raft_consensus.cc:1275] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:37.280622 20931 consensus_queue.cc:1048] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b8479b509c3477bb21a7f8eaa9fef73" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:37.291991 20593 raft_consensus.cc:1275] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Refusing update from remote peer 956d2eb6cb0a422998647448946aae00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:37.294723 20878 consensus_queue.cc:1048] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2ec2796c64f29a00145d5f75f7b62" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:37.313246 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:37.320370 19967 tablet_server.cc:179] TabletServer@127.19.127.193:0 shutting down...
I20260709 13:05:37.350473 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:37.351257 19967 tablet_replica.cc:333] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.352124 19967 raft_consensus.cc:2243] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.352968 19967 raft_consensus.cc:2272] T a9f33b8bce6a478cb6d1414801695d14 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.354857 19967 tablet_replica.cc:333] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.355538 19967 raft_consensus.cc:2243] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.355967 19967 raft_consensus.cc:2272] T a71cd8bd222646d3995a43f62d226b28 P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.358311 19967 tablet_replica.cc:333] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.358919 19967 raft_consensus.cc:2243] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.359462 19967 raft_consensus.cc:2272] T 17172d3dd9d0431ab73310e11af8a823 P a3a2ec2796c64f29a00145d5f75f7b62 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.362578 19967 tablet_replica.cc:333] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.363131 19967 raft_consensus.cc:2243] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.364112 19967 raft_consensus.cc:2272] T 0c8ac689b5ca413993647d45b5fbfd17 P a3a2ec2796c64f29a00145d5f75f7b62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.366173 19967 tablet_replica.cc:333] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.366734 19967 raft_consensus.cc:2243] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.367647 19967 raft_consensus.cc:2272] T d20e3ddf20eb4be997bddfa2d06bb90a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.369339 19967 tablet_replica.cc:333] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.369860 19967 raft_consensus.cc:2243] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.370584 19967 raft_consensus.cc:2272] T d8334cae58bb4bf8872f48c6311c273a P a3a2ec2796c64f29a00145d5f75f7b62 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.372309 19967 tablet_replica.cc:333] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.372816 19967 raft_consensus.cc:2243] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.373260 19967 raft_consensus.cc:2272] T 0b972ae2bc69426aa33cd855b1ae4c1f P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.374989 19967 tablet_replica.cc:333] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.375558 19967 raft_consensus.cc:2243] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.375995 19967 raft_consensus.cc:2272] T 4b79b8b205834585a55c92a7a0eaf03e P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.377708 19967 tablet_replica.cc:333] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.378202 19967 raft_consensus.cc:2243] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.378607 19967 raft_consensus.cc:2272] T a75a764e8e6f4b20ac14649f9a94c543 P a3a2ec2796c64f29a00145d5f75f7b62 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.380884 19967 tablet_replica.cc:333] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62: stopping tablet replica
I20260709 13:05:37.381387 19967 raft_consensus.cc:2243] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.381805 19967 raft_consensus.cc:2272] T fe2b9bdf278a40569029fb629711e517 P a3a2ec2796c64f29a00145d5f75f7b62 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.404600 19967 tablet_server.cc:196] TabletServer@127.19.127.193:0 shutdown complete.
I20260709 13:05:37.426312 19967 tablet_server.cc:179] TabletServer@127.19.127.194:0 shutting down...
I20260709 13:05:37.455363 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:37.456205 19967 tablet_replica.cc:333] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.456946 19967 raft_consensus.cc:2243] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.457486 19967 raft_consensus.cc:2272] T 0b972ae2bc69426aa33cd855b1ae4c1f P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.459467 19967 tablet_replica.cc:333] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.460027 19967 raft_consensus.cc:2243] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.460803 19967 raft_consensus.cc:2272] T 4b79b8b205834585a55c92a7a0eaf03e P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.462733 19967 tablet_replica.cc:333] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.463255 19967 raft_consensus.cc:2243] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.463821 19967 raft_consensus.cc:2272] T fe2b9bdf278a40569029fb629711e517 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.466120 19967 tablet_replica.cc:333] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.466660 19967 raft_consensus.cc:2243] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.467139 19967 raft_consensus.cc:2272] T a75a764e8e6f4b20ac14649f9a94c543 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.469220 19967 tablet_replica.cc:333] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.469748 19967 raft_consensus.cc:2243] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.470189 19967 raft_consensus.cc:2272] T d8334cae58bb4bf8872f48c6311c273a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.472026 19967 tablet_replica.cc:333] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.472520 19967 raft_consensus.cc:2243] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.472940 19967 raft_consensus.cc:2272] T d20e3ddf20eb4be997bddfa2d06bb90a P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.474530 19967 tablet_replica.cc:333] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.475034 19967 raft_consensus.cc:2243] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.475538 19967 raft_consensus.cc:2272] T a9f33b8bce6a478cb6d1414801695d14 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.477072 19967 tablet_replica.cc:333] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.477541 19967 raft_consensus.cc:2243] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.478250 19967 raft_consensus.cc:2272] T a71cd8bd222646d3995a43f62d226b28 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.480387 19967 tablet_replica.cc:333] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.480901 19967 raft_consensus.cc:2243] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.481335 19967 raft_consensus.cc:2272] T 0c8ac689b5ca413993647d45b5fbfd17 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.483136 19967 tablet_replica.cc:333] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73: stopping tablet replica
I20260709 13:05:37.483611 19967 raft_consensus.cc:2243] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.484372 19967 raft_consensus.cc:2272] T 17172d3dd9d0431ab73310e11af8a823 P 9b8479b509c3477bb21a7f8eaa9fef73 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.514184 19967 tablet_server.cc:196] TabletServer@127.19.127.194:0 shutdown complete.
I20260709 13:05:37.533671 19967 tablet_server.cc:179] TabletServer@127.19.127.195:0 shutting down...
W20260709 13:05:37.546403 20706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.127.194:42733: connect: Connection refused (error 111) [suppressed 4 similar messages]
W20260709 13:05:37.550850 20706 consensus_peers.cc:597] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 -> Peer 9b8479b509c3477bb21a7f8eaa9fef73 (127.19.127.194:42733): Couldn't send request to peer 9b8479b509c3477bb21a7f8eaa9fef73. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:42733: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.565935 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:37.566722 19967 tablet_replica.cc:333] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.567674 19967 raft_consensus.cc:2243] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.568320 19967 raft_consensus.cc:2272] T a9f33b8bce6a478cb6d1414801695d14 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.570101 19967 tablet_replica.cc:333] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.570583 19967 raft_consensus.cc:2243] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.570959 19967 raft_consensus.cc:2272] T a71cd8bd222646d3995a43f62d226b28 P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.573033 19967 tablet_replica.cc:333] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.573510 19967 raft_consensus.cc:2243] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.574229 19967 raft_consensus.cc:2272] T 0b972ae2bc69426aa33cd855b1ae4c1f P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.576064 19967 tablet_replica.cc:333] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.576539 19967 raft_consensus.cc:2243] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.576941 19967 raft_consensus.cc:2272] T d8334cae58bb4bf8872f48c6311c273a P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.578576 19967 tablet_replica.cc:333] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.579048 19967 raft_consensus.cc:2243] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.579523 19967 raft_consensus.cc:2272] T d20e3ddf20eb4be997bddfa2d06bb90a P 956d2eb6cb0a422998647448946aae00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.581163 19967 tablet_replica.cc:333] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.581609 19967 raft_consensus.cc:2243] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.581977 19967 raft_consensus.cc:2272] T 17172d3dd9d0431ab73310e11af8a823 P 956d2eb6cb0a422998647448946aae00 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.584589 19967 tablet_replica.cc:333] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.585052 19967 raft_consensus.cc:2243] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.585417 19967 raft_consensus.cc:2272] T 0c8ac689b5ca413993647d45b5fbfd17 P 956d2eb6cb0a422998647448946aae00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.587209 19967 tablet_replica.cc:333] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.587746 19967 raft_consensus.cc:2243] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.588371 19967 raft_consensus.cc:2272] T a75a764e8e6f4b20ac14649f9a94c543 P 956d2eb6cb0a422998647448946aae00 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.590306 19967 tablet_replica.cc:333] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.590735 19967 raft_consensus.cc:2243] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.591121 19967 raft_consensus.cc:2272] T 4b79b8b205834585a55c92a7a0eaf03e P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.592759 19967 tablet_replica.cc:333] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00: stopping tablet replica
I20260709 13:05:37.593151 19967 raft_consensus.cc:2243] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.593802 19967 raft_consensus.cc:2272] T fe2b9bdf278a40569029fb629711e517 P 956d2eb6cb0a422998647448946aae00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.622711 19967 tablet_server.cc:196] TabletServer@127.19.127.195:0 shutdown complete.
I20260709 13:05:37.640834 19967 master.cc:562] Master@127.19.127.254:33007 shutting down...
I20260709 13:05:37.661188 19967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.661711 19967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.662155 19967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8cfa489b35414a3f80aa1415ec73685b: stopping tablet replica
I20260709 13:05:37.681896 19967 master.cc:584] Master@127.19.127.254:33007 shutdown complete.
956d2eb6cb0a422998647448946aae00  1
9b8479b509c3477bb21a7f8eaa9fef73  4
a3a2ec2796c64f29a00145d5f75f7b62  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10589 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:05:37.722551 19967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.127.254:37743
I20260709 13:05:37.723680 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:37.729271 20946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:37.729333 20945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:37.730170 20948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:37.731586 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:37.732913 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:37.733095 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:37.733208 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602337733197 us; error 0 us; skew 500 ppm
I20260709 13:05:37.733675 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:37.736120 19967 webserver.cc:533] Webserver started at http://127.19.127.254:36993/ using document root <none> and password file <none>
I20260709 13:05:37.736565 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:37.736744 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:37.736977 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:37.738245 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/master-0-root/instance:
uuid: "b068e036d3f142f4b255651ac6a0e101"
format_stamp: "Formatted at 2026-07-09 13:05:37 on dist-test-slave-s3n6"
I20260709 13:05:37.742682 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:05:37.746137 20953 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:37.746975 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:05:37.747372 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/master-0-root
uuid: "b068e036d3f142f4b255651ac6a0e101"
format_stamp: "Formatted at 2026-07-09 13:05:37 on dist-test-slave-s3n6"
I20260709 13:05:37.747687 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:37.772275 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:37.773568 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:37.815622 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.254:37743
I20260709 13:05:37.815748 21004 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.254:37743 every 8 connection(s)
I20260709 13:05:37.819676 21005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:37.830597 21005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Bootstrap starting.
I20260709 13:05:37.835742 21005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:37.840482 21005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: No bootstrap required, opened a new log
I20260709 13:05:37.842869 21005 raft_consensus.cc:359] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068e036d3f142f4b255651ac6a0e101" member_type: VOTER }
I20260709 13:05:37.843328 21005 raft_consensus.cc:385] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:37.843662 21005 raft_consensus.cc:740] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b068e036d3f142f4b255651ac6a0e101, State: Initialized, Role: FOLLOWER
I20260709 13:05:37.844342 21005 consensus_queue.cc:260] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068e036d3f142f4b255651ac6a0e101" member_type: VOTER }
I20260709 13:05:37.844877 21005 raft_consensus.cc:399] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:37.845119 21005 raft_consensus.cc:493] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:37.845456 21005 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.850420 21005 raft_consensus.cc:515] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068e036d3f142f4b255651ac6a0e101" member_type: VOTER }
I20260709 13:05:37.851078 21005 leader_election.cc:304] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b068e036d3f142f4b255651ac6a0e101; no voters: 
I20260709 13:05:37.852491 21005 leader_election.cc:290] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:37.852813 21008 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:37.854386 21008 raft_consensus.cc:697] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 1 LEADER]: Becoming Leader. State: Replica: b068e036d3f142f4b255651ac6a0e101, State: Running, Role: LEADER
I20260709 13:05:37.855214 21008 consensus_queue.cc:237] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068e036d3f142f4b255651ac6a0e101" member_type: VOTER }
I20260709 13:05:37.855969 21005 sys_catalog.cc:565] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:37.861196 21009 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b068e036d3f142f4b255651ac6a0e101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068e036d3f142f4b255651ac6a0e101" member_type: VOTER } }
I20260709 13:05:37.861196 21010 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b068e036d3f142f4b255651ac6a0e101. Latest consensus state: current_term: 1 leader_uuid: "b068e036d3f142f4b255651ac6a0e101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068e036d3f142f4b255651ac6a0e101" member_type: VOTER } }
I20260709 13:05:37.861941 21009 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:37.862254 21010 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:37.864540 21014 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:37.873914 21014 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:37.874976 19967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:37.883067 21014 catalog_manager.cc:1383] Generated new cluster ID: a4ea50221a6f4e76b6d7b79f7e56cab5
I20260709 13:05:37.883476 21014 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:37.905287 21014 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:37.906564 21014 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:37.918524 21014 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Generated new TSK 0
I20260709 13:05:37.919210 21014 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:37.941715 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:37.948194 21026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:37.949407 21027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:37.951453 19967 server_base.cc:1061] running on GCE node
W20260709 13:05:37.951529 21029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:37.952813 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:37.953030 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:37.953235 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602337953219 us; error 0 us; skew 500 ppm
I20260709 13:05:37.953964 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:37.956512 19967 webserver.cc:533] Webserver started at http://127.19.127.193:37015/ using document root <none> and password file <none>
I20260709 13:05:37.957006 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:37.957213 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:37.957530 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:37.958699 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/instance:
uuid: "9384358ce43f4e3fac2ae44bdf11c389"
format_stamp: "Formatted at 2026-07-09 13:05:37 on dist-test-slave-s3n6"
I20260709 13:05:37.963635 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:05:37.967578 21034 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:37.968488 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:05:37.968753 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "9384358ce43f4e3fac2ae44bdf11c389"
format_stamp: "Formatted at 2026-07-09 13:05:37 on dist-test-slave-s3n6"
I20260709 13:05:37.969027 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:37.996114 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:37.997383 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:37.999270 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:38.001950 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:38.002147 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:38.002341 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:38.002492 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:38.045831 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:42843
I20260709 13:05:38.045949 21096 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:42843 every 8 connection(s)
I20260709 13:05:38.051121 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:38.058320 21101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:38.059175 21102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:38.061121 21104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:38.063225 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:38.064894 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:38.065187 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:38.065404 21097 heartbeater.cc:344] Connected to a master server at 127.19.127.254:37743
I20260709 13:05:38.065462 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602338065441 us; error 0 us; skew 500 ppm
I20260709 13:05:38.065924 21097 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:38.066357 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:38.066805 21097 heartbeater.cc:507] Master 127.19.127.254:37743 requested a full tablet report, sending...
I20260709 13:05:38.069293 20970 ts_manager.cc:194] Registered new tserver with Master: 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:05:38.069650 19967 webserver.cc:533] Webserver started at http://127.19.127.194:36521/ using document root <none> and password file <none>
I20260709 13:05:38.070233 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:38.070446 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:38.070719 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:38.071818 20970 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55748
I20260709 13:05:38.071908 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/instance:
uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
format_stamp: "Formatted at 2026-07-09 13:05:38 on dist-test-slave-s3n6"
I20260709 13:05:38.076675 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:05:38.080113 21109 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:38.080986 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:05:38.081279 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root
uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
format_stamp: "Formatted at 2026-07-09 13:05:38 on dist-test-slave-s3n6"
I20260709 13:05:38.081609 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:38.100217 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:38.101518 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:38.103174 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:38.105597 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:38.105858 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:38.106119 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:38.106308 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:38.148124 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.194:39351
I20260709 13:05:38.148233 21171 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.194:39351 every 8 connection(s)
I20260709 13:05:38.153291 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:38.161403 21175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:38.163256 21176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:38.165129 21178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:38.165974 21172 heartbeater.cc:344] Connected to a master server at 127.19.127.254:37743
I20260709 13:05:38.166201 19967 server_base.cc:1061] running on GCE node
I20260709 13:05:38.166484 21172 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:38.167255 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:05:38.167420 21172 heartbeater.cc:507] Master 127.19.127.254:37743 requested a full tablet report, sending...
W20260709 13:05:38.167560 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:38.167889 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602338167867 us; error 0 us; skew 500 ppm
I20260709 13:05:38.168565 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:38.169935 20970 ts_manager.cc:194] Registered new tserver with Master: 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:38.171463 19967 webserver.cc:533] Webserver started at http://127.19.127.195:44267/ using document root <none> and password file <none>
I20260709 13:05:38.171672 20970 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55764
I20260709 13:05:38.172020 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:38.172258 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:38.172610 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:38.174166 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/instance:
uuid: "46c1676121b64b7d9bcf2227cc639dd3"
format_stamp: "Formatted at 2026-07-09 13:05:38 on dist-test-slave-s3n6"
I20260709 13:05:38.178651 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.001s
I20260709 13:05:38.182081 21183 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:38.182976 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:05:38.183364 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root
uuid: "46c1676121b64b7d9bcf2227cc639dd3"
format_stamp: "Formatted at 2026-07-09 13:05:38 on dist-test-slave-s3n6"
I20260709 13:05:38.183651 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:38.195400 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:38.196656 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:38.198220 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:38.200899 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:38.201165 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:38.201429 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:38.201609 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:38.241937 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.195:46249
I20260709 13:05:38.242079 21245 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.195:46249 every 8 connection(s)
I20260709 13:05:38.256629 21246 heartbeater.cc:344] Connected to a master server at 127.19.127.254:37743
I20260709 13:05:38.257181 21246 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:38.258129 21246 heartbeater.cc:507] Master 127.19.127.254:37743 requested a full tablet report, sending...
I20260709 13:05:38.260272 20970 ts_manager.cc:194] Registered new tserver with Master: 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:38.261289 19967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016100118s
I20260709 13:05:38.261840 20970 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55780
I20260709 13:05:38.290940 20970 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55790:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:38.297578 20970 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:38.527733 21211 tablet_service.cc:1511] Processing CreateTablet for tablet 47de8d16a3804364a7fbc9bf598946c2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:38.529316 21211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47de8d16a3804364a7fbc9bf598946c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.535121 21062 tablet_service.cc:1511] Processing CreateTablet for tablet 47de8d16a3804364a7fbc9bf598946c2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:38.536748 21062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47de8d16a3804364a7fbc9bf598946c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.555087 21207 tablet_service.cc:1511] Processing CreateTablet for tablet b7a0bb40f8e3428899935fdfa915dfd0 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:38.556576 21207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a0bb40f8e3428899935fdfa915dfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.556638 21129 tablet_service.cc:1511] Processing CreateTablet for tablet ce55364f10c348819e7c63c08633b184 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:38.558118 21129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce55364f10c348819e7c63c08633b184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.538432 21061 tablet_service.cc:1511] Processing CreateTablet for tablet 54fc5a3f4db84eac831870d621ddfa54 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:38.559456 21061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54fc5a3f4db84eac831870d621ddfa54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.560832 21131 tablet_service.cc:1511] Processing CreateTablet for tablet 5196cb9fb3c1488180a6421083b18ff9 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:38.562316 21131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5196cb9fb3c1488180a6421083b18ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.566038 21206 tablet_service.cc:1511] Processing CreateTablet for tablet 9833b15c99c34c269ca3afa7dac3f175 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:38.540525 21059 tablet_service.cc:1511] Processing CreateTablet for tablet e714d9095b9c430bba199e0209c0b946 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:38.542582 21202 tablet_service.cc:1511] Processing CreateTablet for tablet d693d94d4c0943d9ab08114c33fc2e7f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:38.542483 21058 tablet_service.cc:1511] Processing CreateTablet for tablet b7a0bb40f8e3428899935fdfa915dfd0 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:38.544489 21057 tablet_service.cc:1511] Processing CreateTablet for tablet 9833b15c99c34c269ca3afa7dac3f175 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:38.544489 21203 tablet_service.cc:1511] Processing CreateTablet for tablet ce55364f10c348819e7c63c08633b184 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:38.545733 21204 tablet_service.cc:1511] Processing CreateTablet for tablet 9417f68ba15d49c6915ab46793aab93c (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:38.546824 21137 tablet_service.cc:1511] Processing CreateTablet for tablet 47de8d16a3804364a7fbc9bf598946c2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:38.547530 21136 tablet_service.cc:1511] Processing CreateTablet for tablet 54fc5a3f4db84eac831870d621ddfa54 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:38.548543 21055 tablet_service.cc:1511] Processing CreateTablet for tablet 9417f68ba15d49c6915ab46793aab93c (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:38.549239 21135 tablet_service.cc:1511] Processing CreateTablet for tablet 8b6dd6862dd14baba4a82c5a837bc802 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:38.549273 21133 tablet_service.cc:1511] Processing CreateTablet for tablet b7a0bb40f8e3428899935fdfa915dfd0 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:38.550307 21209 tablet_service.cc:1511] Processing CreateTablet for tablet 8b6dd6862dd14baba4a82c5a837bc802 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:38.551172 21134 tablet_service.cc:1511] Processing CreateTablet for tablet e714d9095b9c430bba199e0209c0b946 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:38.552194 21054 tablet_service.cc:1511] Processing CreateTablet for tablet ce55364f10c348819e7c63c08633b184 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:38.553099 21128 tablet_service.cc:1511] Processing CreateTablet for tablet d693d94d4c0943d9ab08114c33fc2e7f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:38.554103 21053 tablet_service.cc:1511] Processing CreateTablet for tablet d693d94d4c0943d9ab08114c33fc2e7f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:38.554893 21208 tablet_service.cc:1511] Processing CreateTablet for tablet e714d9095b9c430bba199e0209c0b946 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:38.535221 21210 tablet_service.cc:1511] Processing CreateTablet for tablet 54fc5a3f4db84eac831870d621ddfa54 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:38.574836 21205 tablet_service.cc:1511] Processing CreateTablet for tablet 5196cb9fb3c1488180a6421083b18ff9 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:38.576309 21137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47de8d16a3804364a7fbc9bf598946c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.581440 21060 tablet_service.cc:1511] Processing CreateTablet for tablet 8b6dd6862dd14baba4a82c5a837bc802 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:38.582789 21130 tablet_service.cc:1511] Processing CreateTablet for tablet 9417f68ba15d49c6915ab46793aab93c (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:38.580188 21134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e714d9095b9c430bba199e0209c0b946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.584586 21128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d693d94d4c0943d9ab08114c33fc2e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.585197 21135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6dd6862dd14baba4a82c5a837bc802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.586392 21136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54fc5a3f4db84eac831870d621ddfa54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.569729 21132 tablet_service.cc:1511] Processing CreateTablet for tablet 9833b15c99c34c269ca3afa7dac3f175 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:38.590533 21133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a0bb40f8e3428899935fdfa915dfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.591478 21132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9833b15c99c34c269ca3afa7dac3f175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.570863 21056 tablet_service.cc:1511] Processing CreateTablet for tablet 5196cb9fb3c1488180a6421083b18ff9 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:38.601418 21130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9417f68ba15d49c6915ab46793aab93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.602823 21059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e714d9095b9c430bba199e0209c0b946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.603565 21058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a0bb40f8e3428899935fdfa915dfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.571277 21206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9833b15c99c34c269ca3afa7dac3f175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.604359 21202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d693d94d4c0943d9ab08114c33fc2e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.607928 21054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce55364f10c348819e7c63c08633b184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.611994 21053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d693d94d4c0943d9ab08114c33fc2e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.613158 21208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e714d9095b9c430bba199e0209c0b946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.613443 21060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6dd6862dd14baba4a82c5a837bc802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.615470 21210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54fc5a3f4db84eac831870d621ddfa54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.619972 21055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9417f68ba15d49c6915ab46793aab93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.620010 21205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5196cb9fb3c1488180a6421083b18ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.623519 21204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9417f68ba15d49c6915ab46793aab93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.623875 21057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9833b15c99c34c269ca3afa7dac3f175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.628227 21056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5196cb9fb3c1488180a6421083b18ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.629200 21209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6dd6862dd14baba4a82c5a837bc802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.633661 21203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce55364f10c348819e7c63c08633b184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.661047 21207 tablet_service.cc:1511] Processing CreateTablet for tablet 12e3a58d24d64816bce2752c6adb7e33 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:38.662945 21207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12e3a58d24d64816bce2752c6adb7e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.666659 21262 tablet_bootstrap.cc:492] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:38.667498 21062 tablet_service.cc:1511] Processing CreateTablet for tablet 12e3a58d24d64816bce2752c6adb7e33 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:38.669023 21261 tablet_bootstrap.cc:492] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:38.670639 21211 tablet_service.cc:1511] Processing CreateTablet for tablet cc53d52fdeaa441b9e9d329e69d94440 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:38.672071 21211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc53d52fdeaa441b9e9d329e69d94440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.669035 21062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12e3a58d24d64816bce2752c6adb7e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.687983 21261 tablet_bootstrap.cc:654] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.694893 21205 tablet_service.cc:1511] Processing CreateTablet for tablet 9028fa09ccd24f5494e4e5de5a8e6e18 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:38.696825 21205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9028fa09ccd24f5494e4e5de5a8e6e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.698367 21054 tablet_service.cc:1511] Processing CreateTablet for tablet cc53d52fdeaa441b9e9d329e69d94440 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:38.700500 21210 tablet_service.cc:1511] Processing CreateTablet for tablet a473266451804da089da06e9c927e7ee (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:38.701924 21210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a473266451804da089da06e9c927e7ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.711882 21054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc53d52fdeaa441b9e9d329e69d94440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.720039 21262 tablet_bootstrap.cc:654] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.726256 21261 tablet_bootstrap.cc:492] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:38.726745 21261 ts_tablet_manager.cc:1403] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.000s
I20260709 13:05:38.729684 21261 raft_consensus.cc:359] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:38.730432 21261 raft_consensus.cc:385] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.730729 21261 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.737532 21062 tablet_service.cc:1511] Processing CreateTablet for tablet 9028fa09ccd24f5494e4e5de5a8e6e18 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:38.738886 21062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9028fa09ccd24f5494e4e5de5a8e6e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.744369 21261 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:38.748065 21262 tablet_bootstrap.cc:492] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:38.748600 21262 ts_tablet_manager.cc:1403] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.082s	user 0.013s	sys 0.001s
I20260709 13:05:38.751732 21262 raft_consensus.cc:359] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:38.752496 21262 raft_consensus.cc:385] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.752794 21262 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.759413 21246 heartbeater.cc:499] Master 127.19.127.254:37743 was elected leader, sending a full tablet report...
I20260709 13:05:38.760491 21261 ts_tablet_manager.cc:1434] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:05:38.761569 21261 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:38.769122 21211 tablet_service.cc:1511] Processing CreateTablet for tablet 1ea0f23227964b96bff6287436e970cd (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:38.770651 21262 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:38.776546 21211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ea0f23227964b96bff6287436e970cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.778374 21137 tablet_service.cc:1511] Processing CreateTablet for tablet 12e3a58d24d64816bce2752c6adb7e33 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:38.779491 21128 tablet_service.cc:1511] Processing CreateTablet for tablet cc53d52fdeaa441b9e9d329e69d94440 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:38.783758 21261 tablet_bootstrap.cc:654] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.833264 21062 tablet_service.cc:1511] Processing CreateTablet for tablet d8a4c442e73243eda44d43ba0eaf0d2a (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:38.834587 21062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a4c442e73243eda44d43ba0eaf0d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.836345 21097 heartbeater.cc:499] Master 127.19.127.254:37743 was elected leader, sending a full tablet report...
I20260709 13:05:38.840197 21262 ts_tablet_manager.cc:1434] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.091s	user 0.017s	sys 0.022s
I20260709 13:05:38.841295 21262 tablet_bootstrap.cc:492] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:38.841848 21056 tablet_service.cc:1511] Processing CreateTablet for tablet de27148f295b4ef5a3f79084b8ed5ae6 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:38.844473 21057 tablet_service.cc:1511] Processing CreateTablet for tablet 731bbc776723436d954fe0b2a30a5b09 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:38.786034 21058 tablet_service.cc:1511] Processing CreateTablet for tablet a473266451804da089da06e9c927e7ee (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:38.785099 21205 tablet_service.cc:1511] Processing CreateTablet for tablet 5a16f225ecc04514a485518bcd2b8112 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:38.846269 21205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a16f225ecc04514a485518bcd2b8112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.793496 21203 tablet_service.cc:1511] Processing CreateTablet for tablet 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:38.850018 21203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d5d573fe0fd4b91af3ea4dd4f6dc7f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.849850 21059 tablet_service.cc:1511] Processing CreateTablet for tablet 0d78b35ac92d402db8a5e3fe45be52b5 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:38.855172 21134 tablet_service.cc:1511] Processing CreateTablet for tablet 1ea0f23227964b96bff6287436e970cd (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:38.797688 21053 tablet_service.cc:1511] Processing CreateTablet for tablet 6e4456f3a9364c7289dfce3163ab4760 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:38.759109 21207 tablet_service.cc:1511] Processing CreateTablet for tablet 6e4456f3a9364c7289dfce3163ab4760 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:38.859728 21209 tablet_service.cc:1511] Processing CreateTablet for tablet 169e564bd720422bb798c652af4fb2eb (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:38.802606 21202 tablet_service.cc:1511] Processing CreateTablet for tablet d8a4c442e73243eda44d43ba0eaf0d2a (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:38.803189 21204 tablet_service.cc:1511] Processing CreateTablet for tablet de27148f295b4ef5a3f79084b8ed5ae6 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:38.843266 21056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de27148f295b4ef5a3f79084b8ed5ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.809696 21210 tablet_service.cc:1511] Processing CreateTablet for tablet 0d78b35ac92d402db8a5e3fe45be52b5 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:38.814576 21211 tablet_service.cc:1511] Processing CreateTablet for tablet 731bbc776723436d954fe0b2a30a5b09 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:38.815886 21137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12e3a58d24d64816bce2752c6adb7e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.861263 21062 tablet_service.cc:1511] Processing CreateTablet for tablet 53c918605cdf4c39b832a695968308a3 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:38.862195 21053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e4456f3a9364c7289dfce3163ab4760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.863575 21057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731bbc776723436d954fe0b2a30a5b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.864588 21130 tablet_service.cc:1511] Processing CreateTablet for tablet a473266451804da089da06e9c927e7ee (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:38.824841 21135 tablet_service.cc:1511] Processing CreateTablet for tablet 6e4456f3a9364c7289dfce3163ab4760 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:38.826159 21054 tablet_service.cc:1511] Processing CreateTablet for tablet 5a16f225ecc04514a485518bcd2b8112 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:38.827847 21206 tablet_service.cc:1511] Processing CreateTablet for tablet 53c918605cdf4c39b832a695968308a3 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:38.827834 21060 tablet_service.cc:1511] Processing CreateTablet for tablet 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:38.789070 21132 tablet_service.cc:1511] Processing CreateTablet for tablet 9028fa09ccd24f5494e4e5de5a8e6e18 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:38.783005 21208 tablet_service.cc:1511] Processing CreateTablet for tablet c80e14318ee44feba007795498031464 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:38.860162 21207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e4456f3a9364c7289dfce3163ab4760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.801174 21061 tablet_service.cc:1511] Processing CreateTablet for tablet 1ea0f23227964b96bff6287436e970cd (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:38.804719 21055 tablet_service.cc:1511] Processing CreateTablet for tablet c80e14318ee44feba007795498031464 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:38.866670 21062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c918605cdf4c39b832a695968308a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.868482 21129 tablet_service.cc:1511] Processing CreateTablet for tablet c80e14318ee44feba007795498031464 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:38.908953 21058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a473266451804da089da06e9c927e7ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.871166 21263 tablet_bootstrap.cc:492] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:38.909981 21055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c80e14318ee44feba007795498031464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.914896 21262 tablet_bootstrap.cc:654] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:38.869364 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:38.889665 21261 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:38.915719 21059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d78b35ac92d402db8a5e3fe45be52b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.897423 21136 tablet_service.cc:1511] Processing CreateTablet for tablet 5a16f225ecc04514a485518bcd2b8112 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:38.916064 21261 ts_tablet_manager.cc:1403] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.155s	user 0.017s	sys 0.002s
I20260709 13:05:38.916355 21054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a16f225ecc04514a485518bcd2b8112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.917090 21210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d78b35ac92d402db8a5e3fe45be52b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.918116 21208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c80e14318ee44feba007795498031464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.918776 21211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731bbc776723436d954fe0b2a30a5b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.919584 21202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a4c442e73243eda44d43ba0eaf0d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.919246 21261 raft_consensus.cc:359] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:38.920046 21261 raft_consensus.cc:385] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:38.920197 21204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de27148f295b4ef5a3f79084b8ed5ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.920384 21261 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:38.924106 21206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c918605cdf4c39b832a695968308a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.925741 21261 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:38.928608 21060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d5d573fe0fd4b91af3ea4dd4f6dc7f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.931689 21209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169e564bd720422bb798c652af4fb2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.932070 21061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ea0f23227964b96bff6287436e970cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.862160 21128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc53d52fdeaa441b9e9d329e69d94440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.935317 21267 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.869746 21131 tablet_service.cc:1511] Processing CreateTablet for tablet 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:38.935971 21267 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:38.936740 21131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d5d573fe0fd4b91af3ea4dd4f6dc7f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.939628 21136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a16f225ecc04514a485518bcd2b8112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.940356 21203 tablet_service.cc:1511] Processing CreateTablet for tablet 223e82a164f445c2b46667d58a5d2d1f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:38.941092 21261 ts_tablet_manager.cc:1434] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.025s	user 0.002s	sys 0.005s
I20260709 13:05:38.941728 21203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223e82a164f445c2b46667d58a5d2d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.942378 21261 tablet_bootstrap.cc:492] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:38.943702 21129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c80e14318ee44feba007795498031464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.943017 21267 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:38.945102 21135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e4456f3a9364c7289dfce3163ab4760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.949434 21134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ea0f23227964b96bff6287436e970cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.950918 21205 tablet_service.cc:1511] Processing CreateTablet for tablet 72edcb8f7f3d437b8e7bb24c612600ab (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:38.951901 21130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a473266451804da089da06e9c927e7ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.952302 21205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72edcb8f7f3d437b8e7bb24c612600ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:38.961823 21263 tablet_bootstrap.cc:654] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.963053 21261 tablet_bootstrap.cc:654] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:38.981511 21133 tablet_service.cc:1511] Processing CreateTablet for tablet d8a4c442e73243eda44d43ba0eaf0d2a (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:38.990718 21058 tablet_service.cc:1511] Processing CreateTablet for tablet 169e564bd720422bb798c652af4fb2eb (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:38.997668 21132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9028fa09ccd24f5494e4e5de5a8e6e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.004287 21055 tablet_service.cc:1511] Processing CreateTablet for tablet 223e82a164f445c2b46667d58a5d2d1f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:39.041796 21133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a4c442e73243eda44d43ba0eaf0d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.061157 21129 tablet_service.cc:1511] Processing CreateTablet for tablet de27148f295b4ef5a3f79084b8ed5ae6 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:39.061321 21056 tablet_service.cc:1511] Processing CreateTablet for tablet 72edcb8f7f3d437b8e7bb24c612600ab (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:39.057821 21058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169e564bd720422bb798c652af4fb2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.073050 21262 tablet_bootstrap.cc:492] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:39.086558 21062 tablet_service.cc:1511] Processing CreateTablet for tablet b8d00e740dba47b2b43be9e4ef1ecb55 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:39.102111 21262 ts_tablet_manager.cc:1403] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.261s	user 0.011s	sys 0.020s
I20260709 13:05:39.104482 21210 tablet_service.cc:1511] Processing CreateTablet for tablet 080f4010b39e40a8b8efb7280faec38f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:39.105196 21262 raft_consensus.cc:359] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.105878 21210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080f4010b39e40a8b8efb7280faec38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.107738 21266 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.107991 21131 tablet_service.cc:1511] Processing CreateTablet for tablet 731bbc776723436d954fe0b2a30a5b09 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:39.108213 21266 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.109328 21062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8d00e740dba47b2b43be9e4ef1ecb55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.105919 21262 raft_consensus.cc:385] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.111702 21262 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.112562 21262 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.114627 21266 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:39.093602 21054 tablet_service.cc:1511] Processing CreateTablet for tablet d4af1b94141049e0a78a816c8ba31d72 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:39.095587 21129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de27148f295b4ef5a3f79084b8ed5ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.131268 21211 tablet_service.cc:1511] Processing CreateTablet for tablet d4af1b94141049e0a78a816c8ba31d72 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:39.131879 21053 tablet_service.cc:1511] Processing CreateTablet for tablet 080f4010b39e40a8b8efb7280faec38f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:39.135633 21267 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.147924 21267 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.148361 21211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4af1b94141049e0a78a816c8ba31d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.149933 21267 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:39.135038 21266 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.150902 21266 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:39.143870 21262 ts_tablet_manager.cc:1434] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.001s
I20260709 13:05:39.140791 21208 tablet_service.cc:1511] Processing CreateTablet for tablet b8d00e740dba47b2b43be9e4ef1ecb55 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:39.139860 21209 tablet_service.cc:1511] Processing CreateTablet for tablet c20ca56847f64504a85694d10f47558e (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:39.154834 21208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8d00e740dba47b2b43be9e4ef1ecb55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.132179 21128 tablet_service.cc:1511] Processing CreateTablet for tablet 0d78b35ac92d402db8a5e3fe45be52b5 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:39.136468 21202 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae1ecaeba734906b89cfdc36c6abaf9 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:39.158290 21131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731bbc776723436d954fe0b2a30a5b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.126513 21055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223e82a164f445c2b46667d58a5d2d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.161305 21209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20ca56847f64504a85694d10f47558e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.162045 21135 tablet_service.cc:1511] Processing CreateTablet for tablet 53c918605cdf4c39b832a695968308a3 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:39.176313 21172 heartbeater.cc:499] Master 127.19.127.254:37743 was elected leader, sending a full tablet report...
I20260709 13:05:39.178022 21262 tablet_bootstrap.cc:492] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:39.178046 21053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080f4010b39e40a8b8efb7280faec38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.173202 21060 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae1ecaeba734906b89cfdc36c6abaf9 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:39.184540 21060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae1ecaeba734906b89cfdc36c6abaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.189441 21263 tablet_bootstrap.cc:492] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:39.190008 21263 ts_tablet_manager.cc:1403] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.319s	user 0.021s	sys 0.027s
I20260709 13:05:39.190100 21054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4af1b94141049e0a78a816c8ba31d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.195137 21263 raft_consensus.cc:359] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.195598 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:39.196019 21263 raft_consensus.cc:385] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.196300 21221 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:39.196365 21263 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.196482 21059 tablet_service.cc:1511] Processing CreateTablet for tablet c20ca56847f64504a85694d10f47558e (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:39.200287 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:39.200551 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:39.201457 21267 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.201180 21130 tablet_service.cc:1511] Processing CreateTablet for tablet 223e82a164f445c2b46667d58a5d2d1f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:39.201849 21267 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.202314 21267 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.198976 21263 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:39.207058 21036 leader_election.cc:343] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.202116 21128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d78b35ac92d402db8a5e3fe45be52b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.211432 21056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72edcb8f7f3d437b8e7bb24c612600ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.201972 21202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae1ecaeba734906b89cfdc36c6abaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.216569 21205 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca4319420e54bc19f621da03c811035 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:39.217247 21137 tablet_service.cc:1511] Processing CreateTablet for tablet 169e564bd720422bb798c652af4fb2eb (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:39.218008 21136 tablet_service.cc:1511] Processing CreateTablet for tablet 72edcb8f7f3d437b8e7bb24c612600ab (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:39.218528 21266 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:39.224342 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:39.224275 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:39.225939 21135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c918605cdf4c39b832a695968308a3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:39.226322 21036 leader_election.cc:343] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.226940 21059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20ca56847f64504a85694d10f47558e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.227367 21261 tablet_bootstrap.cc:492] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:39.227888 21261 ts_tablet_manager.cc:1403] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.286s	user 0.014s	sys 0.000s
I20260709 13:05:39.226780 21036 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:39.218364 21061 tablet_service.cc:1511] Processing CreateTablet for tablet 77143ea923a34396a4cbc31f42dd0a18 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:39.235692 21203 tablet_service.cc:1511] Processing CreateTablet for tablet 3d51215a492444539c4905a13bd0c12d (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:39.210850 21267 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:39.218031 21205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca4319420e54bc19f621da03c811035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.240836 21207 tablet_service.cc:1511] Processing CreateTablet for tablet e8c9b46a490540a2bb0a90f7bfbd1e53 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:39.241384 21267 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:39.242085 21267 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.242275 21137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169e564bd720422bb798c652af4fb2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.243381 21061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77143ea923a34396a4cbc31f42dd0a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.243480 21262 tablet_bootstrap.cc:654] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.242295 21207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8c9b46a490540a2bb0a90f7bfbd1e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.255962 21203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d51215a492444539c4905a13bd0c12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.255854 21204 tablet_service.cc:1511] Processing CreateTablet for tablet 77143ea923a34396a4cbc31f42dd0a18 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:39.257630 21204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77143ea923a34396a4cbc31f42dd0a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.259931 21136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72edcb8f7f3d437b8e7bb24c612600ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.260951 21132 tablet_service.cc:1511] Processing CreateTablet for tablet 080f4010b39e40a8b8efb7280faec38f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:39.262331 21132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080f4010b39e40a8b8efb7280faec38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.264758 21057 tablet_service.cc:1511] Processing CreateTablet for tablet 3d51215a492444539c4905a13bd0c12d (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:39.231140 21261 raft_consensus.cc:359] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.266417 21261 raft_consensus.cc:385] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.266748 21261 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.266932 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:39.267594 21261 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.268196 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:39.269769 21185 leader_election.cc:343] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:39.270619 21185 leader_election.cc:343] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.304782 21134 tablet_service.cc:1511] Processing CreateTablet for tablet b8d00e740dba47b2b43be9e4ef1ecb55 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:39.307821 21130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223e82a164f445c2b46667d58a5d2d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.313215 21131 tablet_service.cc:1511] Processing CreateTablet for tablet d4af1b94141049e0a78a816c8ba31d72 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:39.314239 21134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8d00e740dba47b2b43be9e4ef1ecb55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.313907 21058 tablet_service.cc:1511] Processing CreateTablet for tablet e8c9b46a490540a2bb0a90f7bfbd1e53 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:39.315589 21206 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa2ded676fc4d048c810a5f842100d1 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:39.309768 21057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d51215a492444539c4905a13bd0c12d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:39.325623 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 99336.2us
W20260709 13:05:39.343930 21036 leader_election.cc:343] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.330878 21209 tablet_service.cc:1511] Processing CreateTablet for tablet a0de4eb0c8e649f09be36671e6dcc4cb (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:39.345546 21062 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca4319420e54bc19f621da03c811035 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:39.347949 21053 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa2ded676fc4d048c810a5f842100d1 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:39.349908 21060 tablet_service.cc:1511] Processing CreateTablet for tablet a0de4eb0c8e649f09be36671e6dcc4cb (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:39.342228 21208 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:39.344281 21206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa2ded676fc4d048c810a5f842100d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.355386 21128 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae1ecaeba734906b89cfdc36c6abaf9 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:39.356228 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:39.357717 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:39.364753 21062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca4319420e54bc19f621da03c811035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.369059 21220 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.370059 21131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4af1b94141049e0a78a816c8ba31d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.372684 21209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0de4eb0c8e649f09be36671e6dcc4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.374022 21263 ts_tablet_manager.cc:1434] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.184s	user 0.011s	sys 0.011s
I20260709 13:05:39.374109 21208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.375355 21061 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:39.382356 21220 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:39.384457 21202 tablet_service.cc:1511] Processing CreateTablet for tablet 41268ef73d2c487dab473150fa26d73e (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:39.385828 21202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41268ef73d2c487dab473150fa26d73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.386042 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:39.382611 21129 tablet_service.cc:1511] Processing CreateTablet for tablet c20ca56847f64504a85694d10f47558e (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:39.388128 21129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20ca56847f64504a85694d10f47558e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:39.388221 21187 leader_election.cc:343] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.388764 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:39.389793 21262 tablet_bootstrap.cc:492] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:39.389922 21266 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.400218 21262 ts_tablet_manager.cc:1403] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.222s	user 0.008s	sys 0.023s
I20260709 13:05:39.392597 21261 ts_tablet_manager.cc:1434] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.164s	user 0.009s	sys 0.000s
I20260709 13:05:39.401684 21261 tablet_bootstrap.cc:492] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
W20260709 13:05:39.393923 21036 leader_election.cc:343] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.396693 21263 tablet_bootstrap.cc:492] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:39.396440 21210 tablet_service.cc:1511] Processing CreateTablet for tablet e0631ac443bb4e6dbbbc599dd4b67d2e (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:39.393393 21061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.407927 21128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae1ecaeba734906b89cfdc36c6abaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.408932 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:39.409956 21278 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.410477 21278 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:39.410678 21058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8c9b46a490540a2bb0a90f7bfbd1e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.411341 21278 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:39.391868 21211 tablet_service.cc:1511] Processing CreateTablet for tablet 2fbcef88f59d4f9ea0f01023c7265069 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:39.415258 21053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa2ded676fc4d048c810a5f842100d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.416076 21211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fbcef88f59d4f9ea0f01023c7265069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.418190 21135 tablet_service.cc:1511] Processing CreateTablet for tablet 77143ea923a34396a4cbc31f42dd0a18 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:39.419533 21135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77143ea923a34396a4cbc31f42dd0a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.419534 21060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0de4eb0c8e649f09be36671e6dcc4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.424829 21210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0631ac443bb4e6dbbbc599dd4b67d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.426409 21055 tablet_service.cc:1511] Processing CreateTablet for tablet 41268ef73d2c487dab473150fa26d73e (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:39.435657 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:39.439495 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:39.440330 21266 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.441860 21261 tablet_bootstrap.cc:654] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.447925 21055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41268ef73d2c487dab473150fa26d73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.448359 21056 tablet_service.cc:1511] Processing CreateTablet for tablet 2fbcef88f59d4f9ea0f01023c7265069 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:39.468506 21136 tablet_service.cc:1511] Processing CreateTablet for tablet e8c9b46a490540a2bb0a90f7bfbd1e53 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:39.469275 21056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fbcef88f59d4f9ea0f01023c7265069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.469965 21136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8c9b46a490540a2bb0a90f7bfbd1e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.460256 21203 tablet_service.cc:1511] Processing CreateTablet for tablet 9f10495397de45019165a00ba7c4ec29 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:39.497265 21203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f10495397de45019165a00ba7c4ec29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.438283 21262 raft_consensus.cc:359] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.498499 21262 raft_consensus.cc:385] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.498883 21262 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.473637 21054 tablet_service.cc:1511] Processing CreateTablet for tablet 9f10495397de45019165a00ba7c4ec29 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:39.459810 21059 tablet_service.cc:1511] Processing CreateTablet for tablet e0631ac443bb4e6dbbbc599dd4b67d2e (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:39.500697 21059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0631ac443bb4e6dbbbc599dd4b67d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.502734 21054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f10495397de45019165a00ba7c4ec29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.476295 21204 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5d4dd9fad84e018bc7168755b0ba6b (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:39.482401 21137 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca4319420e54bc19f621da03c811035 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:39.483316 21134 tablet_service.cc:1511] Processing CreateTablet for tablet 3d51215a492444539c4905a13bd0c12d (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:39.491611 21263 tablet_bootstrap.cc:654] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.517853 21137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca4319420e54bc19f621da03c811035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.519270 21134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d51215a492444539c4905a13bd0c12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.534993 21279 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.535763 21279 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.543215 21206 tablet_service.cc:1511] Processing CreateTablet for tablet 6beb666b5d504215a141add24a6d632f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:39.552474 21133 tablet_service.cc:1511] Processing CreateTablet for tablet a0de4eb0c8e649f09be36671e6dcc4cb (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:39.554365 21133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0de4eb0c8e649f09be36671e6dcc4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.557636 21266 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.558336 21266 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.563714 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:39.565654 21187 leader_election.cc:343] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.554816 21057 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5d4dd9fad84e018bc7168755b0ba6b (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:39.538527 21130 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa2ded676fc4d048c810a5f842100d1 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:39.587996 21130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa2ded676fc4d048c810a5f842100d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.590255 21131 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:39.595546 20970 catalog_manager.cc:5697] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.608304 21266 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:39.608989 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:39.612040 20957 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59051.2us
I20260709 13:05:39.657114 21131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.661445 21263 tablet_bootstrap.cc:492] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:39.662220 21263 ts_tablet_manager.cc:1403] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.266s	user 0.012s	sys 0.006s
I20260709 13:05:39.612334 21204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5d4dd9fad84e018bc7168755b0ba6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.617880 21279 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:39.611688 20955 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 60496.1us
I20260709 13:05:39.610752 21132 tablet_service.cc:1511] Processing CreateTablet for tablet 2fbcef88f59d4f9ea0f01023c7265069 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:39.651467 21146 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:39.680334 21060 tablet_service.cc:1511] Processing CreateTablet for tablet 6beb666b5d504215a141add24a6d632f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:39.681409 21261 tablet_bootstrap.cc:492] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:39.675639 21057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5d4dd9fad84e018bc7168755b0ba6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.595551 21262 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.603243 21128 tablet_service.cc:1511] Processing CreateTablet for tablet 41268ef73d2c487dab473150fa26d73e (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:39.691381 21132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fbcef88f59d4f9ea0f01023c7265069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.692543 21061 tablet_service.cc:1511] Processing CreateTablet for tablet d4a635811aa248b599c9361b7abf9f32 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:39.610646 21133 tablet_service.cc:1511] Processing CreateTablet for tablet e0631ac443bb4e6dbbbc599dd4b67d2e (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:39.696792 21128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41268ef73d2c487dab473150fa26d73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.694686 21206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beb666b5d504215a141add24a6d632f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.695611 21261 ts_tablet_manager.cc:1403] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.294s	user 0.019s	sys 0.035s
I20260709 13:05:39.700471 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:39.712631 21266 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.714524 21266 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.714883 21266 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.709115 21061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4a635811aa248b599c9361b7abf9f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.723886 21133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0631ac443bb4e6dbbbc599dd4b67d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.724694 21209 tablet_service.cc:1511] Processing CreateTablet for tablet de53f43e2a0e4a91a1767df521fc2b8c (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:39.726037 21209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de53f43e2a0e4a91a1767df521fc2b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.726598 21210 tablet_service.cc:1511] Processing CreateTablet for tablet d4a635811aa248b599c9361b7abf9f32 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:39.727916 21210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4a635811aa248b599c9361b7abf9f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.732376 21262 ts_tablet_manager.cc:1434] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.308s	user 0.010s	sys 0.006s
I20260709 13:05:39.733529 21262 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:39.733429 21062 tablet_service.cc:1511] Processing CreateTablet for tablet de53f43e2a0e4a91a1767df521fc2b8c (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:39.745767 21062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de53f43e2a0e4a91a1767df521fc2b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.757881 21056 tablet_service.cc:1511] Processing CreateTablet for tablet b59f0de1289c4c98ae7ce302ca584a30 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:39.762497 21053 tablet_service.cc:1511] Processing CreateTablet for tablet b4ee2463d8894a95be9b747f813f9106 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:39.747548 21263 raft_consensus.cc:359] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.763468 21263 raft_consensus.cc:385] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.764186 21263 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.765107 21263 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.762498 21129 tablet_service.cc:1511] Processing CreateTablet for tablet 9f10495397de45019165a00ba7c4ec29 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:39.769073 21129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f10495397de45019165a00ba7c4ec29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.773073 21263 ts_tablet_manager.cc:1434] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.110s	user 0.008s	sys 0.003s
I20260709 13:05:39.772719 21134 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5d4dd9fad84e018bc7168755b0ba6b (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:39.777845 21261 raft_consensus.cc:359] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.778614 21261 raft_consensus.cc:385] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.778899 21261 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.781955 21060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beb666b5d504215a141add24a6d632f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.787583 21056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59f0de1289c4c98ae7ce302ca584a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.800230 21133 tablet_service.cc:1511] Processing CreateTablet for tablet 6beb666b5d504215a141add24a6d632f (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:39.802359 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:39.811589 21263 tablet_bootstrap.cc:492] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:39.812827 21262 tablet_bootstrap.cc:654] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.818060 21261 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.832312 21058 tablet_service.cc:1511] Processing CreateTablet for tablet 79a87917a1b44746a96a84470b77ea16 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:39.840344 21211 tablet_service.cc:1511] Processing CreateTablet for tablet b4ee2463d8894a95be9b747f813f9106 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:39.841790 21211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ee2463d8894a95be9b747f813f9106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.843565 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:39.831151 21266 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.850008 21220 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:39.850364 21054 tablet_service.cc:1511] Processing CreateTablet for tablet d524f5c1a9e540c9946dd77d8f971ef1 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
W20260709 13:05:39.852911 21113 leader_election.cc:343] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.853412 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:39.853608 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:39.854200 21146 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.854648 21053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ee2463d8894a95be9b747f813f9106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.859825 21278 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.860360 21278 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.860731 21267 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:39.862555 21278 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:39.854197 21261 ts_tablet_manager.cc:1434] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.150s	user 0.005s	sys 0.002s
I20260709 13:05:39.826869 21136 tablet_service.cc:1511] Processing CreateTablet for tablet d4a635811aa248b599c9361b7abf9f32 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:39.870373 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:39.865762 21266 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:05:39.885588 21036 leader_election.cc:343] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.885365 21204 tablet_service.cc:1511] Processing CreateTablet for tablet 79a87917a1b44746a96a84470b77ea16 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:39.886250 21058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a87917a1b44746a96a84470b77ea16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.888022 21060 tablet_service.cc:1511] Processing CreateTablet for tablet 293026cede06406bad9e7b5fc490abe2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:39.864296 21203 tablet_service.cc:1511] Processing CreateTablet for tablet b59f0de1289c4c98ae7ce302ca584a30 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:39.874797 21134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5d4dd9fad84e018bc7168755b0ba6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.906371 21059 tablet_service.cc:1511] Processing CreateTablet for tablet 3d158af381004432b3f3f21e53fd19ac (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:39.907842 21055 tablet_service.cc:1511] Processing CreateTablet for tablet 752b0641d0a3410795a788ef8dc8e035 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:39.916515 21128 tablet_service.cc:1511] Processing CreateTablet for tablet 79a87917a1b44746a96a84470b77ea16 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:39.880448 21146 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:39.918278 21279 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.919327 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:39.920352 21266 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.920835 21266 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:39.921711 21266 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.870703 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:39.890610 21062 tablet_service.cc:1511] Processing CreateTablet for tablet 2e5425aa9a9a4fba958ae97f007977d2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:39.892784 21206 tablet_service.cc:1511] Processing CreateTablet for tablet 293026cede06406bad9e7b5fc490abe2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:39.894002 21054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d524f5c1a9e540c9946dd77d8f971ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.924494 21062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e5425aa9a9a4fba958ae97f007977d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.894490 21202 tablet_service.cc:1511] Processing CreateTablet for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:39.894922 21204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a87917a1b44746a96a84470b77ea16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.896095 21209 tablet_service.cc:1511] Processing CreateTablet for tablet d524f5c1a9e540c9946dd77d8f971ef1 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:39.896660 21061 tablet_service.cc:1511] Processing CreateTablet for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:39.899344 21210 tablet_service.cc:1511] Processing CreateTablet for tablet 3d158af381004432b3f3f21e53fd19ac (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:39.900008 21207 tablet_service.cc:1511] Processing CreateTablet for tablet e30c07cef4334b45ade426b26069b26c (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:39.900266 21205 tablet_service.cc:1511] Processing CreateTablet for tablet 752b0641d0a3410795a788ef8dc8e035 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:39.899344 21135 tablet_service.cc:1511] Processing CreateTablet for tablet de53f43e2a0e4a91a1767df521fc2b8c (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:39.892784 21208 tablet_service.cc:1511] Processing CreateTablet for tablet 2e5425aa9a9a4fba958ae97f007977d2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:39.906627 21261 tablet_bootstrap.cc:492] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:39.929939 21133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beb666b5d504215a141add24a6d632f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.932751 21137 tablet_service.cc:1511] Processing CreateTablet for tablet b59f0de1289c4c98ae7ce302ca584a30 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:39.935089 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:39.937934 21056 tablet_service.cc:1511] Processing CreateTablet for tablet e30c07cef4334b45ade426b26069b26c (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:39.938026 21130 tablet_service.cc:1511] Processing CreateTablet for tablet b4ee2463d8894a95be9b747f813f9106 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
W20260709 13:05:39.930649 21187 leader_election.cc:343] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.939559 21058 tablet_service.cc:1511] Processing CreateTablet for tablet 40d1baf8579949fbb0d5d45a18fefe35 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
W20260709 13:05:39.940274 21036 leader_election.cc:343] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:39.940737 21036 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:39.941540 21278 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:39.941310 21132 tablet_service.cc:1511] Processing CreateTablet for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:39.943447 21060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293026cede06406bad9e7b5fc490abe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.947386 21206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293026cede06406bad9e7b5fc490abe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.949579 21057 tablet_service.cc:1511] Processing CreateTablet for tablet 2875dc050a244109a0287cc6dafa8431 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:39.950906 21057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2875dc050a244109a0287cc6dafa8431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.951901 21209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d524f5c1a9e540c9946dd77d8f971ef1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:39.955629 20955 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57872.6us
I20260709 13:05:39.974820 21262 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:39.975967 21262 ts_tablet_manager.cc:1403] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.243s	user 0.017s	sys 0.024s
I20260709 13:05:39.980384 21262 raft_consensus.cc:359] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:39.988936 21131 tablet_service.cc:1511] Processing CreateTablet for tablet d524f5c1a9e540c9946dd77d8f971ef1 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:39.989236 21133 tablet_service.cc:1511] Processing CreateTablet for tablet 293026cede06406bad9e7b5fc490abe2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:39.994246 21137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59f0de1289c4c98ae7ce302ca584a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.990567 21059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d158af381004432b3f3f21e53fd19ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.994455 21202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.023834 21205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 752b0641d0a3410795a788ef8dc8e035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.031721 21131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d524f5c1a9e540c9946dd77d8f971ef1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:39.983798 20957 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 43912.5us
I20260709 13:05:40.036006 21130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ee2463d8894a95be9b747f813f9106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.036736 21208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e5425aa9a9a4fba958ae97f007977d2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:40.041858 21036 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.037331 21058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40d1baf8579949fbb0d5d45a18fefe35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.042712 21207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e30c07cef4334b45ade426b26069b26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.045318 21135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de53f43e2a0e4a91a1767df521fc2b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.047670 21128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a87917a1b44746a96a84470b77ea16. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:40.013685 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:39.995193 21262 raft_consensus.cc:385] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.053004 21211 tablet_service.cc:1511] Processing CreateTablet for tablet 40d1baf8579949fbb0d5d45a18fefe35 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:40.055493 21136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4a635811aa248b599c9361b7abf9f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.056344 21132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.056743 21055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 752b0641d0a3410795a788ef8dc8e035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.058132 21056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e30c07cef4334b45ade426b26069b26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.058760 21210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d158af381004432b3f3f21e53fd19ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.059646 21203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59f0de1289c4c98ae7ce302ca584a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.061097 21061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.061805 21279 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.061159 21211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40d1baf8579949fbb0d5d45a18fefe35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.062300 21279 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.078516 21134 tablet_service.cc:1511] Processing CreateTablet for tablet 2e5425aa9a9a4fba958ae97f007977d2 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:40.088896 21134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e5425aa9a9a4fba958ae97f007977d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.103643 21133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293026cede06406bad9e7b5fc490abe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.108740 21261 tablet_bootstrap.cc:654] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.115756 21134 tablet_service.cc:1511] Processing CreateTablet for tablet 3d158af381004432b3f3f21e53fd19ac (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:40.114459 21205 tablet_service.cc:1511] Processing CreateTablet for tablet 2875dc050a244109a0287cc6dafa8431 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:40.117110 21134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d158af381004432b3f3f21e53fd19ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.117220 21205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2875dc050a244109a0287cc6dafa8431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.051222 21262 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.124099 21262 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.126456 21262 ts_tablet_manager.cc:1434] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.150s	user 0.002s	sys 0.006s
I20260709 13:05:40.127971 21263 tablet_bootstrap.cc:654] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.138441 21262 tablet_bootstrap.cc:492] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.134261 21134 tablet_service.cc:1511] Processing CreateTablet for tablet 752b0641d0a3410795a788ef8dc8e035 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:40.143883 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:40.145437 21113 leader_election.cc:343] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.147468 21279 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.147657 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:05:40.149175 21111 leader_election.cc:343] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.149672 21111 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:40.150856 21279 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.201478 20967 catalog_manager.cc:5697] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.193810 21134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 752b0641d0a3410795a788ef8dc8e035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.264674 21129 tablet_service.cc:1511] Processing CreateTablet for tablet e30c07cef4334b45ade426b26069b26c (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:40.266799 21129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e30c07cef4334b45ade426b26069b26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.275694 21261 tablet_bootstrap.cc:492] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.276424 21261 ts_tablet_manager.cc:1403] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.370s	user 0.036s	sys 0.009s
I20260709 13:05:40.290170 21262 tablet_bootstrap.cc:654] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.299760 21137 tablet_service.cc:1511] Processing CreateTablet for tablet 40d1baf8579949fbb0d5d45a18fefe35 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:40.300426 21266 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.300959 21266 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.303149 21266 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.292672 21261 raft_consensus.cc:359] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.304203 21261 raft_consensus.cc:385] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.304600 21261 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.307036 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.308310 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:40.309333 21187 leader_election.cc:343] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.306711 21261 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:40.310683 21185 leader_election.cc:343] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.311160 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:40.312711 21289 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.314525 21261 ts_tablet_manager.cc:1434] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:05:40.315594 21261 tablet_bootstrap.cc:492] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.317015 21136 tablet_service.cc:1511] Processing CreateTablet for tablet 2875dc050a244109a0287cc6dafa8431 (DEFAULT_TABLE table=test-workload [id=707c24a2500c4eab9b729d6de4e01f10]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:40.318301 21136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2875dc050a244109a0287cc6dafa8431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.320600 21137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40d1baf8579949fbb0d5d45a18fefe35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.321079 21263 tablet_bootstrap.cc:492] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:40.321575 21263 ts_tablet_manager.cc:1403] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.546s	user 0.028s	sys 0.069s
I20260709 13:05:40.327466 21263 raft_consensus.cc:359] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.328207 21263 raft_consensus.cc:385] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.328514 21263 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.335035 21261 tablet_bootstrap.cc:654] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.339022 21263 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.343518 21263 ts_tablet_manager.cc:1434] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 13:05:40.344828 21263 tablet_bootstrap.cc:492] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:40.351318 21262 tablet_bootstrap.cc:492] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.351840 21263 tablet_bootstrap.cc:654] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.351974 21262 ts_tablet_manager.cc:1403] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.214s	user 0.012s	sys 0.006s
I20260709 13:05:40.354266 21262 raft_consensus.cc:359] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.354880 21262 raft_consensus.cc:385] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.355194 21262 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.356168 21262 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.357609 21261 tablet_bootstrap.cc:492] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.358145 21261 ts_tablet_manager.cc:1403] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.000s
I20260709 13:05:40.360798 21262 ts_tablet_manager.cc:1434] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:05:40.361882 21262 tablet_bootstrap.cc:492] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.361974 21261 raft_consensus.cc:359] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.362680 21261 raft_consensus.cc:385] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.362934 21261 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.358309 21263 tablet_bootstrap.cc:492] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:40.363703 21263 ts_tablet_manager.cc:1403] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.005s
I20260709 13:05:40.363790 21261 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.367364 21263 raft_consensus.cc:359] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:40.368399 21263 raft_consensus.cc:385] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.368746 21263 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.369683 21263 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:05:40.370217 21262 tablet_bootstrap.cc:654] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.372287 21263 ts_tablet_manager.cc:1434] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:05:40.373080 21261 ts_tablet_manager.cc:1434] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:05:40.373624 21263 tablet_bootstrap.cc:492] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:40.374217 21261 tablet_bootstrap.cc:492] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.380791 21263 tablet_bootstrap.cc:654] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.382620 21262 tablet_bootstrap.cc:492] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.382957 21261 tablet_bootstrap.cc:654] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.383154 21262 ts_tablet_manager.cc:1403] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 13:05:40.386698 21262 raft_consensus.cc:359] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.387840 21262 raft_consensus.cc:385] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.388231 21262 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.388638 21289 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:40.389326 21262 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.392052 21262 ts_tablet_manager.cc:1434] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:05:40.393477 21262 tablet_bootstrap.cc:492] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.397473 21261 tablet_bootstrap.cc:492] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.398062 21261 ts_tablet_manager.cc:1403] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.009s
I20260709 13:05:40.400410 21262 tablet_bootstrap.cc:654] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.401160 21261 raft_consensus.cc:359] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.401953 21261 raft_consensus.cc:385] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.402266 21261 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.403303 21261 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.406678 21261 ts_tablet_manager.cc:1434] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:40.408084 21261 tablet_bootstrap.cc:492] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.412305 21262 tablet_bootstrap.cc:492] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.412832 21262 ts_tablet_manager.cc:1403] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.012s
I20260709 13:05:40.415767 21262 raft_consensus.cc:359] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.416529 21262 raft_consensus.cc:385] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.416764 21261 tablet_bootstrap.cc:654] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.416841 21262 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.417924 21262 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.419966 21278 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.420575 21278 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.423383 21278 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.426270 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.426537 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:40.426988 21147 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
W20260709 13:05:40.427983 21036 leader_election.cc:343] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.428260 21261 tablet_bootstrap.cc:492] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.428874 21261 ts_tablet_manager.cc:1403] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.011s
I20260709 13:05:40.428761 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:05:40.430222 21262 ts_tablet_manager.cc:1434] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.004s
I20260709 13:05:40.430583 21278 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.430984 21278 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.431258 21262 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.431427 21278 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.435434 21261 raft_consensus.cc:359] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.436223 21261 raft_consensus.cc:385] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.436512 21261 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.437294 21261 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.438302 21262 tablet_bootstrap.cc:654] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.439165 21267 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.439597 21261 ts_tablet_manager.cc:1434] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:05:40.439725 21267 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.440599 21261 tablet_bootstrap.cc:492] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.440827 21278 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.442098 21267 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.442768 21278 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.444089 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.447507 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:40.449225 21261 tablet_bootstrap.cc:654] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.452454 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:05:40.454982 21036 leader_election.cc:343] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.457270 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.456084 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:40.458521 21147 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.460536 21289 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.461292 21289 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.464500 21289 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.465696 21278 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.466928 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.467938 21071 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:40.470067 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:40.472047 21289 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.472721 21289 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.473240 21289 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.480611 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:40.483767 21185 leader_election.cc:343] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.490116 21279 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.490670 21279 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.498378 21279 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.499718 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:40.500308 21266 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.500963 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.500849 21266 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.502168 21289 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.503749 21266 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.504514 21289 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.504066 21261 tablet_bootstrap.cc:492] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
W20260709 13:05:40.505065 21111 leader_election.cc:343] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.505684 21261 ts_tablet_manager.cc:1403] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.065s	user 0.015s	sys 0.006s
I20260709 13:05:40.500314 21146 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.508509 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:40.509106 21261 raft_consensus.cc:359] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:40.509495 21113 leader_election.cc:343] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.509819 21261 raft_consensus.cc:385] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.509954 21113 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:40.509109 21072 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.511767 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:40.513146 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.515576 21279 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:40.517650 21185 leader_election.cc:343] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:40.518429 21185 leader_election.cc:343] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.521449 21147 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:40.522848 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:05:40.523880 21278 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.523713 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.524335 21278 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:40.525180 21278 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:40.526310 21187 leader_election.cc:343] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.526870 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:40.510119 21261 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.532577 21261 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.535940 21289 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.544907 21263 tablet_bootstrap.cc:492] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:40.547811 21263 ts_tablet_manager.cc:1403] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.175s	user 0.045s	sys 0.056s
I20260709 13:05:40.549294 21072 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:40.552657 21263 raft_consensus.cc:359] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.553694 21263 raft_consensus.cc:385] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.554073 21263 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.555099 21263 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.558878 21262 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.559720 21262 ts_tablet_manager.cc:1403] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.129s	user 0.037s	sys 0.059s
I20260709 13:05:40.561674 21263 ts_tablet_manager.cc:1434] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:05:40.562822 21263 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:40.563859 21261 ts_tablet_manager.cc:1434] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.058s	user 0.012s	sys 0.005s
I20260709 13:05:40.564950 21261 tablet_bootstrap.cc:492] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.564903 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.565536 21072 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.570814 21312 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.571746 21312 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:40.572909 21261 tablet_bootstrap.cc:654] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.572706 21312 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.574544 21278 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.575089 21278 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.578088 21278 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.579514 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.580525 21262 raft_consensus.cc:359] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:40.580856 21036 leader_election.cc:343] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.581228 21262 raft_consensus.cc:385] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.581458 21262 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.582046 21262 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.582787 21263 tablet_bootstrap.cc:654] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.584242 21310 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.584306 21262 ts_tablet_manager.cc:1434] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 13:05:40.584823 21310 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.584684 20968 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.585795 21262 tablet_bootstrap.cc:492] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.587177 21310 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.589298 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:40.590889 21185 leader_election.cc:343] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.583082 21313 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:05:40.592901 21261 tablet_bootstrap.cc:492] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.593093 21279 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.593350 21261 ts_tablet_manager.cc:1403] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 13:05:40.593609 21279 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.596038 21261 raft_consensus.cc:359] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.596706 21261 raft_consensus.cc:385] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.596973 21261 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.597400 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:40.597594 21261 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.598641 21279 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.599732 21261 ts_tablet_manager.cc:1434] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:40.600662 21261 tablet_bootstrap.cc:492] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.601509 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.602515 21072 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:40.603992 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:40.604347 21262 tablet_bootstrap.cc:654] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.605336 21279 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.609380 21262 tablet_bootstrap.cc:492] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.609907 21262 ts_tablet_manager.cc:1403] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.003s
I20260709 13:05:40.610181 21261 tablet_bootstrap.cc:654] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.612322 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:40.612811 21262 raft_consensus.cc:359] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.613897 21262 raft_consensus.cc:385] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.614300 21262 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.615418 21262 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.617957 21262 ts_tablet_manager.cc:1434] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:40.614852 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:40.621246 21187 leader_election.cc:343] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:40.621490 21036 leader_election.cc:343] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.621860 21187 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:40.622119 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.623468 21313 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.623705 21278 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.626065 21262 tablet_bootstrap.cc:492] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.629705 21261 tablet_bootstrap.cc:492] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.630187 21261 ts_tablet_manager.cc:1403] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.030s	user 0.004s	sys 0.015s
I20260709 13:05:40.632316 21262 tablet_bootstrap.cc:654] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.632923 21261 raft_consensus.cc:359] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.633634 21261 raft_consensus.cc:385] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.633891 21261 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.634045 21278 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.634580 21278 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.634692 21261 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.636862 21261 ts_tablet_manager.cc:1434] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:05:40.637686 21261 tablet_bootstrap.cc:492] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.638492 21278 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.639395 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:40.640079 21221 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:40.641266 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:40.642133 21278 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.642606 21278 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.642956 21278 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.643725 21261 tablet_bootstrap.cc:654] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.647367 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.648977 21262 tablet_bootstrap.cc:492] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
W20260709 13:05:40.649056 21036 leader_election.cc:343] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.649515 21262 ts_tablet_manager.cc:1403] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.009s
I20260709 13:05:40.652159 21262 raft_consensus.cc:359] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.652380 21278 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.652820 21262 raft_consensus.cc:385] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.653117 21262 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.653797 21262 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.658119 21278 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.658913 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:40.659158 21261 tablet_bootstrap.cc:492] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.659682 21261 ts_tablet_manager.cc:1403] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.005s
W20260709 13:05:40.660112 21036 leader_election.cc:343] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.660624 21262 ts_tablet_manager.cc:1434] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:05:40.661702 21262 tablet_bootstrap.cc:492] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.662314 21261 raft_consensus.cc:359] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.662986 21261 raft_consensus.cc:385] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.663236 21261 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.663968 21261 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.666363 21261 ts_tablet_manager.cc:1434] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:05:40.667248 21261 tablet_bootstrap.cc:492] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.668154 21262 tablet_bootstrap.cc:654] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.673731 21261 tablet_bootstrap.cc:654] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.680974 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:40.681838 21221 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.687132 21262 tablet_bootstrap.cc:492] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.687855 21262 ts_tablet_manager.cc:1403] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.026s	user 0.023s	sys 0.001s
I20260709 13:05:40.691408 21262 raft_consensus.cc:359] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.691820 21221 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:40.692472 21262 raft_consensus.cc:385] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.692775 21262 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.693475 21262 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.696211 21262 ts_tablet_manager.cc:1434] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:05:40.697809 21262 tablet_bootstrap.cc:492] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.698099 21261 tablet_bootstrap.cc:492] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.698520 21261 ts_tablet_manager.cc:1403] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.015s
I20260709 13:05:40.701092 21261 raft_consensus.cc:359] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.701761 21261 raft_consensus.cc:385] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.702172 21261 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.701653 20968 catalog_manager.cc:5697] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.702795 21261 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.703567 21262 tablet_bootstrap.cc:654] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.704991 21261 ts_tablet_manager.cc:1434] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:05:40.705926 21261 tablet_bootstrap.cc:492] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.713295 21261 tablet_bootstrap.cc:654] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.714814 21262 tablet_bootstrap.cc:492] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.715346 21262 ts_tablet_manager.cc:1403] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.007s
I20260709 13:05:40.720721 21262 raft_consensus.cc:359] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.721190 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.721732 21262 raft_consensus.cc:385] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.722148 21262 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.722330 21278 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.723031 21278 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:40.722862 21262 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.724200 21278 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.725121 21261 tablet_bootstrap.cc:492] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.725757 21261 ts_tablet_manager.cc:1403] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:05:40.729410 21261 raft_consensus.cc:359] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.730346 21261 raft_consensus.cc:385] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.730710 21261 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.731673 21261 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.732388 21262 ts_tablet_manager.cc:1434] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.017s	user 0.013s	sys 0.002s
I20260709 13:05:40.733451 21262 tablet_bootstrap.cc:492] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.734517 21261 ts_tablet_manager.cc:1434] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:05:40.735455 21261 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.739394 21262 tablet_bootstrap.cc:654] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.741269 21261 tablet_bootstrap.cc:654] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.751487 21278 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.751827 21262 tablet_bootstrap.cc:492] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.752736 21262 ts_tablet_manager.cc:1403] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 13:05:40.652956 21279 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.752197 21278 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.759406 21278 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.756466 21262 raft_consensus.cc:359] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.761466 21262 raft_consensus.cc:385] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.761749 21262 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.762521 21261 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.762416 21262 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.762959 21261 ts_tablet_manager.cc:1403] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.015s
I20260709 13:05:40.764087 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.764952 21262 ts_tablet_manager.cc:1434] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:05:40.765933 21262 tablet_bootstrap.cc:492] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.766161 21261 raft_consensus.cc:359] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:40.766455 21036 leader_election.cc:343] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.766794 21261 raft_consensus.cc:385] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.767097 21261 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.768095 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:40.767900 21261 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.768931 21221 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:40.770478 21261 ts_tablet_manager.cc:1434] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:40.770888 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.771468 21261 tablet_bootstrap.cc:492] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.771998 21278 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.772284 21262 tablet_bootstrap.cc:654] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.772485 21278 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.772989 21278 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.784032 21262 tablet_bootstrap.cc:492] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.784580 21262 ts_tablet_manager.cc:1403] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 13:05:40.787178 21261 tablet_bootstrap.cc:654] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.788240 21262 raft_consensus.cc:359] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.788977 21262 raft_consensus.cc:385] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.789254 21262 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.790047 21262 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.796586 21262 ts_tablet_manager.cc:1434] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:05:40.797899 21262 tablet_bootstrap.cc:492] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:40.800834 21328 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.801152 21278 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.801379 21328 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.804080 21278 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.805055 21328 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.809192 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:40.809525 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.809921 21219 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:40.812201 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:40.812883 21263 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:40.813498 21263 ts_tablet_manager.cc:1403] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.251s	user 0.048s	sys 0.089s
I20260709 13:05:40.813566 21261 tablet_bootstrap.cc:492] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.814675 21261 ts_tablet_manager.cc:1403] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.043s	user 0.024s	sys 0.006s
I20260709 13:05:40.815467 21262 tablet_bootstrap.cc:654] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.817312 21263 raft_consensus.cc:359] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.817443 21261 raft_consensus.cc:359] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.818238 21263 raft_consensus.cc:385] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.818408 21261 raft_consensus.cc:385] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.818704 21263 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.818857 21261 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.809202 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:40.819772 21263 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.820060 21261 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.822207 21263 ts_tablet_manager.cc:1434] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:40.812824 21221 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.823392 21261 ts_tablet_manager.cc:1434] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:40.823810 21263 tablet_bootstrap.cc:492] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:40.824640 21261 tablet_bootstrap.cc:492] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.827620 21312 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.828109 21312 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.830329 21312 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.832047 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.832794 21146 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:40.833787 21261 tablet_bootstrap.cc:654] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.834151 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:05:40.835549 21036 leader_election.cc:343] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.836244 21263 tablet_bootstrap.cc:654] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.838191 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.838857 21072 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:40.841313 20969 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:40.836225 21036 leader_election.cc:343] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.844264 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.845064 21312 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.845463 21312 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.845830 21312 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.846709 21261 tablet_bootstrap.cc:492] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:40.847359 21261 ts_tablet_manager.cc:1403] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.003s
I20260709 13:05:40.851218 21313 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.851922 21313 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.853225 21328 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.853737 21328 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.854218 21328 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.855147 21310 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.855645 21310 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.856619 21261 raft_consensus.cc:359] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.857534 21261 raft_consensus.cc:385] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.857898 21261 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.858836 21261 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.866569 21289 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.867334 21289 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.872829 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:40.874907 21185 leader_election.cc:343] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.875764 21289 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.877585 21261 ts_tablet_manager.cc:1434] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 13:05:40.879433 21221 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:40.854892 21313 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.894199 21310 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.896023 21278 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.896538 21278 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.896821 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.896909 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.891387 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.898805 21146 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:05:40.890533 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44433.8us
I20260709 13:05:40.899832 21278 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.899876 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.901332 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.902365 21278 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:40.902343 21185 leader_election.cc:343] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.903124 21278 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:40.903736 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.904042 21278 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.904896 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.905503 21072 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:40.897799 21071 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:40.907507 21187 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:40.908340 21313 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:40.905616 21036 leader_election.cc:343] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.908838 21187 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.911545 21266 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.911944 21266 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.912245 21266 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.917135 21328 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.922392 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:40.922438 21187 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.925159 21261 tablet_bootstrap.cc:492] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:40.925406 21313 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:40.927949 21328 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:40.930238 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:40.928828 21221 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
W20260709 13:05:40.931901 21036 leader_election.cc:343] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.921756 21312 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.933179 21313 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.934823 21312 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.936051 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:40.936384 21261 tablet_bootstrap.cc:654] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.936666 21147 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.937711 21266 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.939172 21263 tablet_bootstrap.cc:492] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:40.939327 21036 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:40.939750 21263 ts_tablet_manager.cc:1403] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.116s	user 0.031s	sys 0.026s
I20260709 13:05:40.941793 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:40.942390 21072 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.945230 21313 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.945674 21313 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.951689 21312 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.950824 21263 raft_consensus.cc:359] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.952250 21263 raft_consensus.cc:385] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.952212 21312 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.952806 21278 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.952749 21263 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.953343 21278 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.953728 21278 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.950441 20970 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.957649 21263 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.961230 21147 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:40.961189 21263 ts_tablet_manager.cc:1434] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.021s	user 0.012s	sys 0.000s
I20260709 13:05:40.963589 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:40.963979 21263 tablet_bootstrap.cc:492] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:40.969512 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:40.972029 21221 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.963917 21312 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:40.979776 21328 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.980822 21328 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.981654 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.982640 21262 tablet_bootstrap.cc:492] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:40.970412 21313 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.984369 21263 tablet_bootstrap.cc:654] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.983088 21328 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:40.983153 21185 leader_election.cc:343] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.983091 21262 ts_tablet_manager.cc:1403] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.185s	user 0.060s	sys 0.048s
I20260709 13:05:40.984876 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:40.984606 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:40.997953 21071 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
W20260709 13:05:40.997386 21036 leader_election.cc:343] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:40.992131 21312 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.999449 21312 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:41.000288 21312 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:40.987383 21278 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.003106 21313 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.003542 21278 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.005353 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.006001 21262 raft_consensus.cc:359] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.006757 21262 raft_consensus.cc:385] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:41.006904 21185 leader_election.cc:343] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.007514 21262 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.008838 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.009418 21262 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.009936 21261 tablet_bootstrap.cc:492] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:41.010376 21261 ts_tablet_manager.cc:1403] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.031s
W20260709 13:05:41.011142 21036 leader_election.cc:343] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.012256 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.012852 21219 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:41.014055 21266 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.015182 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:41.015244 21262 ts_tablet_manager.cc:1434] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.001s
I20260709 13:05:41.015918 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.016160 21220 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:41.016856 21262 tablet_bootstrap.cc:492] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
W20260709 13:05:41.017416 21185 leader_election.cc:343] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.008258 21072 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:41.020220 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:41.021210 21071 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.023486 21261 raft_consensus.cc:359] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.024695 21261 raft_consensus.cc:385] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.024956 21261 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.020166 21187 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.025624 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:41.025978 21266 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.026427 21266 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.026806 21266 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.028801 21263 tablet_bootstrap.cc:492] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:41.029500 21263 ts_tablet_manager.cc:1403] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.005s
I20260709 13:05:41.026662 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.031570 21261 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.033123 21261 ts_tablet_manager.cc:1434] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:05:41.033973 21261 tablet_bootstrap.cc:492] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:41.026841 21072 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:41.037233 21328 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.038825 21278 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:05:41.047210 21187 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.043318 21036 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.048586 21339 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.048938 21312 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.043665 21328 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.049075 21278 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.049608 21328 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.053654 21263 raft_consensus.cc:359] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.054589 21263 raft_consensus.cc:385] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.054944 21263 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
W20260709 13:05:41.055437 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:41.055833 21263 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.056463 21339 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.057019 21339 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.049583 21312 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.059337 21328 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.059175 20969 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.061489 21262 tablet_bootstrap.cc:654] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.062058 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.062466 21266 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.063460 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.064069 21146 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:41.065305 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:41.066129 21289 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.066496 21289 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.066828 21289 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.066910 21071 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:41.068770 21263 ts_tablet_manager.cc:1434] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 13:05:41.069801 21263 tablet_bootstrap.cc:492] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:41.062757 21072 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:41.073460 21339 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.073916 21187 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.078491 21313 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.079052 21313 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:41.079568 21289 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.079885 21313 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.081728 21289 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.083369 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:41.083652 21266 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.084297 21072 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:41.083536 21328 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.085635 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:41.086946 21071 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.088634 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.089215 21261 tablet_bootstrap.cc:654] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.090101 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.091230 21146 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:41.093468 21185 leader_election.cc:343] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.095916 21289 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.101305 21289 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.104492 21312 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.108094 21279 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.109709 21279 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.110662 21289 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.111722 21279 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.112869 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.114625 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.115613 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.100358 21328 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.116359 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.117971 21146 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
W20260709 13:05:41.120287 21185 leader_election.cc:343] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.124797 21262 tablet_bootstrap.cc:492] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:41.121850 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:41.125234 21262 ts_tablet_manager.cc:1403] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.109s	user 0.017s	sys 0.002s
I20260709 13:05:41.125300 21263 tablet_bootstrap.cc:654] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.126008 21071 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:41.127159 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.129474 21220 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:41.130375 21313 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.122605 21328 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.130926 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.130975 21313 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.131986 21279 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.132279 21328 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.133150 21313 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.133483 21221 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:41.103849 21187 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.135165 21289 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:41.135360 21185 leader_election.cc:343] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.136750 21289 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:41.136868 21339 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.137621 21312 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.137594 21289 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.138166 21312 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.139240 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.140108 21312 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:05:41.136044 21036 leader_election.cc:343] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.140558 20970 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.141934 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.144882 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.146524 21278 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:41.147758 21185 leader_election.cc:343] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.149451 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.150240 21072 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
W20260709 13:05:41.151911 21036 leader_election.cc:343] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.152894 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.153445 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:41.155411 21262 raft_consensus.cc:359] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.156244 21262 raft_consensus.cc:385] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.156601 21262 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.159833 21261 tablet_bootstrap.cc:492] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:41.160519 21261 ts_tablet_manager.cc:1403] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.127s	user 0.021s	sys 0.015s
I20260709 13:05:41.155434 21221 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:41.163671 21262 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.167623 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.164455 21278 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.168468 21146 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:41.169039 21278 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:41.173560 21262 ts_tablet_manager.cc:1434] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 13:05:41.174860 21262 tablet_bootstrap.cc:492] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:41.170704 21278 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.171263 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.198359 21347 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.206732 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.207623 21220 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
W20260709 13:05:41.220234 21187 leader_election.cc:343] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.220860 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.222075 21339 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.223613 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:41.205724 21261 raft_consensus.cc:359] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.226006 21339 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.237203 21261 raft_consensus.cc:385] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.237319 21339 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.237645 21261 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.237960 21339 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.199083 21347 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.238631 21261 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.247479 21261 ts_tablet_manager.cc:1434] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.087s	user 0.009s	sys 0.000s
I20260709 13:05:41.248404 21261 tablet_bootstrap.cc:492] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:41.252246 21263 tablet_bootstrap.cc:492] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:41.252763 21263 ts_tablet_manager.cc:1403] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.183s	user 0.014s	sys 0.005s
W20260709 13:05:41.255913 21187 leader_election.cc:343] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.256408 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.257037 21262 tablet_bootstrap.cc:654] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.257179 21266 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.249076 21279 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.260859 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.262140 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.264109 21267 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.265381 21347 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.266835 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.259696 21313 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:05:41.267215 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.267549 21071 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 0.
I20260709 13:05:41.259001 21278 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.271622 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:41.271497 21278 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.271737 21263 raft_consensus.cc:359] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.269892 21267 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.272358 21263 raft_consensus.cc:385] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.272850 21263 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.272379 21279 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.273613 21279 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.274142 21263 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.273247 21267 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.274713 21279 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.278853 21267 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.277186 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.280679 21267 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.283041 21313 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.283557 21313 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:41.284919 21187 leader_election.cc:343] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.281080 21267 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.288973 20970 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.293954 21262 tablet_bootstrap.cc:492] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:41.294700 21262 ts_tablet_manager.cc:1403] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.120s	user 0.012s	sys 0.004s
I20260709 13:05:41.296423 21263 ts_tablet_manager.cc:1434] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:05:41.299003 21313 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.290220 21267 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.288141 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.306041 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.306367 21313 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.306741 21221 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:41.307557 21279 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.310735 21279 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.312381 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:41.316941 21261 tablet_bootstrap.cc:654] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.325291 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.327455 21267 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.327945 21312 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:41.329831 21185 leader_election.cc:343] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.329964 21262 raft_consensus.cc:359] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.330785 21262 raft_consensus.cc:385] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.331063 21262 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.330058 21267 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.333005 21263 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:41.331697 21262 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.336599 21262 ts_tablet_manager.cc:1434] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.001s
I20260709 13:05:41.337518 21262 tablet_bootstrap.cc:492] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:41.327961 21339 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.342059 21220 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:41.314023 20970 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.345243 21262 tablet_bootstrap.cc:654] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.347664 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:41.351681 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 69467.4us
I20260709 13:05:41.354158 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:41.354840 21278 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.355223 21328 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.355800 21328 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.356236 21328 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.377554 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:41.378285 21220 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.379685 21328 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.381186 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.381953 21144 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 1: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:05:41.381990 21339 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.382139 21328 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.386685 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.389068 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.389542 21220 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:41.391247 21263 tablet_bootstrap.cc:654] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.392372 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.394126 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:41.394894 21071 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.396193 21261 tablet_bootstrap.cc:492] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:41.396736 21261 ts_tablet_manager.cc:1403] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.149s	user 0.014s	sys 0.002s
W20260709 13:05:41.399938 21036 leader_election.cc:343] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:41.401479 21036 consensus_peers.cc:597] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:41.402354 21036 leader_election.cc:343] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:41.404255 21187 leader_election.cc:343] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.404320 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.404771 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.407912 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.410974 21313 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.414764 21071 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:41.415684 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:41.420729 21262 tablet_bootstrap.cc:492] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:41.421271 21262 ts_tablet_manager.cc:1403] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.084s	user 0.027s	sys 0.046s
I20260709 13:05:41.416074 21261 raft_consensus.cc:359] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.418896 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:41.422199 21261 raft_consensus.cc:385] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.422431 21261 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.422973 21261 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.423760 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.424834 21279 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.425290 21261 ts_tablet_manager.cc:1434] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.002s
I20260709 13:05:41.421584 21187 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.425565 21220 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.426323 21313 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.426813 21313 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:41.427856 21358 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:41.428058 21313 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.430783 21261 tablet_bootstrap.cc:492] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
W20260709 13:05:41.432206 21036 leader_election.cc:343] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.432729 21036 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.434118 21358 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.438689 21220 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:41.438928 21312 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:41.447811 21261 tablet_bootstrap.cc:654] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.448945 21262 raft_consensus.cc:359] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.445144 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:41.449780 21262 raft_consensus.cc:385] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.450219 21262 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.450413 21358 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.450928 21262 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.451073 21358 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:41.452428 20970 catalog_manager.cc:5697] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.453238 21262 ts_tablet_manager.cc:1434] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.002s
I20260709 13:05:41.452611 21358 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.454234 21262 tablet_bootstrap.cc:492] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:41.460142 21263 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:41.460583 21263 ts_tablet_manager.cc:1403] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.128s	user 0.019s	sys 0.011s
W20260709 13:05:41.465643 21036 leader_election.cc:343] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.471688 21263 raft_consensus.cc:359] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.475345 21263 raft_consensus.cc:385] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.469544 21313 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:05:41.475752 21263 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.470659 21262 tablet_bootstrap.cc:654] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.477058 21261 tablet_bootstrap.cc:492] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:41.477521 21261 ts_tablet_manager.cc:1403] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.000s
I20260709 13:05:41.479221 21263 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.480063 21261 raft_consensus.cc:359] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.480763 21261 raft_consensus.cc:385] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.481068 21261 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.481922 21261 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.484423 21261 ts_tablet_manager.cc:1434] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:41.486677 21263 ts_tablet_manager.cc:1434] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:05:41.488029 21263 tablet_bootstrap.cc:492] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:41.489825 21261 tablet_bootstrap.cc:492] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
W20260709 13:05:41.475365 21036 leader_election.cc:343] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.496892 21261 tablet_bootstrap.cc:654] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.499192 21263 tablet_bootstrap.cc:654] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.531870 21313 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 13:05:41.518321 21185 consensus_peers.cc:597] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:41.499573 21358 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
I20260709 13:05:41.509073 21261 tablet_bootstrap.cc:492] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:41.543951 21262 tablet_bootstrap.cc:492] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:41.544708 21261 ts_tablet_manager.cc:1403] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.055s	user 0.018s	sys 0.019s
I20260709 13:05:41.544772 21262 ts_tablet_manager.cc:1403] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.057s
I20260709 13:05:41.503564 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.546639 21357 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.547111 21357 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:41.547509 21262 raft_consensus.cc:359] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.548205 21262 raft_consensus.cc:385] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.548511 21262 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.548107 21357 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.549098 21262 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.551357 21262 ts_tablet_manager.cc:1434] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:05:41.551893 21313 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.552321 21262 tablet_bootstrap.cc:492] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:41.552455 21313 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.552215 20970 catalog_manager.cc:5697] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.556541 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.557271 21072 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:41.559412 21262 tablet_bootstrap.cc:654] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.566939 21263 tablet_bootstrap.cc:492] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:41.567898 21263 ts_tablet_manager.cc:1403] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.080s	user 0.020s	sys 0.000s
I20260709 13:05:41.570611 21289 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:41.571158 21263 raft_consensus.cc:359] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.573026 21263 raft_consensus.cc:385] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.573343 21263 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.574129 21263 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.581161 21187 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:41.583550 21263 ts_tablet_manager.cc:1434] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:05:41.584972 21263 tablet_bootstrap.cc:492] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:41.588905 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.590570 21310 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.590973 21310 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.591341 21310 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:41.592622 21185 leader_election.cc:343] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.585718 21313 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.601298 21261 raft_consensus.cc:359] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.602072 21261 raft_consensus.cc:385] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.602334 21261 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.595497 21357 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:05:41.603578 21261 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.608444 21262 tablet_bootstrap.cc:492] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:41.608675 21261 ts_tablet_manager.cc:1434] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.001s
I20260709 13:05:41.608915 21262 ts_tablet_manager.cc:1403] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.057s	user 0.018s	sys 0.020s
I20260709 13:05:41.610126 21261 tablet_bootstrap.cc:492] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:41.615355 20969 catalog_manager.cc:5697] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.618763 21263 tablet_bootstrap.cc:654] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.621083 21310 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.621411 21262 raft_consensus.cc:359] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.622042 21262 raft_consensus.cc:385] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.622224 21262 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.622782 21262 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.625562 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:41.627648 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:41.623091 21310 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.628269 21071 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:41.631415 21185 leader_election.cc:343] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.633808 21261 tablet_bootstrap.cc:654] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.635438 21279 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.635964 21279 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.636287 21262 ts_tablet_manager.cc:1434] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:05:41.639417 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.646667 21262 tablet_bootstrap.cc:492] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:41.644217 21358 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.648648 21278 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.638077 21279 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.649132 21278 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.638871 21071 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:41.651451 21187 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.648308 21358 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.653409 21339 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.654088 21339 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:41.654145 21358 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.651489 21278 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.655313 21339 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.657341 21289 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.657869 21289 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.660403 21289 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.663561 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.664424 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.663659 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.666260 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:05:41.683895 21185 leader_election.cc:343] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.683980 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.684149 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.686707 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.691681 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.692499 21279 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.692745 21347 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.692975 21279 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:41.693711 21036 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:41.694337 21036 leader_election.cc:343] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:41.694993 21036 leader_election.cc:343] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.695186 21279 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:41.696440 21036 leader_election.cc:343] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.696877 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.697619 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.697736 21278 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.698451 21221 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:41.698529 21262 tablet_bootstrap.cc:654] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.700381 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.707649 21072 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:41.713750 21113 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.717665 21347 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.714956 21279 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.719908 21279 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:41.725420 21187 leader_election.cc:343] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.726107 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.728188 21313 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.728875 21313 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.736194 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:41.738617 21036 leader_election.cc:343] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.739231 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.740300 21278 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.740934 21279 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.743587 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.744434 21220 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:41.745641 21111 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.747089 21357 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:41.749912 21347 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.750982 21347 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.759897 21347 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.770143 21263 tablet_bootstrap.cc:492] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:41.770632 21261 tablet_bootstrap.cc:492] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:41.771533 21261 ts_tablet_manager.cc:1403] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.162s	user 0.006s	sys 0.017s
I20260709 13:05:41.770642 21263 ts_tablet_manager.cc:1403] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.186s	user 0.041s	sys 0.017s
I20260709 13:05:41.774370 21261 raft_consensus.cc:359] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.774729 21263 raft_consensus.cc:359] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.775377 21263 raft_consensus.cc:385] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.775686 21263 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.776458 21263 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.778349 21261 raft_consensus.cc:385] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.778599 21263 ts_tablet_manager.cc:1434] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:05:41.778697 21261 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.781306 20969 catalog_manager.cc:5697] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.783753 21262 tablet_bootstrap.cc:492] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:41.784236 21262 ts_tablet_manager.cc:1403] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.138s	user 0.014s	sys 0.017s
I20260709 13:05:41.779824 21261 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.787171 21262 raft_consensus.cc:359] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.788098 21262 raft_consensus.cc:385] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.788367 21262 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.788622 21261 ts_tablet_manager.cc:1434] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:41.788823 21347 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.789217 21263 tablet_bootstrap.cc:492] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:41.789511 21261 tablet_bootstrap.cc:492] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:41.789305 21347 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.791543 21262 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.792273 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.792933 21071 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 0.
I20260709 13:05:41.793816 21262 ts_tablet_manager.cc:1434] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.001s
I20260709 13:05:41.794729 21262 tablet_bootstrap.cc:492] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:41.794538 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:41.795660 21279 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.796038 21279 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.796427 21279 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.800199 21263 tablet_bootstrap.cc:654] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.805706 21347 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.807210 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.808183 21221 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 0.
I20260709 13:05:41.809525 21279 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.812502 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:41.812763 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:41.813309 21072 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.813603 21221 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.795671 21261 tablet_bootstrap.cc:654] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.818966 21279 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.827805 21072 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:05:41.830046 21261 tablet_bootstrap.cc:492] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:41.829828 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:41.830569 21261 ts_tablet_manager.cc:1403] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.000s
I20260709 13:05:41.830925 21279 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:41.831537 21279 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:05:41.832413 21279 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.833747 21261 raft_consensus.cc:359] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.834509 21261 raft_consensus.cc:385] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.834802 21261 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.838537 21262 tablet_bootstrap.cc:654] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:41.839895 21036 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:41.840672 21221 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:05:41.840503 21261 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.846489 21261 ts_tablet_manager.cc:1434] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 13:05:41.847755 21261 tablet_bootstrap.cc:492] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:41.861975 21263 tablet_bootstrap.cc:492] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:41.862501 21263 ts_tablet_manager.cc:1403] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.073s	user 0.027s	sys 0.039s
I20260709 13:05:41.865056 21261 tablet_bootstrap.cc:654] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.865239 21263 raft_consensus.cc:359] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.866101 21263 raft_consensus.cc:385] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.866413 21263 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.867062 21263 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.869122 21263 ts_tablet_manager.cc:1434] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:05:41.869242 21262 tablet_bootstrap.cc:492] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:41.869695 21262 ts_tablet_manager.cc:1403] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.000s
I20260709 13:05:41.869966 21263 tablet_bootstrap.cc:492] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:41.869436 20969 catalog_manager.cc:5697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.872965 21262 raft_consensus.cc:359] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.873739 21262 raft_consensus.cc:385] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.874048 21262 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.874985 21262 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.876461 21263 tablet_bootstrap.cc:654] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.888152 21262 ts_tablet_manager.cc:1434] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:05:41.904141 21262 tablet_bootstrap.cc:492] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:41.906145 21263 tablet_bootstrap.cc:492] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:41.906800 21263 ts_tablet_manager.cc:1403] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.012s
I20260709 13:05:41.909032 21262 tablet_bootstrap.cc:654] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.908985 21354 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.909389 21263 raft_consensus.cc:359] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.909905 21261 tablet_bootstrap.cc:492] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:41.909578 21354 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.910107 21263 raft_consensus.cc:385] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.910418 21263 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.910545 21261 ts_tablet_manager.cc:1403] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.063s	user 0.023s	sys 0.028s
I20260709 13:05:41.911237 21263 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.911969 21354 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.913285 21263 ts_tablet_manager.cc:1434] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:05:41.914242 21263 tablet_bootstrap.cc:492] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:41.914633 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:41.915251 21221 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:41.914263 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.918030 21261 raft_consensus.cc:359] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.918807 21261 raft_consensus.cc:385] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.919121 21261 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
W20260709 13:05:41.919502 21036 leader_election.cc:343] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.920290 21263 tablet_bootstrap.cc:654] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:41.919961 21261 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.920477 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.921749 21359 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.922438 21359 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.925223 21359 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:41.925927 21359 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.926345 21359 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.926748 21359 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.929363 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:41.932822 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:05:41.935547 21036 leader_election.cc:343] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.941063 21261 ts_tablet_manager.cc:1434] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.030s	user 0.002s	sys 0.003s
W20260709 13:05:41.943591 21036 leader_election.cc:343] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.943786 21263 tablet_bootstrap.cc:492] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:41.944576 21263 ts_tablet_manager.cc:1403] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.012s
I20260709 13:05:41.944437 21036 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:41.947144 21263 raft_consensus.cc:359] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.947540 21354 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.947867 21263 raft_consensus.cc:385] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:41.948354 21263 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:41.949007 21263 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.951243 21263 ts_tablet_manager.cc:1434] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:41.942413 21312 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.952181 21263 tablet_bootstrap.cc:492] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:41.951983 21312 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.954063 21312 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.957873 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.959004 21263 tablet_bootstrap.cc:654] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:41.959544 21187 leader_election.cc:343] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.943847 21261 tablet_bootstrap.cc:492] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:41.964184 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:41.966280 21185 leader_election.cc:343] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:41.966919 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:41.968948 21339 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:41.973477 21339 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.974102 21339 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:41.976986 21339 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:41.979298 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:41.980226 21072 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:41.985266 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:41.988099 21313 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:41.988694 21313 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.989173 21313 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:41.984892 21359 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.002952 21359 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.009372 21312 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.023895 21262 tablet_bootstrap.cc:492] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:42.024417 21262 ts_tablet_manager.cc:1403] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.120s	user 0.045s	sys 0.053s
I20260709 13:05:42.024989 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:42.025027 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:42.027127 21185 leader_election.cc:343] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.027768 21262 raft_consensus.cc:359] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.028506 21262 raft_consensus.cc:385] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.028695 21146 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.028790 21262 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
W20260709 13:05:42.034515 21036 leader_election.cc:343] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.035403 21262 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.037825 21262 ts_tablet_manager.cc:1434] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:05:42.038884 21262 tablet_bootstrap.cc:492] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:42.046032 21262 tablet_bootstrap.cc:654] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.051234 21359 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.052009 21359 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.054698 21359 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.056914 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:42.057832 21221 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.064168 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:42.065892 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:05:42.066331 21036 leader_election.cc:343] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.080194 21261 tablet_bootstrap.cc:654] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.088116 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.089085 21220 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:42.091905 21313 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.096355 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:42.098405 21185 leader_election.cc:343] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.116012 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:42.116714 21071 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.120158 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.121392 21328 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.123397 21328 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.123885 21328 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.124346 21328 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.126201 21071 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:42.128515 21221 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:42.131523 21354 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.148191 21289 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.150874 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.152247 21359 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.165552 21359 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.168690 21359 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.170148 21339 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.151486 21313 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:42.169890 21266 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.171887 21266 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:42.173200 21261 tablet_bootstrap.cc:492] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:42.173723 21261 ts_tablet_manager.cc:1403] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.230s	user 0.025s	sys 0.022s
I20260709 13:05:42.173032 21266 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.179881 21279 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.180624 21279 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.181290 21328 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.182978 21261 raft_consensus.cc:359] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.183732 21261 raft_consensus.cc:385] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.184005 21261 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.185233 21279 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.185763 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:42.187376 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.188019 21339 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.188556 21339 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.188982 21111 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:42.190312 21279 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.191206 21339 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:42.198216 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:42.199218 21147 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:42.184688 21261 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.201984 21261 ts_tablet_manager.cc:1434] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:05:42.206043 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:42.203246 21261 tablet_bootstrap.cc:492] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:42.206534 21262 tablet_bootstrap.cc:492] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:42.213042 21262 ts_tablet_manager.cc:1403] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.174s	user 0.058s	sys 0.089s
W20260709 13:05:42.207840 21185 consensus_peers.cc:597] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.215927 21262 raft_consensus.cc:359] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.216665 21262 raft_consensus.cc:385] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.205026 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:42.216998 21262 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.217670 21262 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.208400 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.218964 21359 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.220196 21262 ts_tablet_manager.cc:1434] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:05:42.220018 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.220147 21359 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:42.221176 21219 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:42.221462 21262 tablet_bootstrap.cc:492] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:42.220242 21328 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.221879 21359 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.223416 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:42.224189 21218 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.226147 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:42.229116 21289 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.229661 21289 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.229286 20969 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:42.231375 21185 leader_election.cc:343] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.231719 21289 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:42.232986 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:42.233626 21278 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:42.234330 21185 leader_election.cc:343] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.234699 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:05:42.236402 21036 leader_election.cc:343] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.237087 21278 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.239601 21147 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.246464 21261 tablet_bootstrap.cc:654] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.247334 21262 tablet_bootstrap.cc:654] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.253139 21263 tablet_bootstrap.cc:492] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:42.253908 21263 ts_tablet_manager.cc:1403] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.302s	user 0.066s	sys 0.103s
I20260709 13:05:42.240792 21278 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:42.247098 21187 leader_election.cc:343] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.263259 21263 raft_consensus.cc:359] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.264071 21263 raft_consensus.cc:385] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.264386 21263 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.264994 21263 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.256188 21278 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.258997 21187 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:42.267365 21339 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.269424 21263 ts_tablet_manager.cc:1434] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 13:05:42.270295 21263 tablet_bootstrap.cc:492] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:42.271389 21328 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.271939 21328 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.274106 21328 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.282935 21279 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.292536 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:42.293641 21279 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.305413 21218 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:42.308763 21278 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.310858 21278 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.316459 21262 tablet_bootstrap.cc:492] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:42.317363 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:42.319440 21262 ts_tablet_manager.cc:1403] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.098s	user 0.028s	sys 0.001s
I20260709 13:05:42.322872 21347 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.323424 21347 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.337030 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:42.331542 21279 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:42.329576 21187 leader_election.cc:343] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.348320 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:42.350128 21347 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.351521 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.358481 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:42.358800 21218 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:42.359104 21143 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.352720 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.356979 21339 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.365502 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.365728 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.366184 21219 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:42.367488 21262 raft_consensus.cc:359] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.368172 21262 raft_consensus.cc:385] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.368474 21262 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.369007 21278 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.367048 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:42.370628 21217 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.370648 21278 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:42.371973 21278 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.373729 21262 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.375654 21263 tablet_bootstrap.cc:654] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.379319 21262 ts_tablet_manager.cc:1434] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.001s
I20260709 13:05:42.380369 21262 tablet_bootstrap.cc:492] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:42.381325 21217 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:42.384946 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:42.386329 21143 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:42.388144 21347 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.390333 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:42.392304 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:42.394040 21279 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.396574 21347 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.403190 21279 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.413460 21261 tablet_bootstrap.cc:492] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:42.414199 21261 ts_tablet_manager.cc:1403] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.211s	user 0.011s	sys 0.013s
I20260709 13:05:42.424823 21279 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:42.434258 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67325.8us
I20260709 13:05:42.440661 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:42.441566 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:42.442600 21359 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.447398 21262 tablet_bootstrap.cc:654] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.443430 21359 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.448096 21359 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.443148 21412 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.454780 21261 raft_consensus.cc:359] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.455850 21261 raft_consensus.cc:385] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.458062 21261 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.457070 21143 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.466444 21261 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.454001 21412 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:42.468231 21412 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.468889 21261 ts_tablet_manager.cc:1434] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 13:05:42.467823 20969 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.469934 21261 tablet_bootstrap.cc:492] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:42.476742 21354 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.480358 21354 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.482942 21354 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.483733 21263 tablet_bootstrap.cc:492] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:42.484191 21263 ts_tablet_manager.cc:1403] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.214s	user 0.021s	sys 0.029s
I20260709 13:05:42.484213 21359 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.488562 21359 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.495138 21263 raft_consensus.cc:359] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.498587 21263 raft_consensus.cc:385] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.500175 21263 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.501042 21263 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:42.489429 21036 leader_election.cc:343] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.503908 21262 tablet_bootstrap.cc:492] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:42.504451 21261 tablet_bootstrap.cc:654] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.504829 21262 ts_tablet_manager.cc:1403] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.125s	user 0.015s	sys 0.007s
I20260709 13:05:42.508821 21263 ts_tablet_manager.cc:1434] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:05:42.509486 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:42.511566 21036 leader_election.cc:343] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.511641 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.512303 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:42.512626 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:42.515180 21218 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.512331 21221 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:42.516938 21263 tablet_bootstrap.cc:492] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:42.522193 21218 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
W20260709 13:05:42.517831 21036 leader_election.cc:343] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.523736 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.525439 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.526016 21354 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.526562 21354 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.526898 21416 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.527638 21416 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:42.528825 21416 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.526928 21354 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.541543 21263 tablet_bootstrap.cc:654] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.548467 21261 tablet_bootstrap.cc:492] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:42.549111 21261 ts_tablet_manager.cc:1403] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.079s	user 0.021s	sys 0.004s
I20260709 13:05:42.553996 21263 tablet_bootstrap.cc:492] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:42.554477 21263 ts_tablet_manager.cc:1403] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.008s
I20260709 13:05:42.557651 21262 raft_consensus.cc:359] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.558404 21262 raft_consensus.cc:385] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.558193 21263 raft_consensus.cc:359] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.558724 21262 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.559038 21263 raft_consensus.cc:385] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.559511 21263 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.560227 21261 raft_consensus.cc:359] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.560235 21263 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.561050 21261 raft_consensus.cc:385] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.561371 21261 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.562039 21261 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.562599 21263 ts_tablet_manager.cc:1434] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:05:42.561810 21262 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.563552 21263 tablet_bootstrap.cc:492] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:42.564106 21261 ts_tablet_manager.cc:1434] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:05:42.564960 21261 tablet_bootstrap.cc:492] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:42.572618 21261 tablet_bootstrap.cc:654] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.573351 21262 ts_tablet_manager.cc:1434] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.050s	user 0.019s	sys 0.006s
I20260709 13:05:42.574404 21262 tablet_bootstrap.cc:492] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:42.581423 21262 tablet_bootstrap.cc:654] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.582757 20969 catalog_manager.cc:5697] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.588572 20969 catalog_manager.cc:5697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.593319 21263 tablet_bootstrap.cc:654] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.603739 21313 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.604334 21313 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.605638 21263 tablet_bootstrap.cc:492] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:42.606417 21313 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:42.607197 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:42.607900 21072 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:42.608178 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:42.607726 21354 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.609135 21187 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:05:42.609702 21185 leader_election.cc:343] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.610008 21313 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.610383 21313 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.610792 21313 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.611570 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:42.611912 21266 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.612390 21266 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:42.613831 21036 leader_election.cc:343] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.614516 21266 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:42.615641 21263 ts_tablet_manager.cc:1403] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.010s
I20260709 13:05:42.616375 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:42.617903 21185 leader_election.cc:343] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.618487 21263 raft_consensus.cc:359] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.619604 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:42.620199 21218 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:42.630316 21185 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.631397 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:42.633107 21187 leader_election.cc:343] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.633711 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:42.634678 21266 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.640864 21412 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.649507 21218 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:42.650856 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.651989 21354 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:42.657505 21036 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.659742 21147 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.619167 21263 raft_consensus.cc:385] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.660713 21263 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.661502 21263 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.662489 21412 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.662979 21412 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.666599 21412 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.667980 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:42.671888 21036 leader_election.cc:343] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.673144 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:05:42.674985 21036 leader_election.cc:343] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.675558 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.676898 21412 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.678902 21313 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.681834 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:42.682443 21072 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.685598 21339 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.686708 21339 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.690788 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:42.691715 21070 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:42.691511 21266 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.692957 21339 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:42.694173 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:42.694893 21143 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:42.683233 21354 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.696079 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:42.696488 21354 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:42.697518 21339 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:42.697358 21354 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.698000 21339 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.698658 21339 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:42.685894 20968 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.709331 21262 tablet_bootstrap.cc:492] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:42.709926 21262 ts_tablet_manager.cc:1403] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.136s	user 0.033s	sys 0.028s
I20260709 13:05:42.711527 21263 ts_tablet_manager.cc:1434] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.096s	user 0.005s	sys 0.003s
I20260709 13:05:42.713254 21263 tablet_bootstrap.cc:492] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:42.716882 21262 raft_consensus.cc:359] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.717603 21262 raft_consensus.cc:385] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.718115 21261 tablet_bootstrap.cc:492] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:42.718048 21262 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.718581 21261 ts_tablet_manager.cc:1403] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.154s	user 0.053s	sys 0.082s
I20260709 13:05:42.720127 21313 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:42.720700 21262 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.721809 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:42.723083 21185 leader_election.cc:343] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.717567 21072 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:42.725996 21262 ts_tablet_manager.cc:1434] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:05:42.727645 21262 tablet_bootstrap.cc:492] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:42.727089 21187 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.732735 21313 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.727137 21339 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.733285 21313 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:42.738011 21261 raft_consensus.cc:359] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.738235 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:42.738785 21261 raft_consensus.cc:385] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.738965 21072 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:42.739092 21261 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.739205 21313 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.802294 21263 tablet_bootstrap.cc:654] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.814491 21072 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:05:42.779250 21359 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.834456 21412 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:42.837397 21359 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.858363 21359 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.865278 21347 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.865993 21347 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.868634 21347 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.870776 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.876705 21347 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.877874 21347 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.881052 21347 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.881616 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.882216 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:42.882325 21220 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:42.883184 21221 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:42.882421 21261 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.889561 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:42.890618 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:42.892356 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:42.893114 21072 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:05:42.894975 21036 leader_election.cc:343] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:42.895258 21263 tablet_bootstrap.cc:492] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:42.895135 21339 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:42.896063 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:42.897600 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:42.898811 21262 tablet_bootstrap.cc:654] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.900277 21409 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.901060 21409 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.902161 21187 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:42.903939 21263 ts_tablet_manager.cc:1403] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.184s	user 0.013s	sys 0.003s
I20260709 13:05:42.906759 21263 raft_consensus.cc:359] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.907474 21263 raft_consensus.cc:385] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.907756 21263 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.908003 21354 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:42.910336 21261 ts_tablet_manager.cc:1434] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.191s	user 0.030s	sys 0.060s
I20260709 13:05:42.910429 21409 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:42.911434 21261 tablet_bootstrap.cc:492] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:42.912032 21339 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:42.912544 21339 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:42.913503 21339 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.915905 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:42.917275 21412 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:42.922528 21412 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.922904 21412 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.916352 20968 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.908418 21263 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:42.921789 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58272us
W20260709 13:05:42.932478 21036 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.946259 21263 ts_tablet_manager.cc:1434] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.001s
I20260709 13:05:42.947353 21263 tablet_bootstrap.cc:492] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:42.950178 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:42.957135 21261 tablet_bootstrap.cc:654] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.963927 21143 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:42.958956 20968 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.972803 21263 tablet_bootstrap.cc:654] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.980013 21412 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:42.987732 21412 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:42.988108 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:42.988725 21220 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.991459 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:42.999869 21036 leader_election.cc:343] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.000727 21412 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.007074 20970 catalog_manager.cc:5697] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 2, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.016815 21220 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:43.014580 21143 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:05:43.020929 21261 tablet_bootstrap.cc:492] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:43.021435 21261 ts_tablet_manager.cc:1403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.110s	user 0.014s	sys 0.009s
I20260709 13:05:43.023115 21412 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.026096 21262 tablet_bootstrap.cc:492] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:43.026567 21262 ts_tablet_manager.cc:1403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.299s	user 0.047s	sys 0.035s
I20260709 13:05:43.027849 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.030048 21261 raft_consensus.cc:359] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.030782 21261 raft_consensus.cc:385] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.031066 21261 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.031776 21261 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.032349 21262 raft_consensus.cc:359] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.032938 21412 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.033205 21262 raft_consensus.cc:385] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.033916 21354 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.033614 21412 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.034509 21354 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:43.033638 21262 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.037134 21261 ts_tablet_manager.cc:1434] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:05:43.037014 21354 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.038229 21261 tablet_bootstrap.cc:492] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:43.039460 21262 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.042304 21412 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.046254 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:43.046126 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:43.047389 21218 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:43.047943 21262 ts_tablet_manager.cc:1434] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
W20260709 13:05:43.048029 21036 leader_election.cc:343] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.048964 21262 tablet_bootstrap.cc:492] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:43.048764 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.049595 21359 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.049955 21359 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.050309 21359 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.069383 21359 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.070001 21262 tablet_bootstrap.cc:654] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.072505 21412 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.076705 21359 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.096036 21412 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.100287 21412 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.101297 21261 tablet_bootstrap.cc:654] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.097146 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:43.103917 21263 tablet_bootstrap.cc:492] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:43.104404 21263 ts_tablet_manager.cc:1403] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.157s	user 0.026s	sys 0.026s
I20260709 13:05:43.105890 21412 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.103950 21221 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.104297 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:43.111639 21146 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:43.112542 21313 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.113464 21313 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.114506 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:43.115409 21219 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:43.116206 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:43.117372 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:05:43.118196 21036 leader_election.cc:343] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.117856 21354 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.122138 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:43.118695 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:43.120922 21354 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.125916 21072 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
W20260709 13:05:43.125975 21185 leader_election.cc:343] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.117769 21313 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:43.121094 21412 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:43.129984 21412 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.131760 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.133013 21313 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.133396 21354 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.133414 21313 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.133999 21412 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:43.134841 21221 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:43.136051 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:43.136399 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:43.133983 21313 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:43.138139 21036 leader_election.cc:343] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.139607 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.140625 21354 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.141574 21354 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:43.142359 21354 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.145982 21289 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.146392 20968 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.160207 21263 raft_consensus.cc:359] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.161103 21263 raft_consensus.cc:385] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.161587 21263 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.162431 21263 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.164047 21263 ts_tablet_manager.cc:1434] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.059s	user 0.002s	sys 0.004s
I20260709 13:05:43.164979 21263 tablet_bootstrap.cc:492] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:43.167397 21261 tablet_bootstrap.cc:492] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:43.167908 21261 ts_tablet_manager.cc:1403] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.130s	user 0.023s	sys 0.015s
I20260709 13:05:43.172091 21261 raft_consensus.cc:359] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.170055 21412 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.172936 21261 raft_consensus.cc:385] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.173278 21261 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.175644 21261 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.176136 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:43.176748 21143 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:43.178200 21261 ts_tablet_manager.cc:1434] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:43.179422 21261 tablet_bootstrap.cc:492] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:43.179937 21313 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.187536 21217 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:05:43.187558 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:43.187835 21143 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:05:43.188755 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:43.189392 21220 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:43.196327 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:43.199330 21220 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:05:43.200174 21263 tablet_bootstrap.cc:654] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.200819 21412 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:43.201014 21185 consensus_peers.cc:597] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:43.200169 21354 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:43.202226 21354 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:43.203362 21261 tablet_bootstrap.cc:654] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.203059 21354 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.208477 21313 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:43.209662 21409 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.210144 21409 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.211225 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.212337 21409 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.212858 21359 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.214195 21262 tablet_bootstrap.cc:492] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:43.214816 21262 ts_tablet_manager.cc:1403] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.166s	user 0.024s	sys 0.001s
I20260709 13:05:43.219609 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:43.221619 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:43.222191 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:43.222942 21217 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:43.223636 21146 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.224234 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:43.225960 21359 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.228682 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:43.229786 21409 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:43.232534 21185 leader_election.cc:343] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.258872 21409 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.277571 21412 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.278115 21071 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:43.280478 21262 raft_consensus.cc:359] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.285827 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.286479 21262 raft_consensus.cc:385] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.287029 21339 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.287352 21262 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.287568 21339 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:43.288084 21262 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.288427 21339 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.289746 21262 ts_tablet_manager.cc:1434] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.002s
I20260709 13:05:43.290746 21262 tablet_bootstrap.cc:492] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:43.307194 21261 tablet_bootstrap.cc:492] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:43.307747 21261 ts_tablet_manager.cc:1403] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.129s	user 0.037s	sys 0.051s
I20260709 13:05:43.309659 21262 tablet_bootstrap.cc:654] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.318732 21261 raft_consensus.cc:359] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.319562 21261 raft_consensus.cc:385] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.319834 21261 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.320503 21261 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.323572 20969 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.325798 20969 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 2, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.327836 21261 ts_tablet_manager.cc:1434] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:05:43.328960 21261 tablet_bootstrap.cc:492] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:43.374665 21071 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:43.378067 21313 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.378727 21263 tablet_bootstrap.cc:492] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:43.379243 21263 ts_tablet_manager.cc:1403] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.214s	user 0.017s	sys 0.005s
I20260709 13:05:43.386008 21263 raft_consensus.cc:359] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.389895 21263 raft_consensus.cc:385] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.390295 21263 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.391050 21263 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.393283 21263 ts_tablet_manager.cc:1434] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 13:05:43.389557 21261 tablet_bootstrap.cc:654] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.386749 21354 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:43.399546 21263 tablet_bootstrap.cc:492] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:43.405148 20970 catalog_manager.cc:5697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.406967 21263 tablet_bootstrap.cc:654] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.417132 21261 tablet_bootstrap.cc:492] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:43.417627 21261 ts_tablet_manager.cc:1403] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.089s	user 0.006s	sys 0.005s
I20260709 13:05:43.421768 21262 tablet_bootstrap.cc:492] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:43.422307 21262 ts_tablet_manager.cc:1403] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.132s	user 0.017s	sys 0.020s
I20260709 13:05:43.431478 21146 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:43.433775 21263 tablet_bootstrap.cc:492] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:43.434279 21263 ts_tablet_manager.cc:1403] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.004s
I20260709 13:05:43.437006 21263 raft_consensus.cc:359] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.437688 21263 raft_consensus.cc:385] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.438024 21263 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.438464 21313 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.438709 21263 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.440266 21263 ts_tablet_manager.cc:1434] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:05:43.441792 21262 raft_consensus.cc:359] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.442514 21262 raft_consensus.cc:385] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.442751 21262 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.443370 21262 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.419662 21261 raft_consensus.cc:359] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.444700 21261 raft_consensus.cc:385] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.445010 21261 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.445281 21262 ts_tablet_manager.cc:1434] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:05:43.445624 21261 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.447046 21263 tablet_bootstrap.cc:492] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:43.447996 21261 ts_tablet_manager.cc:1434] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.030s	user 0.002s	sys 0.005s
I20260709 13:05:43.448911 21261 tablet_bootstrap.cc:492] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:43.453614 21263 tablet_bootstrap.cc:654] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.454953 21261 tablet_bootstrap.cc:654] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.446163 21262 tablet_bootstrap.cc:492] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:43.458597 21409 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.459120 21409 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.460454 21431 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.462285 21409 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.463047 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:43.465390 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:43.461191 21431 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.466897 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:43.468097 21409 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.469426 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:43.470574 21262 tablet_bootstrap.cc:654] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.470418 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:43.470881 21187 leader_election.cc:343] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:43.472138 21185 leader_election.cc:343] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.472606 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:43.473552 21313 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.474695 21261 tablet_bootstrap.cc:492] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:43.475167 21261 ts_tablet_manager.cc:1403] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.003s
I20260709 13:05:43.475247 21431 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:43.476707 21262 tablet_bootstrap.cc:492] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:43.477356 21262 ts_tablet_manager.cc:1403] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.005s
I20260709 13:05:43.478094 21261 raft_consensus.cc:359] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.478782 21261 raft_consensus.cc:385] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.479058 21261 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.479713 21261 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.481652 21262 raft_consensus.cc:359] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.482322 21262 raft_consensus.cc:385] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.482576 21262 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.482756 21261 ts_tablet_manager.cc:1434] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:43.483554 21262 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.485596 21262 ts_tablet_manager.cc:1434] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:05:43.488564 21262 tablet_bootstrap.cc:492] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:43.494776 21409 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.495711 21409 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.496470 21263 tablet_bootstrap.cc:492] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:43.496968 21263 ts_tablet_manager.cc:1403] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.050s	user 0.003s	sys 0.010s
I20260709 13:05:43.498394 21262 tablet_bootstrap.cc:654] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.499248 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:43.501075 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:43.500355 21263 raft_consensus.cc:359] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.501700 21261 tablet_bootstrap.cc:492] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:43.501971 21263 raft_consensus.cc:385] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.502378 21263 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.503511 21113 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:43.503325 21263 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.504397 21347 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.505537 21409 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.508697 21261 tablet_bootstrap.cc:654] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.525130 21263 ts_tablet_manager.cc:1434] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.028s	user 0.011s	sys 0.012s
I20260709 13:05:43.526197 21263 tablet_bootstrap.cc:492] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:43.527086 21261 tablet_bootstrap.cc:492] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:43.531373 21262 tablet_bootstrap.cc:492] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:43.531441 21146 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.531924 21262 ts_tablet_manager.cc:1403] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.016s
I20260709 13:05:43.534015 21263 tablet_bootstrap.cc:654] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.534993 21262 raft_consensus.cc:359] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.535678 21262 raft_consensus.cc:385] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.535941 21262 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.536547 21262 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.538517 21262 ts_tablet_manager.cc:1434] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:43.539445 21262 tablet_bootstrap.cc:492] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:43.545349 21262 tablet_bootstrap.cc:654] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.531638 21261 ts_tablet_manager.cc:1403] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.009s
I20260709 13:05:43.550050 21261 raft_consensus.cc:359] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.550905 21261 raft_consensus.cc:385] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.551210 21261 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.552007 21261 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.553692 21261 ts_tablet_manager.cc:1434] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:05:43.554728 21261 tablet_bootstrap.cc:492] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:43.565023 21263 tablet_bootstrap.cc:492] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:43.565770 21261 tablet_bootstrap.cc:654] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.566255 21263 ts_tablet_manager.cc:1403] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.003s
I20260709 13:05:43.566598 21262 tablet_bootstrap.cc:492] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:43.567114 21262 ts_tablet_manager.cc:1403] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.028s	user 0.023s	sys 0.003s
I20260709 13:05:43.569357 21263 raft_consensus.cc:359] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.570116 21263 raft_consensus.cc:385] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.570583 21263 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.571513 21263 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.570693 21262 raft_consensus.cc:359] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.574119 21262 raft_consensus.cc:385] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.574586 21262 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.576483 21263 ts_tablet_manager.cc:1434] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:05:43.575584 21262 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.577538 21263 tablet_bootstrap.cc:492] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:43.578894 21262 ts_tablet_manager.cc:1434] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:05:43.580240 21262 tablet_bootstrap.cc:492] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:43.584229 21263 tablet_bootstrap.cc:654] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.588573 21262 tablet_bootstrap.cc:654] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.600802 21431 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.601501 21431 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.602639 21146 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.604269 21431 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:43.605619 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:43.607491 21185 leader_election.cc:343] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.612461 21220 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:43.614678 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:43.616470 21071 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:43.617626 21359 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:43.618194 21036 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:43.619195 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.620577 21431 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.620977 21431 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.621296 21431 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.628010 21147 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.629171 21359 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.629669 21359 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.634583 21359 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.635519 21313 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.636155 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:43.637025 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:43.637073 21218 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:43.637769 21143 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:43.639734 21431 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.640354 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:43.636183 21313 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.641185 21354 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.641832 21431 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:43.642531 21354 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.642685 21313 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:43.642925 21354 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.644551 21354 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.647590 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:43.649959 21185 leader_election.cc:343] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.650715 21262 tablet_bootstrap.cc:492] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:43.652194 21262 ts_tablet_manager.cc:1403] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.072s	user 0.022s	sys 0.012s
I20260709 13:05:43.655125 21262 raft_consensus.cc:359] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.655910 21262 raft_consensus.cc:385] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.656191 21262 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.656842 21262 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.658281 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:43.662093 21261 tablet_bootstrap.cc:492] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:43.662585 21261 ts_tablet_manager.cc:1403] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.108s	user 0.026s	sys 0.041s
I20260709 13:05:43.664114 21262 ts_tablet_manager.cc:1434] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:05:43.663987 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:43.664659 21071 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.665136 21262 tablet_bootstrap.cc:492] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:43.664691 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:43.667995 21070 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
W20260709 13:05:43.668071 21185 leader_election.cc:343] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.674129 21359 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.680190 21354 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.682090 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.684343 21431 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.684859 21431 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.685177 21431 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.704102 21071 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:43.714125 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.714696 21144 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.715329 21313 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.715938 21313 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:43.717057 21313 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.710754 21412 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.727154 21412 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.730759 21412 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.739398 21431 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.744127 21431 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:43.743992 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:43.744717 21071 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:43.749322 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 80316.7us
I20260709 13:05:43.750777 21261 raft_consensus.cc:359] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.751825 21261 raft_consensus.cc:385] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.752357 21261 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.753275 21261 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.755909 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:43.756877 21217 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:43.756306 21261 ts_tablet_manager.cc:1434] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.001s
I20260709 13:05:43.760031 21263 tablet_bootstrap.cc:492] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:43.760629 21263 ts_tablet_manager.cc:1403] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.183s	user 0.016s	sys 0.032s
I20260709 13:05:43.761106 21261 tablet_bootstrap.cc:492] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:43.763044 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:43.765983 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:43.766556 21146 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.767057 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:43.769979 21145 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 13:05:43.776032 21185 leader_election.cc:343] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.780086 21262 tablet_bootstrap.cc:654] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.780715 21263 raft_consensus.cc:359] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.781477 21263 raft_consensus.cc:385] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.781730 21263 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.782361 21263 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.791905 21146 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:43.792515 21354 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.795295 21071 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:43.798249 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:43.802901 21263 ts_tablet_manager.cc:1434] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.002s
I20260709 13:05:43.806639 21263 tablet_bootstrap.cc:492] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
W20260709 13:05:43.807544 21036 leader_election.cc:343] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.815112 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.807386 21354 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.806120 21412 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.810024 21278 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:05:43.817844 21313 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.832763 21313 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:43.833706 21313 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.822263 21412 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.838743 21412 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.819792 21354 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.839612 21354 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.852913 21475 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:05:43.872759 20968 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.890094 21261 tablet_bootstrap.cc:654] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.893306 21479 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.894068 21479 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.895568 21266 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.897455 21479 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:43.898536 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:43.898857 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:43.899516 21071 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
W20260709 13:05:43.900588 21185 leader_election.cc:343] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.901450 21187 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:43.902599 21479 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.903159 21479 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.903556 21479 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:43.922928 21185 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:43.925602 21263 tablet_bootstrap.cc:654] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.902527 21354 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:43.933481 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 81158.7us
I20260709 13:05:43.938643 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:43.943174 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:43.943826 21220 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.945801 21416 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.947224 21412 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.947748 21412 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.950057 21412 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.953904 21479 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.972864 20968 catalog_manager.cc:5697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.943773 21354 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:43.979825 21479 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:43.971946 21416 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:43.980679 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:43.981755 21261 tablet_bootstrap.cc:492] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:43.981426 21416 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:43.982209 21261 ts_tablet_manager.cc:1403] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.221s	user 0.014s	sys 0.015s
I20260709 13:05:43.977044 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:05:43.992686 21185 leader_election.cc:343] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.992188 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:43.993259 21479 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.999234 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:44.006211 21070 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.007181 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:43.999013 21221 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:44.038995 21431 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.050474 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:44.051396 21216 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:44.004477 21217 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.056484 21431 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.058853 21263 tablet_bootstrap.cc:492] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:44.059635 21263 ts_tablet_manager.cc:1403] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.253s	user 0.039s	sys 0.018s
I20260709 13:05:44.066401 21262 tablet_bootstrap.cc:492] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:44.066990 21262 ts_tablet_manager.cc:1403] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.402s	user 0.056s	sys 0.033s
I20260709 13:05:44.075953 21261 raft_consensus.cc:359] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.076927 21261 raft_consensus.cc:385] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.077383 21261 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.061491 21431 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:44.004470 21479 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.088933 21261 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.090731 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:44.091835 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:44.092881 21263 raft_consensus.cc:359] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.093631 21263 raft_consensus.cc:385] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.093927 21263 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.095469 21263 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.104899 21263 ts_tablet_manager.cc:1434] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 13:05:44.106761 21263 tablet_bootstrap.cc:492] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:44.110306 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:44.094799 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:44.111804 21261 ts_tablet_manager.cc:1434] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.129s	user 0.009s	sys 0.007s
I20260709 13:05:44.113636 21476 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:44.114017 21476 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.114349 21476 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.112272 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:44.117004 21482 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:44.117415 21482 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.117771 21482 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:44.100752 21185 leader_election.cc:343] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.125089 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:44.127533 21070 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
W20260709 13:05:44.149381 21036 leader_election.cc:343] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.152077 21479 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:44.158071 21187 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:44.160807 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:44.163431 21144 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.155114 21220 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:44.165022 21491 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.165584 21491 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:44.166406 21491 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.170291 21261 tablet_bootstrap.cc:492] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
W20260709 13:05:44.173908 21036 leader_election.cc:343] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:44.172847 21187 leader_election.cc:343] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:44.181762 21187 leader_election.cc:343] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.182212 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:44.182873 21313 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:44.186857 21036 leader_election.cc:343] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.210980 21217 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:44.215111 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:44.216893 21412 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.217411 21412 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:44.218227 21412 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.191170 21482 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.223645 21262 raft_consensus.cc:359] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.224442 21262 raft_consensus.cc:385] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.224642 21262 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.226691 21262 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.232801 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:44.234259 21036 leader_election.cc:343] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.233063 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:44.237502 21482 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.240715 21262 ts_tablet_manager.cc:1434] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.173s	user 0.008s	sys 0.000s
I20260709 13:05:44.240535 21476 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.241658 21262 tablet_bootstrap.cc:492] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:44.244738 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:44.245225 21261 tablet_bootstrap.cc:654] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.245368 21145 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.247474 21263 tablet_bootstrap.cc:654] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.249915 20969 catalog_manager.cc:5697] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.254392 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:05:44.260190 21185 leader_election.cc:343] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.255612 21217 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.265480 21266 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.277484 20970 catalog_manager.cc:5697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:44.269701 21145 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:44.282611 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:44.293350 21221 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.287947 21217 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:44.277838 21185 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:44.279934 21476 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.246513 21491 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:05:44.311669 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:44.317569 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:44.328585 21262 tablet_bootstrap.cc:654] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.329682 21412 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.335656 21496 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.336256 21412 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:44.337441 21412 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:44.338418 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78276.5us
I20260709 13:05:44.331859 21476 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.381383 21221 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:44.372761 21476 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:44.408018 21263 tablet_bootstrap.cc:492] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:44.408542 21263 ts_tablet_manager.cc:1403] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.302s	user 0.015s	sys 0.052s
I20260709 13:05:44.407389 20969 catalog_manager.cc:5697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.420825 21476 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:44.426928 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:44.433940 21261 tablet_bootstrap.cc:492] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:44.434468 21261 ts_tablet_manager.cc:1403] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.264s	user 0.033s	sys 0.035s
I20260709 13:05:44.435420 21263 raft_consensus.cc:359] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.436090 21263 raft_consensus.cc:385] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.436269 21263 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.436708 21263 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.439719 21465 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:05:44.440291 21465 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.441339 21500 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.441819 21500 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.448324 21263 ts_tablet_manager.cc:1434] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 13:05:44.449232 21263 tablet_bootstrap.cc:492] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
W20260709 13:05:44.454655 21185 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:44.462177 21262 tablet_bootstrap.cc:492] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:44.462859 21262 ts_tablet_manager.cc:1403] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.221s	user 0.018s	sys 0.002s
I20260709 13:05:44.465466 21500 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.466797 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:44.467473 21261 raft_consensus.cc:359] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.468246 21261 raft_consensus.cc:385] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.468588 21261 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.469342 21261 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.471992 21261 ts_tablet_manager.cc:1434] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.002s
I20260709 13:05:44.473023 21261 tablet_bootstrap.cc:492] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:44.475996 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:44.477437 21113 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:44.478591 21500 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.479249 21263 tablet_bootstrap.cc:654] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.478830 21262 raft_consensus.cc:359] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.481689 21262 raft_consensus.cc:385] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.481932 21262 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.482496 21262 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:44.492568 21185 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:44.494997 21262 ts_tablet_manager.cc:1434] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:05:44.496490 21262 tablet_bootstrap.cc:492] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:44.517009 21465 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.518366 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:44.518704 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:44.520166 21261 tablet_bootstrap.cc:654] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.521066 21113 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:44.521463 21412 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:44.522358 21465 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:44.524259 21262 tablet_bootstrap.cc:654] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.554848 21263 tablet_bootstrap.cc:492] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:44.555370 21263 ts_tablet_manager.cc:1403] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.106s	user 0.009s	sys 0.007s
I20260709 13:05:44.555751 21262 tablet_bootstrap.cc:492] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:44.556222 21262 ts_tablet_manager.cc:1403] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.060s	user 0.014s	sys 0.009s
I20260709 13:05:44.557981 21263 raft_consensus.cc:359] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.559924 21262 raft_consensus.cc:359] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.560629 21262 raft_consensus.cc:385] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.560465 21476 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:44.560984 21262 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.561714 21262 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.563849 21262 ts_tablet_manager.cc:1434] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:44.564628 21261 tablet_bootstrap.cc:492] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:44.564765 21262 tablet_bootstrap.cc:492] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:44.565202 21261 ts_tablet_manager.cc:1403] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.092s	user 0.013s	sys 0.009s
I20260709 13:05:44.567198 21495 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.567699 21495 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.568240 21261 raft_consensus.cc:359] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.569985 21495 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:44.570739 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:44.571130 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:44.572381 21187 leader_election.cc:343] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:44.572880 21185 leader_election.cc:343] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.573307 21185 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:44.569098 21261 raft_consensus.cc:385] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.574009 21261 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.574276 21495 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.575243 21261 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.578262 21261 ts_tablet_manager.cc:1434] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.004s
I20260709 13:05:44.580096 21261 tablet_bootstrap.cc:492] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:44.574276 21262 tablet_bootstrap.cc:654] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.587949 21261 tablet_bootstrap.cc:654] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.593436 21261 tablet_bootstrap.cc:492] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:44.593786 21262 tablet_bootstrap.cc:492] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:44.593902 21261 ts_tablet_manager.cc:1403] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.014s	user 0.003s	sys 0.006s
I20260709 13:05:44.594254 21262 ts_tablet_manager.cc:1403] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.001s
I20260709 13:05:44.558686 21263 raft_consensus.cc:385] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.596925 21263 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.597326 21262 raft_consensus.cc:359] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.598254 21262 raft_consensus.cc:385] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.597711 21263 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.598551 21262 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.599203 21261 raft_consensus.cc:359] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.600426 21261 raft_consensus.cc:385] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.600039 21262 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.600685 21263 ts_tablet_manager.cc:1434] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.002s
I20260709 13:05:44.600745 21261 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.601689 21263 tablet_bootstrap.cc:492] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:44.601784 21261 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.602592 21262 ts_tablet_manager.cc:1434] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 13:05:44.603644 21262 tablet_bootstrap.cc:492] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:44.655082 21278 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.637653 20968 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.662246 20968 catalog_manager.cc:5697] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.670083 21261 ts_tablet_manager.cc:1434] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.000s
W20260709 13:05:44.672420 21036 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:44.656278 21278 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.678716 21278 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.678987 21266 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.682592 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:44.685285 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:44.686296 21221 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
W20260709 13:05:44.688839 21036 leader_election.cc:343] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.684393 21263 tablet_bootstrap.cc:654] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.680855 21261 tablet_bootstrap.cc:492] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:44.680106 21262 tablet_bootstrap.cc:654] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.690593 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:44.697734 21482 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:44.698172 21482 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.698560 21482 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:44.711005 21185 consensus_peers.cc:597] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:44.714617 21261 tablet_bootstrap.cc:654] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.717338 21476 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.718245 21476 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.721448 21476 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.723939 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:44.724982 21221 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:44.726434 21476 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.727696 21476 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.728089 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:44.732086 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:44.734649 21278 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:44.736286 21278 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.736671 21278 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.743742 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:44.746266 21263 tablet_bootstrap.cc:492] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:44.746660 21476 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.747107 21263 ts_tablet_manager.cc:1403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.146s	user 0.012s	sys 0.014s
I20260709 13:05:44.747427 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:05:44.749450 21036 leader_election.cc:343] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.740839 21482 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.753428 21482 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.754619 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:44.755554 21278 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.755582 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:44.756542 21215 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.757818 21278 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.750976 21263 raft_consensus.cc:359] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.759457 21263 raft_consensus.cc:385] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.759856 21263 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.762308 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:44.766085 21036 leader_election.cc:343] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.770622 21262 tablet_bootstrap.cc:492] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:44.768952 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:44.776731 21221 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.784713 21496 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.785612 21496 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.812660 21495 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.813452 21221 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:44.813635 21495 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.820134 21215 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:44.811928 21262 ts_tablet_manager.cc:1403] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.209s	user 0.024s	sys 0.011s
I20260709 13:05:44.821345 21495 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:44.822500 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:44.824321 21185 leader_election.cc:343] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.826174 21261 tablet_bootstrap.cc:492] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:44.826689 21261 ts_tablet_manager.cc:1403] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.146s	user 0.025s	sys 0.012s
I20260709 13:05:44.830511 21261 raft_consensus.cc:359] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.832304 21261 raft_consensus.cc:385] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.832515 21261 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.833438 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:44.831813 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:44.839756 21187 leader_election.cc:343] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.840261 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:44.841284 21266 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:44.841475 21187 leader_election.cc:343] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.840209 21261 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.843267 21144 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.843724 21261 ts_tablet_manager.cc:1434] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 13:05:44.844905 21261 tablet_bootstrap.cc:492] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:44.850306 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:44.851063 21476 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.851164 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:44.851596 21476 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:44.852633 21476 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.855876 21263 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.856685 21354 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:44.856874 21496 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:44.857086 21465 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.857722 21482 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:44.857618 21465 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.858753 21354 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:44.859931 21354 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.863799 21261 tablet_bootstrap.cc:654] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.865478 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:44.866528 21216 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:44.866392 21262 raft_consensus.cc:359] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:44.867537 21036 leader_election.cc:343] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.868042 21465 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.867204 21262 raft_consensus.cc:385] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:44.868831 21262 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.869529 21262 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.871228 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:44.873313 21263 ts_tablet_manager.cc:1434] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.126s	user 0.012s	sys 0.000s
I20260709 13:05:44.867983 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:44.876487 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:44.876844 21263 tablet_bootstrap.cc:492] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:44.878018 21500 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.879652 21262 ts_tablet_manager.cc:1434] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
W20260709 13:05:44.880505 21036 leader_election.cc:343] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:44.881814 21036 leader_election.cc:343] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.884405 21278 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.886818 21278 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.887353 21278 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.889467 21278 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.893397 21278 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:05:44.897627 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:44.907088 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:44.908239 21262 tablet_bootstrap.cc:492] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
W20260709 13:05:44.909224 21185 leader_election.cc:343] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.909675 21185 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:44.894156 21465 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.912499 21036 leader_election.cc:343] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.913722 21495 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.917764 21465 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.920449 21263 tablet_bootstrap.cc:654] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:44.921043 21465 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.915469 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:44.926587 21500 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.926769 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:44.928452 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:44.942921 21036 leader_election.cc:343] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:44.943542 21036 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:44.955470 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:44.958917 21500 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.962265 21507 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.981743 21507 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.982568 21261 tablet_bootstrap.cc:492] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:44.982241 21507 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:44.983064 21261 ts_tablet_manager.cc:1403] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.004s
I20260709 13:05:44.984468 21507 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:44.987195 21145 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.993139 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:44.994234 21476 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:44.995146 21354 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.997774 21359 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:45.001346 21036 leader_election.cc:343] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.001125 21261 raft_consensus.cc:359] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.002419 21261 raft_consensus.cc:385] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.002782 21261 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.021564 21261 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.021117 20967 catalog_manager.cc:5697] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:45.008157 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:45.023752 21216 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:45.026172 21261 ts_tablet_manager.cc:1434] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:05:45.026489 21262 tablet_bootstrap.cc:654] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.026811 20967 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:45.036602 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:45.039340 21263 tablet_bootstrap.cc:492] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:45.039860 21263 ts_tablet_manager.cc:1403] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.163s	user 0.026s	sys 0.015s
I20260709 13:05:45.042418 21476 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.047847 21476 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.048367 21476 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.052915 21261 tablet_bootstrap.cc:492] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:45.077293 21263 raft_consensus.cc:359] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.078178 21263 raft_consensus.cc:385] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.078486 21263 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.079264 21263 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.081648 21263 ts_tablet_manager.cc:1434] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 13:05:45.082851 21263 tablet_bootstrap.cc:492] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:45.069865 21476 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.106931 21495 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.107517 21495 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.109565 21495 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:45.111078 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:45.111770 21069 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:45.114480 21187 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:45.116617 21496 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.117004 21496 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.117311 21496 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.120961 21261 tablet_bootstrap.cc:654] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.122745 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:45.123855 21476 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:45.124419 21185 leader_election.cc:343] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.127696 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:45.128106 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:45.128304 21221 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.128715 21496 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.130820 21496 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:45.131517 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:45.132087 21069 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.144740 21221 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:45.145128 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:45.146534 21185 leader_election.cc:343] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.147316 21262 tablet_bootstrap.cc:492] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:45.147809 21262 ts_tablet_manager.cc:1403] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.240s	user 0.013s	sys 0.011s
W20260709 13:05:45.148914 21036 leader_election.cc:343] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.149591 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:45.154834 21476 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:45.162429 21069 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:45.162752 21476 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:45.163723 21476 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.164541 21187 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:45.166399 21266 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:45.166896 21266 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:45.167742 21266 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.169199 21263 tablet_bootstrap.cc:654] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.170825 21262 raft_consensus.cc:359] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.171633 21262 raft_consensus.cc:385] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.171938 21262 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.173157 21262 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.181216 21262 ts_tablet_manager.cc:1434] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:05:45.195051 21262 tablet_bootstrap.cc:492] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:45.198939 21261 tablet_bootstrap.cc:492] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:45.199467 21261 ts_tablet_manager.cc:1403] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.155s	user 0.018s	sys 0.011s
I20260709 13:05:45.200943 21263 tablet_bootstrap.cc:492] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:45.201488 21263 ts_tablet_manager.cc:1403] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.119s	user 0.016s	sys 0.010s
I20260709 13:05:45.203627 21261 raft_consensus.cc:359] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.204384 21261 raft_consensus.cc:385] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.204819 21261 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.205567 21261 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.207712 21261 ts_tablet_manager.cc:1434] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:45.206768 20968 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.208827 21261 tablet_bootstrap.cc:492] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:45.213112 21262 tablet_bootstrap.cc:654] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.215194 21496 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.216012 21261 tablet_bootstrap.cc:654] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.215749 21496 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.217895 21496 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:45.204358 21263 raft_consensus.cc:359] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.218616 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:45.219172 21263 raft_consensus.cc:385] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.219491 21263 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
W20260709 13:05:45.220041 21187 leader_election.cc:343] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.220202 21263 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.221556 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:45.222326 21263 ts_tablet_manager.cc:1434] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
W20260709 13:05:45.223423 21185 leader_election.cc:343] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.223979 21263 tablet_bootstrap.cc:492] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:45.223920 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:45.225299 20970 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:45.228005 21266 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:45.230396 21263 tablet_bootstrap.cc:654] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.248518 21262 tablet_bootstrap.cc:492] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:45.248974 21262 ts_tablet_manager.cc:1403] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.003s
I20260709 13:05:45.249178 21261 tablet_bootstrap.cc:492] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:45.249629 21261 ts_tablet_manager.cc:1403] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.002s
I20260709 13:05:45.251245 21262 raft_consensus.cc:359] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.251889 21262 raft_consensus.cc:385] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.252111 21262 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.252100 21261 raft_consensus.cc:359] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.252904 21261 raft_consensus.cc:385] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.253155 21261 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.253748 21261 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.255976 21261 ts_tablet_manager.cc:1434] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:45.257083 21261 tablet_bootstrap.cc:492] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:45.258680 21262 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.260509 21262 ts_tablet_manager.cc:1434] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 13:05:45.261776 21262 tablet_bootstrap.cc:492] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:45.269567 21495 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.270295 21495 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.280872 21495 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:45.283869 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:45.284722 21069 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:45.294432 21476 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.296743 21262 tablet_bootstrap.cc:654] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.292615 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:45.298556 21495 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.298969 21495 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.298782 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:45.299321 21495 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:45.300413 21185 leader_election.cc:343] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.300870 21507 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.305766 21261 tablet_bootstrap.cc:654] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.310649 21263 tablet_bootstrap.cc:492] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:45.310222 21495 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.311327 21263 ts_tablet_manager.cc:1403] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.088s	user 0.010s	sys 0.033s
I20260709 13:05:45.315613 21263 raft_consensus.cc:359] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.316327 21263 raft_consensus.cc:385] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.316654 21263 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.320715 21263 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.324771 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:45.325074 21495 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:45.325448 21069 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.326663 21263 ts_tablet_manager.cc:1434] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 13:05:45.327706 21263 tablet_bootstrap.cc:492] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:45.330587 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:45.332808 21185 leader_election.cc:343] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.335932 21069 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:45.338089 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:45.342784 21495 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:45.343369 21495 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:05:45.344388 21036 consensus_peers.cc:597] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.344323 21495 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.350833 21263 tablet_bootstrap.cc:654] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.353149 21261 tablet_bootstrap.cc:492] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:45.353083 21145 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.353731 21261 ts_tablet_manager.cc:1403] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.097s	user 0.008s	sys 0.012s
I20260709 13:05:45.356889 21261 raft_consensus.cc:359] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.357671 21261 raft_consensus.cc:385] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.357934 21261 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.358542 21261 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.360627 21261 ts_tablet_manager.cc:1434] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:05:45.361761 21261 tablet_bootstrap.cc:492] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:45.374724 21412 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:45.392397 21262 tablet_bootstrap.cc:492] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:45.393106 21262 ts_tablet_manager.cc:1403] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.132s	user 0.013s	sys 0.010s
I20260709 13:05:45.396852 21262 raft_consensus.cc:359] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.397852 21262 raft_consensus.cc:385] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.398216 21262 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.409586 21496 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.410331 21496 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.405488 21262 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.420253 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:45.437310 21496 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:45.438872 21261 tablet_bootstrap.cc:654] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.440640 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:45.442631 21187 leader_election.cc:343] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:45.443055 21185 leader_election.cc:343] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.443477 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:45.444449 21495 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:45.450590 21263 tablet_bootstrap.cc:492] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:45.451141 21263 ts_tablet_manager.cc:1403] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.124s	user 0.039s	sys 0.045s
I20260709 13:05:45.455440 21507 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.455958 21507 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.459303 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:45.460791 21036 leader_election.cc:343] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.460958 21507 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:45.462313 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:45.464293 21221 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:45.470060 21262 ts_tablet_manager.cc:1434] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.077s	user 0.010s	sys 0.002s
I20260709 13:05:45.480280 21262 tablet_bootstrap.cc:492] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:45.480896 20968 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.482743 21261 tablet_bootstrap.cc:492] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:45.483232 21261 ts_tablet_manager.cc:1403] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.122s	user 0.020s	sys 0.003s
I20260709 13:05:45.486114 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:45.486814 21507 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.487167 21507 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.487624 21507 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.489105 21263 raft_consensus.cc:359] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.489943 21263 raft_consensus.cc:385] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.490309 21263 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.489970 21261 raft_consensus.cc:359] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.490855 21261 raft_consensus.cc:385] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.491133 21261 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.491444 21263 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.492055 21261 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.493671 21263 ts_tablet_manager.cc:1434] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:05:45.494218 21261 ts_tablet_manager.cc:1434] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:05:45.494621 21263 tablet_bootstrap.cc:492] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:45.495479 21261 tablet_bootstrap.cc:492] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:45.495950 21141 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.496130 21262 tablet_bootstrap.cc:654] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.501554 21263 tablet_bootstrap.cc:654] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.502254 21261 tablet_bootstrap.cc:654] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.529775 21263 tablet_bootstrap.cc:492] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:45.529775 21262 tablet_bootstrap.cc:492] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:45.530428 21263 ts_tablet_manager.cc:1403] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.009s
I20260709 13:05:45.530486 21262 ts_tablet_manager.cc:1403] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.051s	user 0.010s	sys 0.003s
I20260709 13:05:45.501761 21507 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.533593 21263 raft_consensus.cc:359] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.534447 21263 raft_consensus.cc:385] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.534919 21263 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.535755 21263 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.538441 21263 ts_tablet_manager.cc:1434] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:45.537616 21262 raft_consensus.cc:359] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.533619 21261 tablet_bootstrap.cc:492] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:45.539058 21262 raft_consensus.cc:385] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.539258 21261 ts_tablet_manager.cc:1403] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.044s	user 0.004s	sys 0.029s
I20260709 13:05:45.539435 21262 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.540213 21262 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.540827 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:45.542400 21036 leader_election.cc:343] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.544178 21262 ts_tablet_manager.cc:1434] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 13:05:45.545306 21262 tablet_bootstrap.cc:492] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:45.545100 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:45.546527 21215 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.547605 21263 tablet_bootstrap.cc:492] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:45.548794 21261 raft_consensus.cc:359] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.549476 21261 raft_consensus.cc:385] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.549741 21261 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.550381 21261 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.552510 21507 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:45.553937 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:45.554400 21496 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:45.556496 21215 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:45.562139 21261 ts_tablet_manager.cc:1434] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:05:45.565757 21263 tablet_bootstrap.cc:654] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.567130 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:45.572640 21475 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:45.573348 21475 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:45.574796 21261 tablet_bootstrap.cc:492] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:45.605618 21145 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.574433 21475 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.676265 21262 tablet_bootstrap.cc:654] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.684616 21143 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.686010 21476 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.686501 21476 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.690790 21431 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.691366 21431 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.693799 21431 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:45.696048 21476 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:45.714028 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:45.746765 21496 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.759052 21496 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.761282 21496 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:45.768792 21261 tablet_bootstrap.cc:654] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:45.768683 21036 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.776695 21263 tablet_bootstrap.cc:492] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:45.777240 21263 ts_tablet_manager.cc:1403] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.230s	user 0.036s	sys 0.041s
I20260709 13:05:45.779702 21266 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.780251 21266 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.787540 20968 catalog_manager.cc:5697] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:45.794005 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:45.859793 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:45.865082 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:45.866597 21221 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:45.863431 21354 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:05:45.860831 21507 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.872040 21507 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.873539 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:45.874073 21507 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:45.874019 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:45.874289 21069 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:45.875461 21263 raft_consensus.cc:359] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.876205 21263 raft_consensus.cc:385] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.876510 21263 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.877990 21262 tablet_bootstrap.cc:492] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
W20260709 13:05:45.878335 21187 leader_election.cc:343] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.878579 21262 ts_tablet_manager.cc:1403] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.333s	user 0.009s	sys 0.020s
I20260709 13:05:45.879511 21263 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.879200 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:05:45.881764 21187 leader_election.cc:343] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.885816 21548 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.886265 21548 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.886649 21548 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.886857 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:45.881237 21266 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:45.890424 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:45.898351 21507 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:45.898792 21507 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.899125 21507 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:45.899200 21263 ts_tablet_manager.cc:1434] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.122s	user 0.008s	sys 0.008s
I20260709 13:05:45.899333 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:45.900713 21263 tablet_bootstrap.cc:492] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
W20260709 13:05:45.921245 21036 leader_election.cc:343] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.946550 21562 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.955649 21562 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.963497 21500 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:05:45.964325 21500 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.945564 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:45.944723 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:45.970366 21216 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:45.970441 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:45.963543 21562 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:45.973875 21558 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.974561 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:45.982183 21263 tablet_bootstrap.cc:654] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:45.968739 21261 tablet_bootstrap.cc:492] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:45.983029 21261 ts_tablet_manager.cc:1403] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.408s	user 0.017s	sys 0.038s
W20260709 13:05:45.983166 21185 leader_election.cc:343] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.984457 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:45.984863 21262 raft_consensus.cc:359] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.985833 21262 raft_consensus.cc:385] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:45.986171 21262 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.986984 21262 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:45.992489 21431 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:45.995126 21262 ts_tablet_manager.cc:1434] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.116s	user 0.006s	sys 0.003s
W20260709 13:05:45.992069 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48698.2us
W20260709 13:05:45.999166 21036 leader_election.cc:343] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:45.999615 21558 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:46.000566 21185 leader_election.cc:343] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.001773 21558 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:46.001946 21185 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.002542 21185 leader_election.cc:343] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.004312 21185 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.004102 21507 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.005085 21495 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.006211 21507 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.007640 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:46.010416 21113 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:05:46.011344 21113 leader_election.cc:343] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.013434 21558 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.014343 21561 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:46.010145 21261 raft_consensus.cc:359] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.018064 21261 raft_consensus.cc:385] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.018445 21261 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.019126 21261 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.020239 21262 tablet_bootstrap.cc:492] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:46.020594 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.021445 21221 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 0.
I20260709 13:05:46.024788 21500 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.029042 21548 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:46.045812 21185 consensus_peers.cc:597] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.031745 21111 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.046994 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.047483 21500 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.047863 21500 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.048184 21500 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.050172 21548 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:46.051606 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:46.054033 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.059207 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:46.059949 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:46.060518 21071 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.060892 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.062088 21354 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.062489 21354 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.062799 21354 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:46.054055 21185 leader_election.cc:343] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.063750 21558 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.064845 21261 ts_tablet_manager.cc:1434] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.000s
I20260709 13:05:46.067090 21071 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:46.071909 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:46.074992 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:46.075786 21221 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.081501 21354 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.086079 21221 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:46.088624 21354 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.088775 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.089767 21495 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.089743 21500 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.090441 21495 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:46.100085 21263 tablet_bootstrap.cc:492] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.100615 21263 ts_tablet_manager.cc:1403] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.200s	user 0.020s	sys 0.022s
I20260709 13:05:46.101073 21261 tablet_bootstrap.cc:492] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
W20260709 13:05:46.101760 21036 leader_election.cc:343] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.091347 21495 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.105046 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:05:46.109157 21113 leader_election.cc:343] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.112327 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:46.115897 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.117311 21568 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.117393 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:46.117838 21568 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:46.118023 21215 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.119326 21262 tablet_bootstrap.cc:654] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.120887 21412 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.121417 21412 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.121840 21143 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.123528 21412 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.118710 21568 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.143005 21263 raft_consensus.cc:359] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.144461 21263 raft_consensus.cc:385] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.144732 21215 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:46.144768 21263 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.146613 21263 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.149256 21263 ts_tablet_manager.cc:1434] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:05:46.150082 21500 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.156795 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:46.157228 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:46.157649 21215 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.150717 21263 tablet_bootstrap.cc:492] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.150411 20967 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:46.165328 21263 tablet_bootstrap.cc:654] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:46.167538 21036 leader_election.cc:343] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.170363 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.170979 21221 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:46.151432 21495 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.154254 21261 tablet_bootstrap.cc:654] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.174373 21495 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:46.176342 21036 leader_election.cc:343] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.181310 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.210355 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:46.207937 21475 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.211864 21475 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:46.212733 21475 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.214845 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.208149 21495 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:46.215893 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:46.217618 21185 leader_election.cc:343] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.219020 21568 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:46.219919 21187 leader_election.cc:343] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.220227 21187 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.224951 21568 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.227906 21142 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.228018 21568 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.225219 21561 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.224694 20967 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.236548 21215 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:05:46.238045 21111 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.239679 21500 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.240201 21500 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:05:46.241096 21500 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.244093 21262 tablet_bootstrap.cc:492] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:46.244627 21262 ts_tablet_manager.cc:1403] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.225s	user 0.027s	sys 0.012s
I20260709 13:05:46.242334 21555 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:46.257381 21261 tablet_bootstrap.cc:492] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:46.257565 21263 tablet_bootstrap.cc:492] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.258255 21261 ts_tablet_manager.cc:1403] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.165s	user 0.015s	sys 0.018s
I20260709 13:05:46.258285 21263 ts_tablet_manager.cc:1403] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.108s	user 0.024s	sys 0.024s
I20260709 13:05:46.261636 21261 raft_consensus.cc:359] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.261816 21263 raft_consensus.cc:359] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.262624 21261 raft_consensus.cc:385] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.263016 21261 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.263875 21261 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.262643 21263 raft_consensus.cc:385] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.265552 21263 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.263520 21262 raft_consensus.cc:359] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.266513 21262 raft_consensus.cc:385] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.266647 21261 ts_tablet_manager.cc:1434] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 13:05:46.266862 21262 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.267627 21262 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.268123 21261 tablet_bootstrap.cc:492] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:46.266537 21263 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.269122 21568 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.270426 21263 ts_tablet_manager.cc:1434] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:05:46.271103 21568 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.272114 21263 tablet_bootstrap.cc:492] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.277107 21262 ts_tablet_manager.cc:1434] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:05:46.277756 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:46.278360 21262 tablet_bootstrap.cc:492] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
W20260709 13:05:46.280040 21036 leader_election.cc:343] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.280512 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:46.281363 21215 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.275233 21261 tablet_bootstrap.cc:654] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.275735 21568 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:05:46.289445 20967 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.295471 21215 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:46.298779 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.300450 21475 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.301752 21475 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:46.303445 20968 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.302635 21475 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.312907 21262 tablet_bootstrap.cc:654] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.326507 21263 tablet_bootstrap.cc:654] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.353953 21261 tablet_bootstrap.cc:492] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:46.354722 21261 ts_tablet_manager.cc:1403] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.087s	user 0.006s	sys 0.009s
I20260709 13:05:46.356207 21263 tablet_bootstrap.cc:492] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.356724 21263 ts_tablet_manager.cc:1403] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.085s	user 0.015s	sys 0.000s
I20260709 13:05:46.357417 21261 raft_consensus.cc:359] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.358363 21261 raft_consensus.cc:385] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.359165 20968 catalog_manager.cc:5697] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.359864 21263 raft_consensus.cc:359] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.360675 21263 raft_consensus.cc:385] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.360982 21263 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.361630 21263 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.363269 21262 tablet_bootstrap.cc:492] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:46.358697 21261 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.363791 21262 ts_tablet_manager.cc:1403] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.086s	user 0.010s	sys 0.010s
I20260709 13:05:46.363826 21263 ts_tablet_manager.cc:1434] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:46.366451 21262 raft_consensus.cc:359] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.365640 21263 tablet_bootstrap.cc:492] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.366997 21262 raft_consensus.cc:385] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.367540 21262 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.368327 21262 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.364231 21261 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.370626 21262 ts_tablet_manager.cc:1434] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:46.371699 21262 tablet_bootstrap.cc:492] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:46.374197 21263 tablet_bootstrap.cc:654] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.377631 21262 tablet_bootstrap.cc:654] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.386368 21263 tablet_bootstrap.cc:492] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.386927 21263 ts_tablet_manager.cc:1403] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.010s
I20260709 13:05:46.389614 21263 raft_consensus.cc:359] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.391496 21263 raft_consensus.cc:385] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.392952 21263 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.393996 21263 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.395601 21261 ts_tablet_manager.cc:1434] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.041s	user 0.000s	sys 0.007s
I20260709 13:05:46.396092 21263 ts_tablet_manager.cc:1434] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:05:46.397956 21263 tablet_bootstrap.cc:492] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.396211 21262 tablet_bootstrap.cc:492] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:46.403642 21262 ts_tablet_manager.cc:1403] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.012s
I20260709 13:05:46.404294 21263 tablet_bootstrap.cc:654] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.406369 21262 raft_consensus.cc:359] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.407081 21262 raft_consensus.cc:385] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.407385 21262 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.408152 21548 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.408087 21262 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.408645 21548 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.410324 21262 ts_tablet_manager.cc:1434] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:46.411649 21262 tablet_bootstrap.cc:492] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:46.412577 21261 tablet_bootstrap.cc:492] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:46.414685 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:46.413080 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:46.415988 21548 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:05:46.416348 21185 leader_election.cc:343] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:46.416873 21187 leader_election.cc:343] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.417380 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.418576 21495 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.413463 21263 tablet_bootstrap.cc:492] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.420408 21261 tablet_bootstrap.cc:654] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.420722 21263 ts_tablet_manager.cc:1403] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.000s
I20260709 13:05:46.422431 21262 tablet_bootstrap.cc:654] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.423709 21263 raft_consensus.cc:359] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.424679 21263 raft_consensus.cc:385] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.428894 21263 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.429782 21263 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.432176 21263 ts_tablet_manager.cc:1434] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:05:46.433198 21263 tablet_bootstrap.cc:492] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.439428 21263 tablet_bootstrap.cc:654] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.440408 21262 tablet_bootstrap.cc:492] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:46.440912 21262 ts_tablet_manager.cc:1403] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.007s
I20260709 13:05:46.442592 21261 tablet_bootstrap.cc:492] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:46.443267 21261 ts_tablet_manager.cc:1403] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.005s
I20260709 13:05:46.445320 21142 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.455013 21261 raft_consensus.cc:359] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.456389 21261 raft_consensus.cc:385] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.456792 21261 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.458400 21261 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.461398 21261 ts_tablet_manager.cc:1434] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:05:46.443625 21262 raft_consensus.cc:359] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.463763 21262 raft_consensus.cc:385] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.464049 21262 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.464802 21263 tablet_bootstrap.cc:492] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.464717 21262 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.465330 21263 ts_tablet_manager.cc:1403] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.008s
I20260709 13:05:46.466318 21262 ts_tablet_manager.cc:1434] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:05:46.467298 21262 tablet_bootstrap.cc:492] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:46.468602 21261 tablet_bootstrap.cc:492] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:46.468473 21555 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.469283 21555 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.471449 21555 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.474053 21262 tablet_bootstrap.cc:654] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.473951 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.474647 21215 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:46.475255 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:46.476476 21261 tablet_bootstrap.cc:654] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:46.477744 21036 leader_election.cc:343] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.477530 21263 raft_consensus.cc:359] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.478451 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.478617 21263 raft_consensus.cc:385] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.479043 21263 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.479964 21555 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.480430 21555 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.480783 21555 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.480064 21263 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.483250 21263 ts_tablet_manager.cc:1434] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:05:46.484935 21263 tablet_bootstrap.cc:492] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.489981 21555 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.496344 21263 tablet_bootstrap.cc:654] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.513629 21555 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.514494 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:46.516274 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:46.516902 21215 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.522984 21215 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:46.526399 21261 tablet_bootstrap.cc:492] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:46.526917 21261 ts_tablet_manager.cc:1403] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.011s
W20260709 13:05:46.530452 21036 leader_election.cc:343] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.531992 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.533607 21475 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.533938 21141 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.534144 21475 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.536182 21475 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.537010 21475 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.540007 21495 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:46.546025 21475 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:46.546981 21475 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.550802 21262 tablet_bootstrap.cc:492] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:46.551545 21262 ts_tablet_manager.cc:1403] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.084s	user 0.028s	sys 0.019s
I20260709 13:05:46.555521 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:46.556777 21263 tablet_bootstrap.cc:492] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.557787 21263 ts_tablet_manager.cc:1403] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.022s
I20260709 13:05:46.559669 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.563840 21263 raft_consensus.cc:359] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.564574 21263 raft_consensus.cc:385] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.564849 21263 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.567353 21558 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.567885 21558 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.568503 21263 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.570130 21558 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.561312 21262 raft_consensus.cc:359] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.571019 21262 raft_consensus.cc:385] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.571314 21262 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.572118 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.571933 21262 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.572947 21221 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:46.574465 21262 ts_tablet_manager.cc:1434] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 13:05:46.570181 21263 ts_tablet_manager.cc:1434] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:05:46.575683 21262 tablet_bootstrap.cc:492] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:46.576284 21263 tablet_bootstrap.cc:492] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.576473 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:46.575991 21036 leader_election.cc:343] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.584264 21263 tablet_bootstrap.cc:654] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.585701 21143 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.591195 21262 tablet_bootstrap.cc:654] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.591846 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.595503 21558 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.619159 21141 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.631021 21558 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.632089 21266 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:46.560735 21261 raft_consensus.cc:359] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.642644 21266 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.643671 21266 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.646052 21266 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:46.646829 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:46.647883 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:46.648217 21187 leader_election.cc:343] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:46.650466 21185 leader_election.cc:343] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.651049 21185 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.652019 21548 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.642834 21261 raft_consensus.cc:385] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.658885 21261 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.659662 21261 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.661218 21261 ts_tablet_manager.cc:1434] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.134s	user 0.008s	sys 0.000s
I20260709 13:05:46.665815 21215 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:46.682669 21431 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.683183 21431 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.685678 21431 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:46.690114 21261 tablet_bootstrap.cc:492] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:46.691133 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:46.691452 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:46.693048 21187 leader_election.cc:343] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:46.693542 21185 leader_election.cc:343] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.693991 21555 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.694005 21185 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.694644 21555 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.695094 21601 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.702082 21555 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.712548 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.713339 21568 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:46.715416 21263 tablet_bootstrap.cc:492] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.715416 21262 tablet_bootstrap.cc:492] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:46.715937 21263 ts_tablet_manager.cc:1403] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.140s	user 0.024s	sys 0.021s
I20260709 13:05:46.716215 21262 ts_tablet_manager.cc:1403] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.141s	user 0.023s	sys 0.084s
I20260709 13:05:46.719874 21568 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.719656 21263 raft_consensus.cc:359] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.720489 21568 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.720528 21263 raft_consensus.cc:385] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.721140 21263 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.721171 21568 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:46.721455 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:46.721957 21263 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.723069 21262 raft_consensus.cc:359] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.723730 21262 raft_consensus.cc:385] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.723984 21262 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.724138 21263 ts_tablet_manager.cc:1434] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:46.725483 21261 tablet_bootstrap.cc:654] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.726604 21263 tablet_bootstrap.cc:492] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.728185 21262 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.729645 21568 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.734213 21263 tablet_bootstrap.cc:654] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.731906 21568 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.742125 21262 ts_tablet_manager.cc:1434] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 13:05:46.742354 21261 tablet_bootstrap.cc:492] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:46.742848 21261 ts_tablet_manager.cc:1403] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.081s	user 0.012s	sys 0.005s
I20260709 13:05:46.743252 21262 tablet_bootstrap.cc:492] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:46.745627 21261 raft_consensus.cc:359] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.746328 21261 raft_consensus.cc:385] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.746680 21261 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.747560 21261 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.749450 21263 tablet_bootstrap.cc:492] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.750020 21263 ts_tablet_manager.cc:1403] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.004s
I20260709 13:05:46.752557 21263 raft_consensus.cc:359] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.752709 21558 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:46.753226 21263 raft_consensus.cc:385] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.753525 21263 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.754041 21262 tablet_bootstrap.cc:654] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.755148 21261 ts_tablet_manager.cc:1434] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.004s
I20260709 13:05:46.754319 21263 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.756196 21261 tablet_bootstrap.cc:492] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:46.756902 21263 ts_tablet_manager.cc:1434] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:46.762271 21568 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.762531 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:46.762821 21568 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.764806 21568 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.765225 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.765982 21261 tablet_bootstrap.cc:654] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.767426 21263 tablet_bootstrap.cc:492] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.772233 21069 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.774718 21263 tablet_bootstrap.cc:654] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.781230 21262 tablet_bootstrap.cc:492] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:46.781064 21558 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:46.781293 20968 catalog_manager.cc:5697] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.783501 21262 ts_tablet_manager.cc:1403] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.006s
I20260709 13:05:46.787055 21262 raft_consensus.cc:359] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.787981 21261 tablet_bootstrap.cc:492] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:46.788509 21261 ts_tablet_manager.cc:1403] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.000s
I20260709 13:05:46.793218 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:46.793974 21220 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.792619 21261 raft_consensus.cc:359] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.795037 21261 raft_consensus.cc:385] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.795328 21261 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.796113 21263 tablet_bootstrap.cc:492] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.796070 21261 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.796626 21263 ts_tablet_manager.cc:1403] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.009s
I20260709 13:05:46.798298 21261 ts_tablet_manager.cc:1434] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:46.787989 21262 raft_consensus.cc:385] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.799245 21263 raft_consensus.cc:359] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.799829 21262 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.800009 21263 raft_consensus.cc:385] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.800282 21263 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.800493 21262 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.800906 21263 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.802899 21262 ts_tablet_manager.cc:1434] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:05:46.803947 21262 tablet_bootstrap.cc:492] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:46.804641 21263 ts_tablet_manager.cc:1434] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:46.805610 21263 tablet_bootstrap.cc:492] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.809720 21262 tablet_bootstrap.cc:654] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.799247 21261 tablet_bootstrap.cc:492] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:46.811339 21263 tablet_bootstrap.cc:654] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.813800 21601 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.814325 21601 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.816759 21601 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:46.818401 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:46.818789 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:46.819108 21068 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:46.819677 21140 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:46.819036 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:46.820372 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:46.821695 21261 tablet_bootstrap.cc:654] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.822201 21601 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.823109 21601 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.823571 21601 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.825057 21220 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:46.831594 21261 tablet_bootstrap.cc:492] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:46.831804 21143 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.832122 21261 ts_tablet_manager.cc:1403] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.005s
I20260709 13:05:46.835552 21263 tablet_bootstrap.cc:492] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:46.836063 21263 ts_tablet_manager.cc:1403] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.004s
I20260709 13:05:46.838866 21263 raft_consensus.cc:359] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.839672 21263 raft_consensus.cc:385] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.839944 21263 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.842398 21263 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.850934 21263 ts_tablet_manager.cc:1434] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:05:46.852144 21263 tablet_bootstrap.cc:492] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:46.857090 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.856918 21601 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:46.861195 21036 leader_election.cc:343] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:46.862432 21036 leader_election.cc:343] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.864362 21555 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:05:46.864264 21558 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.869598 21558 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.871596 21558 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.872219 21140 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.874394 21261 raft_consensus.cc:359] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.874825 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.875653 21261 raft_consensus.cc:385] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.876065 21261 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.877020 21261 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.880079 21261 ts_tablet_manager.cc:1434] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 13:05:46.881567 21261 tablet_bootstrap.cc:492] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:46.882215 21218 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:46.889896 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:05:46.890136 21036 leader_election.cc:343] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:46.891927 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:46.890938 21036 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.893719 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.888239 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:46.893368 21555 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.895138 21475 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:46.894124 21558 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.895699 21475 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:46.896359 21354 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:46.898206 21475 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.898715 21555 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:46.908330 21262 tablet_bootstrap.cc:492] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:46.908883 21262 ts_tablet_manager.cc:1403] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.105s	user 0.027s	sys 0.021s
I20260709 13:05:46.912941 21262 raft_consensus.cc:359] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.913936 21262 raft_consensus.cc:385] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:46.914289 21262 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.920027 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:46.920641 21146 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.926048 21568 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.926597 21568 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.928671 21568 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:46.928794 21263 tablet_bootstrap.cc:654] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:46.917061 21601 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:46.924563 21262 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.934651 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:46.937852 21412 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:46.938719 21412 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:46.943216 21262 ts_tablet_manager.cc:1434] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.001s
I20260709 13:05:46.945766 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:46.946457 21140 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:46.949318 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:46.949990 21215 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:46.951731 21431 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.952427 21431 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:46.975138 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:05:46.980728 21036 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.983215 21555 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:46.983870 21555 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.984342 21555 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:46.986754 21261 tablet_bootstrap.cc:654] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.018597 21146 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:47.040593 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:47.086035 21500 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.086787 21500 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.108451 21263 tablet_bootstrap.cc:492] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:47.109086 21263 ts_tablet_manager.cc:1403] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.257s	user 0.007s	sys 0.011s
I20260709 13:05:47.109300 21555 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.132679 21263 raft_consensus.cc:359] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.133745 21263 raft_consensus.cc:385] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.134114 21263 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.128772 21475 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.138192 21263 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.138504 21475 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.140453 21475 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:47.141135 21431 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:47.141733 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:47.144014 21555 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:47.144644 21111 leader_election.cc:343] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.148754 21263 ts_tablet_manager.cc:1434] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:05:47.149816 21263 tablet_bootstrap.cc:492] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:47.164599 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:47.165405 21262 tablet_bootstrap.cc:492] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:47.165316 21069 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.172423 21261 tablet_bootstrap.cc:492] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:47.173094 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:47.174177 21261 ts_tablet_manager.cc:1403] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.293s	user 0.011s	sys 0.025s
I20260709 13:05:47.174177 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:47.148358 21500 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:47.176527 21431 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:47.179162 21187 leader_election.cc:343] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.180536 21431 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:47.181818 21431 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:47.191498 21113 leader_election.cc:343] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.192013 21113 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:47.193481 21616 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:47.179678 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:47.198974 21261 raft_consensus.cc:359] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.199764 21261 raft_consensus.cc:385] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.200073 21261 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
W20260709 13:05:47.200119 21185 leader_election.cc:343] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.201144 21568 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.202548 21185 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:47.205739 21069 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:47.200824 21261 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.206794 21568 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.209954 21263 tablet_bootstrap.cc:654] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.217649 21262 tablet_bootstrap.cc:654] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.216678 21561 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:47.240787 21261 ts_tablet_manager.cc:1434] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.003s
I20260709 13:05:47.235131 21568 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:47.268693 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:47.269366 21144 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.269502 21261 tablet_bootstrap.cc:492] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:47.263095 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:47.283656 21431 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.284700 21431 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.288138 20969 catalog_manager.cc:5697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.303051 21431 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:05:47.304960 21036 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:47.306804 21144 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:47.308725 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:47.309149 21263 tablet_bootstrap.cc:492] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:47.309449 21214 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:47.309628 21263 ts_tablet_manager.cc:1403] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.160s	user 0.032s	sys 0.016s
I20260709 13:05:47.310761 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:47.311399 21214 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.312332 21261 tablet_bootstrap.cc:654] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.312600 21263 raft_consensus.cc:359] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.313354 21263 raft_consensus.cc:385] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.313616 21263 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
W20260709 13:05:47.318856 21036 leader_election.cc:343] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.319715 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:47.320636 21263 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.321403 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:47.321990 21142 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:47.328533 21568 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:47.329137 21568 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:47.331759 21261 tablet_bootstrap.cc:492] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:47.332237 21261 ts_tablet_manager.cc:1403] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.080s	user 0.000s	sys 0.013s
I20260709 13:05:47.336359 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:47.336123 21263 ts_tablet_manager.cc:1434] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.002s
I20260709 13:05:47.339882 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:47.346551 21568 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.343775 21561 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:47.350106 21561 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.352066 21561 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.359831 21263 tablet_bootstrap.cc:492] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:47.354844 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:47.362358 21068 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:47.361605 21261 raft_consensus.cc:359] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.363683 21261 raft_consensus.cc:385] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.363941 21261 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.364614 21261 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.366537 21214 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:47.367794 21261 ts_tablet_manager.cc:1434] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.004s
I20260709 13:05:47.372788 21261 tablet_bootstrap.cc:492] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
W20260709 13:05:47.376530 21173 tablet.cc:2406] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:47.391556 21263 tablet_bootstrap.cc:654] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:47.391690 21036 leader_election.cc:343] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.397244 21261 tablet_bootstrap.cc:654] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.399896 21624 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:47.406940 21036 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:47.407437 21262 tablet_bootstrap.cc:492] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:47.407691 21624 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:47.407900 21262 ts_tablet_manager.cc:1403] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.243s	user 0.014s	sys 0.031s
I20260709 13:05:47.408071 21624 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.408407 21624 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.410692 21262 raft_consensus.cc:359] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.411473 21262 raft_consensus.cc:385] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.411753 21262 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.412439 21262 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.413987 21262 ts_tablet_manager.cc:1434] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:47.415016 21262 tablet_bootstrap.cc:492] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:47.378588 21561 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.414921 20967 catalog_manager.cc:5697] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.419646 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:47.421171 21143 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.419924 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:47.420070 21561 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:47.423667 21214 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:47.428903 21143 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:47.430243 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:47.431384 21601 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:47.431521 21036 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:47.432047 21601 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:47.419942 21624 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.433122 21601 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.434468 21624 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:47.436314 21262 tablet_bootstrap.cc:654] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.436669 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:47.437507 21145 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:47.440444 21561 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.440960 21561 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.442927 21561 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:47.444411 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:47.445569 21185 leader_election.cc:343] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.445927 21263 tablet_bootstrap.cc:492] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:47.446401 21263 ts_tablet_manager.cc:1403] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.093s	user 0.018s	sys 0.009s
I20260709 13:05:47.449093 21263 raft_consensus.cc:359] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.449740 21263 raft_consensus.cc:385] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.450121 21263 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.451231 21263 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.454334 21262 tablet_bootstrap.cc:492] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:47.454912 21262 ts_tablet_manager.cc:1403] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.001s
I20260709 13:05:47.454322 21263 ts_tablet_manager.cc:1434] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:47.457412 21263 tablet_bootstrap.cc:492] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:47.457473 21262 raft_consensus.cc:359] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.458225 21262 raft_consensus.cc:385] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.458503 21262 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.459251 21262 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.464159 21262 ts_tablet_manager.cc:1434] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:05:47.465457 21262 tablet_bootstrap.cc:492] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:47.466751 21263 tablet_bootstrap.cc:654] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.470623 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:47.472126 21262 tablet_bootstrap.cc:654] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.484858 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:47.485311 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:47.487514 21068 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:47.487829 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:47.488610 21070 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:47.488515 21624 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:47.489192 21624 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.489614 21624 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.497653 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:47.498454 21601 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:47.498792 21601 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.499109 21601 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.504503 21036 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:47.506237 21568 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:47.507231 21601 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.509609 21261 tablet_bootstrap.cc:492] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:47.510130 21261 ts_tablet_manager.cc:1403] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.138s	user 0.020s	sys 0.014s
I20260709 13:05:47.510416 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:47.511838 21185 leader_election.cc:343] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.513587 21261 raft_consensus.cc:359] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.514580 21261 raft_consensus.cc:385] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.514931 21261 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.515648 21601 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:47.516800 21601 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.517227 21601 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.519953 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:47.520141 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:47.521570 21185 leader_election.cc:343] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.522264 21070 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.522563 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:47.498997 21624 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.527561 21616 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:47.527163 21187 leader_election.cc:343] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.527765 21568 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:47.528057 21616 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.529361 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:47.529997 21601 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:47.529866 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:47.530295 21616 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:47.530685 21561 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:47.532035 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:05:47.533313 21111 leader_election.cc:343] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.534539 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:47.535466 21068 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 0.
I20260709 13:05:47.540345 20969 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.541272 21113 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:05:47.542693 21616 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:47.543211 21616 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.543653 21616 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.541193 21070 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
W20260709 13:05:47.551373 21036 leader_election.cc:343] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.555963 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:47.556818 21561 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:47.557353 21561 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:47.557920 21624 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:47.558254 21561 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.559175 21261 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.561525 21261 ts_tablet_manager.cc:1434] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 13:05:47.562866 21261 tablet_bootstrap.cc:492] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Bootstrap starting.
I20260709 13:05:47.570235 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:47.571141 21215 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.572095 21263 tablet_bootstrap.cc:492] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:47.572616 21263 ts_tablet_manager.cc:1403] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.116s	user 0.011s	sys 0.001s
I20260709 13:05:47.556944 21616 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.585686 21616 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:47.587328 21215 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:47.588438 21262 tablet_bootstrap.cc:492] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:47.588569 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:47.583024 21263 raft_consensus.cc:359] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.588903 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:47.588923 21262 ts_tablet_manager.cc:1403] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.124s	user 0.023s	sys 0.081s
I20260709 13:05:47.589772 21624 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:47.590287 21624 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:47.591116 21624 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:47.591727 21111 leader_election.cc:343] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:47.592260 21262 raft_consensus.cc:359] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.592990 21262 raft_consensus.cc:385] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.592808 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:47.593346 21262 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.593618 21069 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.594126 21262 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.595539 21263 raft_consensus.cc:385] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.595834 21263 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.596683 21262 ts_tablet_manager.cc:1434] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:47.597797 21262 tablet_bootstrap.cc:492] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:47.606520 21262 tablet_bootstrap.cc:654] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.618793 21261 tablet_bootstrap.cc:654] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.614099 21069 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:05:47.634092 21113 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:05:47.644954 21616 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:47.645654 21616 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:05:47.648859 21263 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.646514 21616 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.655185 21263 ts_tablet_manager.cc:1434] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.082s	user 0.005s	sys 0.000s
I20260709 13:05:47.709321 21262 tablet_bootstrap.cc:492] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:47.710162 21262 ts_tablet_manager.cc:1403] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.113s	user 0.042s	sys 0.037s
I20260709 13:05:47.713948 21262 raft_consensus.cc:359] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.715042 21262 raft_consensus.cc:385] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.715512 21262 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.716470 21262 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.719502 21262 ts_tablet_manager.cc:1434] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:05:47.721134 21262 tablet_bootstrap.cc:492] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:47.729739 21262 tablet_bootstrap.cc:654] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.733975 21601 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.734685 21601 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.735296 20967 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.736764 21601 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:47.745582 21262 tablet_bootstrap.cc:492] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:47.746395 21262 ts_tablet_manager.cc:1403] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.003s
I20260709 13:05:47.749902 21262 raft_consensus.cc:359] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.750955 21262 raft_consensus.cc:385] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.751407 21262 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.752524 21262 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.754225 21263 tablet_bootstrap.cc:492] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:47.757215 21262 ts_tablet_manager.cc:1434] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:05:47.777649 21262 tablet_bootstrap.cc:492] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:47.783774 21261 tablet_bootstrap.cc:492] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: No bootstrap required, opened a new log
I20260709 13:05:47.784523 21261 ts_tablet_manager.cc:1403] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent bootstrapping tablet: real 0.222s	user 0.044s	sys 0.048s
I20260709 13:05:47.785633 21263 tablet_bootstrap.cc:654] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.788007 21261 raft_consensus.cc:359] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.793165 21261 raft_consensus.cc:385] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.793466 21261 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.794113 21261 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.796233 21262 tablet_bootstrap.cc:654] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.801002 21261 ts_tablet_manager.cc:1434] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:05:47.803556 21263 tablet_bootstrap.cc:492] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:47.804093 21263 ts_tablet_manager.cc:1403] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.009s
I20260709 13:05:47.804620 21262 tablet_bootstrap.cc:492] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:47.805135 21262 ts_tablet_manager.cc:1403] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.002s
I20260709 13:05:47.806988 21263 raft_consensus.cc:359] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.808589 21263 raft_consensus.cc:385] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.808315 21262 raft_consensus.cc:359] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.808904 21263 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.809099 21262 raft_consensus.cc:385] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.809391 21262 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.809696 21263 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.810140 21262 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.812011 21263 ts_tablet_manager.cc:1434] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:47.812198 21262 ts_tablet_manager.cc:1434] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:47.813139 21263 tablet_bootstrap.cc:492] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:47.813139 21262 tablet_bootstrap.cc:492] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Bootstrap starting.
I20260709 13:05:47.821316 21263 tablet_bootstrap.cc:654] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.822728 21262 tablet_bootstrap.cc:654] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.831868 21262 tablet_bootstrap.cc:492] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: No bootstrap required, opened a new log
I20260709 13:05:47.832541 21262 ts_tablet_manager.cc:1403] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.005s
I20260709 13:05:47.835628 21262 raft_consensus.cc:359] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.836617 21262 raft_consensus.cc:385] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.837059 21262 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.837694 21263 tablet_bootstrap.cc:492] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:47.838332 21263 ts_tablet_manager.cc:1403] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.003s
I20260709 13:05:47.837718 21262 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.840644 21262 ts_tablet_manager.cc:1434] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:05:47.841492 21263 raft_consensus.cc:359] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.842291 21263 raft_consensus.cc:385] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.842525 21263 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.843056 21263 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:47.844250 21263 ts_tablet_manager.cc:1434] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:05:48.029047 21507 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.029798 21507 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.044975 21507 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.057993 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:48.058666 21070 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:48.071298 21500 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.071936 21500 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.074177 21500 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.074697 21500 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.075138 21500 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.076926 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:48.078063 21500 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.079428 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:48.075129 21661 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.084616 21659 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.085642 21659 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.087739 21659 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.088260 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:48.088881 21143 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:48.104085 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:48.104518 21145 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.080906 21661 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.106091 21143 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.120100 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:48.138124 21661 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.144582 21658 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.145370 21658 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.152894 21661 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.167147 21658 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.197767 21185 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:05:48.210774 21629 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.299599 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 101979us
I20260709 13:05:47.844996 21263 tablet_bootstrap.cc:492] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:48.326761 21629 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.344386 21629 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.323947 21666 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.382151 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:48.386732 21141 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:48.445890 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 116608us
I20260709 13:05:48.339653 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:48.426946 21666 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.464506 21670 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.456996 21666 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.482942 21661 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.494164 21661 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.501802 21661 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.495050 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:48.506553 21215 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:48.512036 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:48.515915 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:48.516578 21673 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.517022 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:48.517051 21673 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:05:48.521992 21036 leader_election.cc:343] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.526238 21263 tablet_bootstrap.cc:654] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.527251 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:48.533272 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:48.534603 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:48.535216 21219 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:48.530479 21221 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
W20260709 13:05:48.543143 21036 leader_election.cc:343] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.541095 21212 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.542392 21668 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:05:48.536230 21111 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:48.556648 21673 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:48.571000 21147 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.596223 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:48.608525 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79887us
W20260709 13:05:48.617707 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48627.7us
I20260709 13:05:48.620534 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:48.626035 21561 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.654206 21263 tablet_bootstrap.cc:492] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:48.654891 21263 ts_tablet_manager.cc:1403] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.810s	user 0.027s	sys 0.011s
I20260709 13:05:48.631980 21670 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:05:48.660799 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:48.663527 21666 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.664201 21561 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.673580 21661 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:48.675163 21561 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:48.677567 21666 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:48.679670 21601 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:48.680403 21263 raft_consensus.cc:359] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.681151 21263 raft_consensus.cc:385] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.681509 21263 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.682123 21263 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.670594 21659 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:48.689287 21678 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:05:48.691603 21263 ts_tablet_manager.cc:1434] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.004s
I20260709 13:05:48.694406 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:48.691248 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:48.695154 21070 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:48.696596 21660 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:48.696563 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:48.685953 21659 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.697587 20967 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 79) took 1131 ms. Trace:
I20260709 13:05:48.697904 21660 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:48.698009 21659 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.698794 21660 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:48.698810 20969 catalog_manager.cc:5697] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:48.708055 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:48.709796 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:48.705780 20968 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.711933 21263 tablet_bootstrap.cc:492] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:48.713223 21659 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.719480 21685 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.720011 21685 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.721644 21263 tablet_bootstrap.cc:654] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.722501 21685 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:48.724073 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:48.724779 21069 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 0.
I20260709 13:05:48.726171 21475 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:48.715476 21662 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.727404 21659 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.727660 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:48.727726 21662 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.728583 21662 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.743183 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:48.744052 21217 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 0.
I20260709 13:05:48.746044 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:48.747097 21659 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.747601 21659 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.748394 21659 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.748528 21068 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.753270 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:48.754128 21070 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:48.757488 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:48.759136 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:48.759262 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:05:48.762348 21036 leader_election.cc:343] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:48.768023 21672 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.319s	user 0.009s	sys 0.025s
I20260709 13:05:48.768158 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:48.768784 21212 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:48.770371 21036 leader_election.cc:343] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.766592 21660 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:48.775871 21036 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:48.778061 21666 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.698050 20967 rpcz_store.cc:276] 0709 13:05:47.566131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.566286 (+   155us) service_pool.cc:224] Handling call
0709 13:05:48.697496 (+1131210us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:47.769108 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:47.770317 (+  1209us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:47.782359 (+ 12042us) write_op.cc:435] Acquired schema lock
0709 13:05:47.782399 (+    40us) tablet.cc:662] Decoding operations
0709 13:05:47.782905 (+   506us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:47.782981 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:47.783031 (+    50us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:47.783296 (+   265us) tablet.cc:701] Row locks acquired
0709 13:05:47.783323 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:05:47.783470 (+   147us) write_op.cc:270] Start()
0709 13:05:47.783595 (+   125us) write_op.cc:276] Timestamp: P: 1783602347783424 usec, L: 0
0709 13:05:47.783617 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:05:47.784278 (+   661us) log.cc:844] Serialized 543 byte log entry
0709 13:05:47.786545 (+  2267us) op_driver.cc:464] REPLICATION: finished
0709 13:05:47.786954 (+   409us) write_op.cc:301] APPLY: starting
0709 13:05:47.787034 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:47.787151 (+   117us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:47.787256 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:47.787463 (+   207us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:47.787610 (+   147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:47.787656 (+    46us) write_op.cc:312] APPLY: finished
0709 13:05:47.787865 (+   209us) log.cc:844] Serialized 22 byte log entry
0709 13:05:47.788055 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:47.788137 (+    82us) write_op.cc:454] Released schema lock
0709 13:05:47.788182 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:47.788481 (+   299us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:47.788508 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":136687,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":224,"apply.run_cpu_time_us":1490,"apply.run_wall_time_us":1490,"num_ops":1,"prepare.queue_time_us":133,"prepare.run_cpu_time_us":8745,"prepare.run_wall_time_us":16296,"replication_time_us":2826}]]}
I20260709 13:05:48.757524 21659 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.774513 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:48.792179 21671 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:48.793358 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:48.802462 21685 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.803035 21685 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.796727 21142 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.803480 21685 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:48.853624 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52921.8us
W20260709 13:05:48.854687 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56268.7us
I20260709 13:05:48.860854 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:05:48.862315 21036 leader_election.cc:343] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.864272 21263 tablet_bootstrap.cc:492] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:48.864816 21263 ts_tablet_manager.cc:1403] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.153s	user 0.026s	sys 0.020s
I20260709 13:05:48.902195 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:48.909381 21631 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:48.920996 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08001ddc80 after lost signal to thread 19970
I20260709 13:05:48.929840 21475 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:48.932883 21631 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.939138 21631 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.940502 21212 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:48.942363 21657 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.942889 21657 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.944701 21263 raft_consensus.cc:359] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.945092 21657 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.945423 21263 raft_consensus.cc:385] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.946305 21263 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.916415 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:48.949558 21263 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.959656 21659 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:48.961336 21263 ts_tablet_manager.cc:1434] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 13:05:48.962455 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:05:48.961877 21185 leader_election.cc:343] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.965803 21263 tablet_bootstrap.cc:492] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
W20260709 13:05:48.968205 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08001bdf60 after lost signal to thread 21004
I20260709 13:05:48.970562 21068 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:48.956212 21662 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:48.999825 21685 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.001737 21662 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:49.005322 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:48.992327 21263 tablet_bootstrap.cc:654] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:48.998293 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800096f00 after lost signal to thread 21096
W20260709 13:05:49.022076 21185 leader_election.cc:343] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:49.028877 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800217be0 after lost signal to thread 21171
I20260709 13:05:49.029284 21674 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:49.030365 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08002759c0 after lost signal to thread 21245
W20260709 13:05:49.030809 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800275a00 after lost signal to thread 21246
I20260709 13:05:49.032413 21685 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:49.032964 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:49.033751 21068 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:05:49.031253 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800275880 after lost signal to thread 21266
W20260709 13:05:49.042273 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79895.8us
I20260709 13:05:49.011368 21671 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:05:49.047660 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:49.041261 21674 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:49.052536 21697 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:49.041446 21142 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:49.044286 21694 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.052165 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:49.058722 21658 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:49.063374 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:49.069365 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:49.059948 21697 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:49.110822 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44398.5us
W20260709 13:05:49.115808 21036 leader_election.cc:343] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:49.070263 21658 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:05:49.122970 21185 leader_election.cc:343] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:49.125605 21185 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:49.075181 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:49.132499 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:49.133074 21221 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.066243 21694 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.110062 21697 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.139639 21263 tablet_bootstrap.cc:492] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:49.140262 21263 ts_tablet_manager.cc:1403] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.175s	user 0.022s	sys 0.004s
I20260709 13:05:49.137670 21673 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:05:49.158329 21036 leader_election.cc:343] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:49.159209 21221 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:49.120702 21631 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.144376 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:49.166103 21631 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:49.166287 21683 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:49.145959 21681 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.131564 21674 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.162319 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:05:49.195169 21702 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:05:49.167730 21658 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.202464 21263 raft_consensus.cc:359] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.203207 21263 raft_consensus.cc:385] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:49.203518 21263 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.207831 21263 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.212431 21263 ts_tablet_manager.cc:1434] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.001s
I20260709 13:05:49.195243 21681 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.243854 21661 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.256821 21694 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:49.299218 21661 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.308429 21697 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.352172 21661 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:49.370625 21697 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:05:49.372829 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 238153us
I20260709 13:05:49.379818 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:05:49.377418 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:49.381718 21067 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.385625 21218 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:49.391686 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:49.388999 21681 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:49.405575 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:49.406397 21221 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:49.401144 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:05:49.406317 21185 leader_election.cc:343] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:49.398434 21067 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:05:49.407816 21263 tablet_bootstrap.cc:492] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:49.418496 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:49.419932 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:49.420148 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:49.424130 21070 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:49.426647 20969 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 105) took 1664 ms. Trace:
I20260709 13:05:49.427066 20968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 111) took 1319 ms. Trace:
I20260709 13:05:49.429121 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:49.430192 21212 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:49.433425 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:49.433794 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:49.434847 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:49.437007 21660 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:49.445014 21263 tablet_bootstrap.cc:654] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:49.427443 20968 rpcz_store.cc:276] 0709 13:05:48.107734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.107907 (+   173us) service_pool.cc:224] Handling call
0709 13:05:49.427011 (+1319104us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:48.979702 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:48.980275 (+   573us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:48.980303 (+    28us) write_op.cc:435] Acquired schema lock
0709 13:05:48.980321 (+    18us) tablet.cc:662] Decoding operations
0709 13:05:48.980660 (+   339us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:48.980703 (+    43us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:48.980733 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:48.980900 (+   167us) tablet.cc:701] Row locks acquired
0709 13:05:48.980918 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:05:48.981073 (+   155us) write_op.cc:270] Start()
0709 13:05:48.981181 (+   108us) write_op.cc:276] Timestamp: P: 1783602348981026 usec, L: 0
0709 13:05:48.981203 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:05:48.981805 (+   602us) log.cc:844] Serialized 543 byte log entry
0709 13:05:49.052925 (+ 71120us) op_driver.cc:464] REPLICATION: finished
0709 13:05:49.228214 (+175289us) write_op.cc:301] APPLY: starting
0709 13:05:49.228286 (+    72us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:49.228383 (+    97us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:49.228478 (+    95us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:49.228658 (+   180us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:49.228806 (+   148us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:49.228853 (+    47us) write_op.cc:312] APPLY: finished
0709 13:05:49.229034 (+   181us) log.cc:844] Serialized 22 byte log entry
0709 13:05:49.229184 (+   150us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:49.229260 (+    76us) write_op.cc:454] Released schema lock
0709 13:05:49.229298 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:49.229589 (+   291us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:49.229614 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":542354,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":175085,"apply.run_cpu_time_us":1359,"apply.run_wall_time_us":1359,"num_ops":1,"prepare.queue_time_us":265814,"prepare.run_cpu_time_us":2331,"prepare.run_wall_time_us":2330,"replication_time_us":71596,"thread_start_us":163337,"threads_started":1}]]}
I20260709 13:05:49.427064 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:49.458557 21144 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.461180 21718 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:49.462739 21036 leader_election.cc:343] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:49.470084 21113 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:49.471233 21660 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.472365 21677 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:49.485798 21677 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:05:49.486616 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:49.487202 21221 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.486766 21677 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.487780 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:49.427040 20969 rpcz_store.cc:276] 0709 13:05:47.762564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.763264 (+   700us) service_pool.cc:224] Handling call
0709 13:05:49.426584 (+1663320us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:48.777798 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:48.778381 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:48.778410 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:05:48.778430 (+    20us) tablet.cc:662] Decoding operations
0709 13:05:48.788915 (+ 10485us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:48.793945 (+  5030us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:48.797344 (+  3399us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:48.843196 (+ 45852us) tablet.cc:701] Row locks acquired
0709 13:05:48.845704 (+  2508us) write_op.cc:260] PREPARE: finished
0709 13:05:48.873785 (+ 28081us) write_op.cc:270] Start()
0709 13:05:48.892737 (+ 18952us) write_op.cc:276] Timestamp: P: 1783602348863048 usec, L: 0
0709 13:05:48.893263 (+   526us) op_driver.cc:348] REPLICATION: starting
0709 13:05:48.955908 (+ 62645us) log.cc:844] Serialized 543 byte log entry
0709 13:05:49.016496 (+ 60588us) op_driver.cc:464] REPLICATION: finished
0709 13:05:49.142418 (+125922us) write_op.cc:301] APPLY: starting
0709 13:05:49.143991 (+  1573us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:49.144100 (+   109us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:49.176719 (+ 32619us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:49.206943 (+ 30224us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:49.218011 (+ 11068us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:49.219721 (+  1710us) write_op.cc:312] APPLY: finished
0709 13:05:49.227161 (+  7440us) log.cc:844] Serialized 22 byte log entry
0709 13:05:49.227405 (+   244us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:49.227482 (+    77us) write_op.cc:454] Released schema lock
0709 13:05:49.227519 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:49.227859 (+   340us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:49.227884 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":624852,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":93733,"apply.run_cpu_time_us":2691,"apply.run_wall_time_us":91697,"num_ops":1,"prepare.queue_time_us":74967,"prepare.run_cpu_time_us":5284,"prepare.run_wall_time_us":201719,"raft.queue_time_us":67927,"raft.run_cpu_time_us":37301,"raft.run_wall_time_us":201408,"replication_time_us":123062,"thread_start_us":153380,"threads_started":4,"wal-append.queue_time_us":19214}]]}
I20260709 13:05:49.495949 21700 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:49.508337 21221 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:49.511021 21700 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:49.513144 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:49.516880 21660 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:49.512048 21700 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.522173 21710 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:49.530218 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:49.553814 21657 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:49.567462 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:05:49.569738 21629 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:49.570335 21629 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:49.571158 21629 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.673013 21263 tablet_bootstrap.cc:492] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:49.673933 21263 ts_tablet_manager.cc:1403] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.266s	user 0.017s	sys 0.050s
I20260709 13:05:49.673830 20969 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.679363 20969 catalog_manager.cc:5697] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.680008 21718 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:49.694801 21263 raft_consensus.cc:359] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.695851 21263 raft_consensus.cc:385] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:49.696185 21263 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.697005 21263 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.705423 21263 ts_tablet_manager.cc:1434] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 13:05:49.707921 21263 tablet_bootstrap.cc:492] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:49.708344 21626 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:05:49.721858 21263 tablet_bootstrap.cc:654] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:49.738571 21263 tablet_bootstrap.cc:492] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:49.739106 21263 ts_tablet_manager.cc:1403] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.001s
I20260709 13:05:49.742305 21263 raft_consensus.cc:359] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.743121 21263 raft_consensus.cc:385] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:49.743429 21263 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.744184 21263 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.746605 21263 ts_tablet_manager.cc:1434] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:49.747812 21263 tablet_bootstrap.cc:492] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Bootstrap starting.
I20260709 13:05:49.757169 21263 tablet_bootstrap.cc:654] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:49.759783 21668 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.768666 21561 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.770226 21719 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:49.781764 21263 tablet_bootstrap.cc:492] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: No bootstrap required, opened a new log
I20260709 13:05:49.782580 21263 ts_tablet_manager.cc:1403] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.005s
I20260709 13:05:49.785420 21263 raft_consensus.cc:359] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.786178 21263 raft_consensus.cc:385] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:49.786460 21263 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.787350 21263 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.789911 21263 ts_tablet_manager.cc:1434] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:05:49.831799 20968 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.835809 20968 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:49.877128 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:49.887945 20969 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.889454 20969 catalog_manager.cc:5697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.982901 21718 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.983428 21718 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:49.985543 21718 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:49.987155 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:49.987730 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:49.988319 21216 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:49.989575 21111 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:49.990393 21718 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.999552 21718 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:50.050719 21718 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.051306 21718 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:50.054639 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:50.055033 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:50.055627 21069 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:50.054042 21718 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:50.057008 21113 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:50.057873 21718 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:50.069083 21718 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:50.068969 21685 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:50.080510 21729 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:50.088624 21657 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:50.094285 21719 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:50.103329 21718 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.103925 21718 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:50.108099 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:05:50.109439 21067 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:05:50.111827 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:50.106211 21718 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:50.117067 21216 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 1: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:05:50.121567 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:05:50.123647 21718 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:50.137593 21729 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:50.175803 21718 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:50.193850 21657 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:50.911887 21018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:51.217645 21626 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.218252 21626 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:51.220502 21626 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:51.221331 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:05:51.221869 21144 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:51.221895 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:05:51.222512 21220 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:05:51.222970 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:51.223737 21626 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:51.224186 21626 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.224548 21626 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.231905 21626 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:51.234040 21626 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:51.235232 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:05:51.235766 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:05:51.235898 21144 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.236354 21220 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:51.247146 21144 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:05:51.248504 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:05:51.249732 21668 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:51.250190 21668 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:05:51.250418 21220 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:05:51.251040 21668 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:05:51.280354 20968 catalog_manager.cc:5697] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 0 to 2, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:51.406913 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:51.449797 21756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:51.449963 21755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:51.451076 19967 server_base.cc:1061] running on GCE node
W20260709 13:05:51.451468 21758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:51.452950 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:51.453298 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:51.453548 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602351453517 us; error 0 us; skew 500 ppm
I20260709 13:05:51.454147 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:51.458622 19967 webserver.cc:533] Webserver started at http://127.19.127.196:36051/ using document root <none> and password file <none>
I20260709 13:05:51.459318 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:51.459568 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:51.459951 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:51.461200 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-3-root/instance:
uuid: "b70fc04de37e4239a8336534f0c9c0fb"
format_stamp: "Formatted at 2026-07-09 13:05:51 on dist-test-slave-s3n6"
I20260709 13:05:51.466460 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:05:51.473654 21763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:51.475541 19967 fs_manager.cc:730] Time spent opening block manager: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:05:51.475860 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-3-root
uuid: "b70fc04de37e4239a8336534f0c9c0fb"
format_stamp: "Formatted at 2026-07-09 13:05:51 on dist-test-slave-s3n6"
I20260709 13:05:51.476182 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:51.494771 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:51.496564 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:51.498759 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:51.502007 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:51.502269 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:51.502571 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:51.502779 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:51.635265 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.196:33295
I20260709 13:05:51.635433 21825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.196:33295 every 8 connection(s)
I20260709 13:05:51.641510 19967 auto_leader_rebalancer-test.cc:384] add a tserver: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:51.662230 21826 heartbeater.cc:344] Connected to a master server at 127.19.127.254:37743
I20260709 13:05:51.664485 21826 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:51.665657 21826 heartbeater.cc:507] Master 127.19.127.254:37743 requested a full tablet report, sending...
I20260709 13:05:51.668241 20968 ts_manager.cc:194] Registered new tserver with Master: b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:05:51.670121 20968 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38938
I20260709 13:05:51.693374 21139 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:51.695137 21748 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:51.719751 21212 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:51.728324 21747 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:52.069977 21212 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:52.080971 21143 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:52.082334 21719 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:52.083803 21069 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:52.085587 21561 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:52.111047 21719 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.114756 21139 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.115392 21069 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:05:52.116328 21184 consensus_peers.cc:597] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d693d94d4c0943d9ab08114c33fc2e7f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:52.129168 20969 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:52.145816 21067 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:52.179510 21212 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:52.179781 21139 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:52.181571 21729 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:52.182364 21748 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:52.209571 21035 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b4ee2463d8894a95be9b747f813f9106. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:52.215010 21668 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.221709 21143 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.228550 21212 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.273150 20967 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.277405 21216 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:52.296033 21069 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:52.296638 21184 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e30c07cef4334b45ade426b26069b26c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:52.298290 21561 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:52.298817 21143 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:52.300688 21561 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:52.330618 21835 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.336020 21069 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.342784 21139 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.364724 21069 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:52.373278 20967 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:52.375828 21035 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 169e564bd720422bb798c652af4fb2eb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:52.376680 21216 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:52.378450 21144 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:52.381147 21626 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:52.382648 21729 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:52.403659 21668 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.411965 21144 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.443471 20967 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:52.467406 21212 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:52.587251 21860 ts_tablet_manager.cc:933] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:52.628611 21860 tablet_copy_client.cc:323] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:05:52.673880 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet d693d94d4c0943d9ab08114c33fc2e7f from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:05:52.674466 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet d693d94d4c0943d9ab08114c33fc2e7f from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-d693d94d4c0943d9ab08114c33fc2e7f
I20260709 13:05:52.681090 21231 tablet_copy_source_session.cc:215] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:52.681701 21826 heartbeater.cc:499] Master 127.19.127.254:37743 was elected leader, sending a full tablet report...
I20260709 13:05:52.698452 21860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d693d94d4c0943d9ab08114c33fc2e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:52.715991 21860 tablet_copy_client.cc:806] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:52.716704 21860 tablet_copy_client.cc:670] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:52.722293 21860 tablet_copy_client.cc:538] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:52.734419 21860 tablet_bootstrap.cc:492] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:05:52.767136 21860 tablet_bootstrap.cc:492] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.768422 21860 tablet_bootstrap.cc:492] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:05:52.769133 21860 ts_tablet_manager.cc:1403] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.036s	user 0.023s	sys 0.008s
I20260709 13:05:52.772778 21860 raft_consensus.cc:359] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:52.773758 21860 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:05:52.774494 21860 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:52.793591 21860 ts_tablet_manager.cc:1434] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.005s
I20260709 13:05:52.811204 21865 ts_tablet_manager.cc:933] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:05:52.812796 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-d693d94d4c0943d9ab08114c33fc2e7f received from {username='slave'} at 127.0.0.1:54620
I20260709 13:05:52.813503 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-d693d94d4c0943d9ab08114c33fc2e7f on tablet d693d94d4c0943d9ab08114c33fc2e7f with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:52.840134 21865 tablet_copy_client.cc:323] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:05:52.880108 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet b4ee2463d8894a95be9b747f813f9106 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:05:52.880738 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet b4ee2463d8894a95be9b747f813f9106 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-b4ee2463d8894a95be9b747f813f9106
I20260709 13:05:52.888770 21082 tablet_copy_source_session.cc:215] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:52.892553 21865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ee2463d8894a95be9b747f813f9106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:52.917738 21860 ts_tablet_manager.cc:933] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:52.918200 21865 tablet_copy_client.cc:806] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:52.918781 21865 tablet_copy_client.cc:670] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:52.927520 21865 tablet_copy_client.cc:538] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:52.928083 21860 tablet_copy_client.cc:323] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:05:52.946413 21865 tablet_bootstrap.cc:492] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:05:52.952605 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet e30c07cef4334b45ade426b26069b26c from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:05:52.953271 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet e30c07cef4334b45ade426b26069b26c from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-e30c07cef4334b45ade426b26069b26c
I20260709 13:05:52.983392 21865 tablet_bootstrap.cc:492] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:52.984866 21865 tablet_bootstrap.cc:492] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:05:52.985633 21865 ts_tablet_manager.cc:1403] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.039s	user 0.030s	sys 0.000s
I20260709 13:05:52.992034 21865 raft_consensus.cc:359] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:52.993028 21865 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:05:52.993728 21865 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:53.001698 21865 ts_tablet_manager.cc:1434] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.004s
I20260709 13:05:53.011111 21231 tablet_copy_source_session.cc:215] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:53.015415 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b4ee2463d8894a95be9b747f813f9106 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:05:53.020056 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b4ee2463d8894a95be9b747f813f9106 on tablet b4ee2463d8894a95be9b747f813f9106 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:53.020001 21871 ts_tablet_manager.cc:933] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:05:53.035533 21801 raft_consensus.cc:1217] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:53.063390 21860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e30c07cef4334b45ade426b26069b26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:53.122257 21860 tablet_copy_client.cc:806] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:53.122939 21860 tablet_copy_client.cc:670] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:53.130067 21860 tablet_copy_client.cc:538] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:53.139871 21871 tablet_copy_client.cc:323] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:05:53.143175 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet 169e564bd720422bb798c652af4fb2eb from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:05:53.144007 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet 169e564bd720422bb798c652af4fb2eb from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-169e564bd720422bb798c652af4fb2eb
I20260709 13:05:53.148759 21860 tablet_bootstrap.cc:492] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:05:53.158634 21082 tablet_copy_source_session.cc:215] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:53.170491 21871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169e564bd720422bb798c652af4fb2eb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:53.175987 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51782.5us
I20260709 13:05:53.188674 21871 tablet_copy_client.cc:806] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:53.189419 21871 tablet_copy_client.cc:670] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:53.193017 21860 tablet_bootstrap.cc:492] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.202057 21860 tablet_bootstrap.cc:492] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:05:53.202839 21860 ts_tablet_manager.cc:1403] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.054s	user 0.019s	sys 0.005s
I20260709 13:05:53.193109 21871 tablet_copy_client.cc:538] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:53.215461 21871 tablet_bootstrap.cc:492] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:05:53.226161 21860 raft_consensus.cc:359] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:53.227481 21860 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:05:53.228274 21860 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:53.233901 21801 raft_consensus.cc:1217] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:53.239650 21860 ts_tablet_manager.cc:1434] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.001s
I20260709 13:05:53.243785 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-e30c07cef4334b45ade426b26069b26c received from {username='slave'} at 127.0.0.1:54620
I20260709 13:05:53.244251 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-e30c07cef4334b45ade426b26069b26c on tablet e30c07cef4334b45ade426b26069b26c with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:53.272111 21871 tablet_bootstrap.cc:492] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:53.273172 21871 tablet_bootstrap.cc:492] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:05:53.274041 21871 ts_tablet_manager.cc:1403] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.059s	user 0.022s	sys 0.010s
I20260709 13:05:53.278381 21871 raft_consensus.cc:359] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:53.279438 21871 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:05:53.280196 21871 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:53.289410 21871 ts_tablet_manager.cc:1434] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:05:53.298520 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-169e564bd720422bb798c652af4fb2eb received from {username='slave'} at 127.0.0.1:60302
I20260709 13:05:53.298988 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-169e564bd720422bb798c652af4fb2eb on tablet 169e564bd720422bb798c652af4fb2eb with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:53.305989 21801 raft_consensus.cc:1217] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:53.432538 21801 raft_consensus.cc:1217] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:53.615630 21858 raft_consensus.cc:1064] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:05:53.619783 21858 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:53.625490 21874 raft_consensus.cc:1064] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:05:53.629953 21874 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:53.694581 21220 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:53.696411 21144 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:53.699889 21801 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:53.701556 21748 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:53.708400 21668 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:53.709524 21144 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:53.710033 21668 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:53.710808 21872 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:53.713223 21800 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:53.714952 21719 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:53.739943 21069 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:53.748016 21719 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:53.784195 21835 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:05:53.787259 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 81796.7us
I20260709 13:05:53.798156 21139 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:53.806126 21800 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:53.824150 21719 raft_consensus.cc:1064] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:05:53.824357 21069 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:53.825789 20967 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:53.827145 21719 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:53.847657 21800 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:05:53.851536 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46386.5us
I20260709 13:05:53.849532 21858 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:05:53.875778 21894 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:53.900537 21220 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:53.905527 21141 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:53.907579 21872 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:53.924041 21800 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:53.925585 21719 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:53.933821 21139 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:53.935678 21886 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:53.940358 21835 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:53.942564 21799 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:53.949095 21216 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:53.950744 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d693d94d4c0943d9ab08114c33fc2e7f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:53.959576 21143 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:53.971983 21069 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:53.972034 21800 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:53.992148 20968 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:53.990425 20967 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:54.026248 21056 tablet_service.cc:1558] Processing DeleteTablet for tablet d693d94d4c0943d9ab08114c33fc2e7f with delete_type TABLET_DATA_TOMBSTONED (TS 9384358ce43f4e3fac2ae44bdf11c389 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20260709 13:05:54.049862 21892 raft_consensus.cc:1064] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:05:54.050645 21067 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:54.053076 21892 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:54.065490 21800 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:54.049768 21835 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:54.067116 21880 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:54.074200 21748 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.079577 21800 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.096853 21835 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.107556 21143 raft_consensus.cc:2955] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.105156 21139 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.128439 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b4ee2463d8894a95be9b747f813f9106 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:54.148032 20967 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.157931 21142 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:54.145035 21069 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:05:54.185070 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 81884.2us
I20260709 13:05:54.184087 21800 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.188902 21874 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:05:54.192716 21902 tablet_replica.cc:333] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:05:54.201097 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet b4ee2463d8894a95be9b747f813f9106 with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:05:54.213176 21799 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:54.215058 21895 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:54.233670 21908 tablet_replica.cc:333] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:05:54.254715 20967 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.259929 21221 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:54.290594 21902 raft_consensus.cc:2243] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:54.291510 21902 raft_consensus.cc:2272] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:54.296712 21908 raft_consensus.cc:2243] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:54.297559 21908 raft_consensus.cc:2272] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:54.309837 21892 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:54.316471 21146 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:54.339766 21894 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:54.341933 21902 ts_tablet_manager.cc:1916] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:54.343052 21147 raft_consensus.cc:2955] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.346786 21908 ts_tablet_manager.cc:1916] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:54.392773 21902 ts_tablet_manager.cc:1929] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:54.393392 21902 log.cc:1199] T d693d94d4c0943d9ab08114c33fc2e7f P 9384358ce43f4e3fac2ae44bdf11c389: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/wals/d693d94d4c0943d9ab08114c33fc2e7f
I20260709 13:05:54.395619 20957 catalog_manager.cc:5028] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): tablet d693d94d4c0943d9ab08114c33fc2e7f (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:05:54.412328 21893 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.430155 21221 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.450937 21799 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.462013 20968 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.469659 21140 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.495923 21908 ts_tablet_manager.cc:1929] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:54.496402 21908 log.cc:1199] T b4ee2463d8894a95be9b747f813f9106 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/b4ee2463d8894a95be9b747f813f9106
I20260709 13:05:54.502803 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet b4ee2463d8894a95be9b747f813f9106 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:05:54.503594 21071 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:54.517056 21799 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:54.519961 21748 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:05:54.524497 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50637.6us
I20260709 13:05:54.528843 21875 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.543954 21799 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.568830 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 169e564bd720422bb798c652af4fb2eb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:54.586859 21221 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:54.595182 21800 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:54.597659 21899 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:54.600451 21071 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:54.601892 21835 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:54.610474 20968 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.610615 21901 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.612685 21070 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.620085 21221 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:54.625815 21799 raft_consensus.cc:2955] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.632498 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet 169e564bd720422bb798c652af4fb2eb with delete_type TABLET_DATA_TOMBSTONED (TS 5e5e65640c8b4f5ca7eadc6a14312c55 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49704
I20260709 13:05:54.641953 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e30c07cef4334b45ade426b26069b26c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:54.643705 21912 tablet_replica.cc:333] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:05:54.644740 21912 raft_consensus.cc:2243] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:54.645460 21912 raft_consensus.cc:2272] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:54.647732 20968 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:54.649787 21912 ts_tablet_manager.cc:1916] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:54.668084 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet e30c07cef4334b45ade426b26069b26c with delete_type TABLET_DATA_TOMBSTONED (TS 5e5e65640c8b4f5ca7eadc6a14312c55 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49704
I20260709 13:05:54.689888 21912 ts_tablet_manager.cc:1929] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:54.690407 21912 log.cc:1199] T 169e564bd720422bb798c652af4fb2eb P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/wals/169e564bd720422bb798c652af4fb2eb
I20260709 13:05:54.700827 21912 tablet_replica.cc:333] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:05:54.701879 21912 raft_consensus.cc:2243] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:54.702615 21912 raft_consensus.cc:2272] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:54.707154 21912 ts_tablet_manager.cc:1916] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:54.745801 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet 169e564bd720422bb798c652af4fb2eb (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:05:54.802062 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet 169e564bd720422bb798c652af4fb2eb with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:49704
I20260709 13:05:54.806532 21912 ts_tablet_manager.cc:1929] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:54.807438 21912 log.cc:1199] T e30c07cef4334b45ade426b26069b26c P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/wals/e30c07cef4334b45ade426b26069b26c
I20260709 13:05:54.804695 21895 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:05:54.818161 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet e30c07cef4334b45ade426b26069b26c (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:05:54.820056 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet 169e564bd720422bb798c652af4fb2eb (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:05:54.853533 21221 raft_consensus.cc:2955] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:55.758587 21221 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:55.767827 21142 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:55.770037 21913 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:55.770470 21071 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:55.772475 21184 consensus_peers.cc:597] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fbcef88f59d4f9ea0f01023c7265069. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.775041 21920 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:55.790970 21899 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:55.793079 21071 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:55.797509 21140 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:55.811609 21070 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:55.825667 20968 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:55.840054 21147 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:55.840271 21035 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d4a635811aa248b599c9361b7abf9f32. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.841611 21221 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:55.843128 21668 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:55.848869 21894 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:55.867441 21894 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:55.880340 21140 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:55.897751 21216 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:55.903244 21071 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:05:55.918661 21035 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1fa2ded676fc4d048c810a5f842100d1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.922925 21212 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:55.923662 21140 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:55.926190 21895 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:55.929219 21928 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:55.964730 21895 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:55.973722 21142 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:55.971065 20968 catalog_manager.cc:5697] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:55.986653 21216 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:55.997264 21212 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:56.006032 20968 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.035596 21142 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:56.036239 21071 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:56.036365 21184 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ffee3ff6d914b9a8bf36e54cc2da28d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:56.037147 21913 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:56.038103 21920 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:56.072008 21913 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:56.074832 21147 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:56.076067 21071 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:56.092181 20967 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:56.202471 21939 ts_tablet_manager.cc:933] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:56.205091 21939 tablet_copy_client.cc:323] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:05:56.207203 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 2fbcef88f59d4f9ea0f01023c7265069 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:05:56.207844 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 2fbcef88f59d4f9ea0f01023c7265069 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-2fbcef88f59d4f9ea0f01023c7265069
I20260709 13:05:56.236480 21231 tablet_copy_source_session.cc:215] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:56.239616 21939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fbcef88f59d4f9ea0f01023c7265069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.264827 21939 tablet_copy_client.cc:806] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:56.265508 21939 tablet_copy_client.cc:670] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:56.274629 21939 tablet_copy_client.cc:538] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:56.288467 21939 tablet_bootstrap.cc:492] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:05:56.291803 21941 ts_tablet_manager.cc:933] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:05:56.294402 21941 tablet_copy_client.cc:323] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:05:56.298833 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet d4a635811aa248b599c9361b7abf9f32 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:05:56.306012 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet d4a635811aa248b599c9361b7abf9f32 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-d4a635811aa248b599c9361b7abf9f32
I20260709 13:05:56.336071 21082 tablet_copy_source_session.cc:215] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:56.370618 21941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4a635811aa248b599c9361b7abf9f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.396451 21939 tablet_bootstrap.cc:492] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:56.397600 21939 tablet_bootstrap.cc:492] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:05:56.398372 21939 ts_tablet_manager.cc:1403] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.110s	user 0.023s	sys 0.007s
I20260709 13:05:56.404678 21941 tablet_copy_client.cc:806] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:56.405367 21941 tablet_copy_client.cc:670] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:56.405912 21939 raft_consensus.cc:359] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:56.406980 21939 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:05:56.407881 21939 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:56.413044 21939 ts_tablet_manager.cc:1434] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.006s
I20260709 13:05:56.464694 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-2fbcef88f59d4f9ea0f01023c7265069 received from {username='slave'} at 127.0.0.1:54620
I20260709 13:05:56.465255 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-2fbcef88f59d4f9ea0f01023c7265069 on tablet 2fbcef88f59d4f9ea0f01023c7265069 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:56.475992 21939 ts_tablet_manager.cc:933] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:05:56.478353 21939 tablet_copy_client.cc:323] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:05:56.480926 21081 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet 1fa2ded676fc4d048c810a5f842100d1 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:05:56.481627 21081 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet 1fa2ded676fc4d048c810a5f842100d1 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-1fa2ded676fc4d048c810a5f842100d1
I20260709 13:05:56.490504 21081 tablet_copy_source_session.cc:215] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:56.498389 21939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa2ded676fc4d048c810a5f842100d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.517562 21941 tablet_copy_client.cc:538] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:56.580408 21950 ts_tablet_manager.cc:933] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:56.589787 21939 tablet_copy_client.cc:806] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:56.590543 21939 tablet_copy_client.cc:670] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:56.599313 21939 tablet_copy_client.cc:538] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:56.617775 21939 tablet_bootstrap.cc:492] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:05:56.637912 21950 tablet_copy_client.cc:323] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:05:56.676671 21941 tablet_bootstrap.cc:492] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:05:56.688190 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:05:56.688846 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 5ffee3ff6d914b9a8bf36e54cc2da28d from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-5ffee3ff6d914b9a8bf36e54cc2da28d
I20260709 13:05:56.704483 21231 tablet_copy_source_session.cc:215] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:05:56.707409 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53422.4us
I20260709 13:05:56.708889 21950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.738739 21941 tablet_bootstrap.cc:492] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:56.739867 21941 tablet_bootstrap.cc:492] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:05:56.740604 21941 ts_tablet_manager.cc:1403] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.064s	user 0.017s	sys 0.016s
I20260709 13:05:56.741135 21799 raft_consensus.cc:1217] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:56.744094 21941 raft_consensus.cc:359] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:56.744979 21941 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:05:56.748056 21941 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:56.750679 21941 ts_tablet_manager.cc:1434] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.005s
I20260709 13:05:56.754674 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-d4a635811aa248b599c9361b7abf9f32 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:05:56.755141 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-d4a635811aa248b599c9361b7abf9f32 on tablet d4a635811aa248b599c9361b7abf9f32 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:56.767912 21939 tablet_bootstrap.cc:492] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:56.768882 21939 tablet_bootstrap.cc:492] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:05:56.769586 21939 ts_tablet_manager.cc:1403] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.152s	user 0.022s	sys 0.002s
I20260709 13:05:56.773300 21950 tablet_copy_client.cc:806] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:56.774019 21950 tablet_copy_client.cc:670] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:56.775628 21939 raft_consensus.cc:359] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:56.776610 21939 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:05:56.777336 21939 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:56.780148 21939 ts_tablet_manager.cc:1434] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:05:56.781842 21950 tablet_copy_client.cc:538] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:56.783072 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-1fa2ded676fc4d048c810a5f842100d1 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:05:56.783528 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-1fa2ded676fc4d048c810a5f842100d1 on tablet 1fa2ded676fc4d048c810a5f842100d1 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:56.792387 21950 tablet_bootstrap.cc:492] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:05:56.813005 21950 tablet_bootstrap.cc:492] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:56.814185 21950 tablet_bootstrap.cc:492] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:05:56.814842 21950 ts_tablet_manager.cc:1403] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 13:05:56.818220 21950 raft_consensus.cc:359] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:56.819082 21950 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:05:56.819741 21950 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:56.825834 21950 ts_tablet_manager.cc:1434] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:05:56.828055 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-5ffee3ff6d914b9a8bf36e54cc2da28d received from {username='slave'} at 127.0.0.1:54620
I20260709 13:05:56.828512 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-5ffee3ff6d914b9a8bf36e54cc2da28d on tablet 5ffee3ff6d914b9a8bf36e54cc2da28d with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:56.991631 21799 raft_consensus.cc:1217] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:57.033082 21799 raft_consensus.cc:1217] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:57.265224 21799 raft_consensus.cc:1217] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:57.291627 21920 raft_consensus.cc:1064] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:05:57.298197 21920 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:57.310418 21799 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.312040 21147 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.312445 21913 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.313431 21920 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:57.315037 21069 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.318557 21913 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.329644 21913 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.336760 21147 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.351969 21799 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.354882 21069 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.371697 20968 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.415593 21216 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:57.437572 21800 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:57.438723 21142 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:57.439208 21913 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:57.441237 21945 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:57.446065 21945 raft_consensus.cc:1064] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:05:57.451429 21966 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.457144 21142 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.457170 21799 raft_consensus.cc:2955] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.448762 21945 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:57.475430 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2fbcef88f59d4f9ea0f01023c7265069 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:57.478493 20967 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.501200 21799 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.503952 21147 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.504061 21913 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.508140 21913 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.513669 21071 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.515106 21945 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:57.527262 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08002036a0 after lost signal to thread 19970
I20260709 13:05:57.529045 21056 tablet_service.cc:1558] Processing DeleteTablet for tablet 2fbcef88f59d4f9ea0f01023c7265069 with delete_type TABLET_DATA_TOMBSTONED (TS 9384358ce43f4e3fac2ae44bdf11c389 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
W20260709 13:05:57.530164 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08002759e0 after lost signal to thread 21004
W20260709 13:05:57.531090 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08001ddb20 after lost signal to thread 21096
W20260709 13:05:57.531895 21242 debug-util.cc:398] Leaking SignalData structure 0x7b080023ec00 after lost signal to thread 21171
W20260709 13:05:57.532656 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08001487a0 after lost signal to thread 21245
W20260709 13:05:57.533419 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08002d6cc0 after lost signal to thread 21825
W20260709 13:05:57.593187 21973 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:05:57.637555 21945 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:05:57.650024 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48984us
I20260709 13:05:57.657737 21147 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.671900 21975 tablet_replica.cc:333] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:05:57.677146 21975 raft_consensus.cc:2243] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:57.681468 21975 raft_consensus.cc:2272] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:57.681514 20967 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.686458 21978 raft_consensus.cc:1064] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:05:57.689788 21978 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:05:57.692330 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 91554.2us
W20260709 13:05:57.693017 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 82571.7us
I20260709 13:05:57.707718 21975 ts_tablet_manager.cc:1916] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:57.769474 21147 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.771076 21799 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.772990 21956 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:57.773756 21975 ts_tablet_manager.cc:1929] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:57.775357 21975 log.cc:1199] T 2fbcef88f59d4f9ea0f01023c7265069 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/wals/2fbcef88f59d4f9ea0f01023c7265069
I20260709 13:05:57.778400 20957 catalog_manager.cc:5028] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): tablet 2fbcef88f59d4f9ea0f01023c7265069 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:05:57.778910 21218 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.780246 21067 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.786228 21978 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:57.805855 21985 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:57.821863 21978 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.824858 21799 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.826113 21221 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.840654 21147 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.845875 20970 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.883060 21067 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:57.888418 21981 raft_consensus.cc:1064] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:05:57.894349 21981 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:57.895507 21799 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:57.899459 21978 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:57.903311 21140 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:57.909884 21985 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.916482 21983 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:57.919337 21801 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.926426 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1fa2ded676fc4d048c810a5f842100d1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:57.933466 21800 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.935333 21981 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.937525 21147 raft_consensus.cc:2955] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:57.943655 21218 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.947427 21142 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:57.951890 21956 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.953339 20969 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.961642 21956 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.970989 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fa2ded676fc4d048c810a5f842100d1 with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:05:58.000896 21985 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.015311 21993 tablet_replica.cc:333] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:05:58.017297 21147 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.021658 21218 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.031529 21993 raft_consensus.cc:2243] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.009848 21800 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.035735 20969 catalog_manager.cc:5697] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:58.040768 21801 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.049309 21993 raft_consensus.cc:2272] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.053130 21993 ts_tablet_manager.cc:1916] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:58.062724 21218 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:58.072079 21069 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:05:58.087565 21801 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:58.089160 21920 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:58.092455 21801 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:58.093976 21069 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:58.097564 21147 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:58.097612 21920 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:58.094897 21993 ts_tablet_manager.cc:1929] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:58.095536 21981 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:58.103659 21993 log.cc:1199] T 1fa2ded676fc4d048c810a5f842100d1 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/1fa2ded676fc4d048c810a5f842100d1
I20260709 13:05:58.105753 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet 1fa2ded676fc4d048c810a5f842100d1 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:05:58.107566 21982 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.115558 21801 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.119840 21978 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.123835 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:58.124058 21069 raft_consensus.cc:2955] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.125761 21800 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.139480 21985 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:58.153463 20968 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:58.157483 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d4a635811aa248b599c9361b7abf9f32 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:58.166349 20967 catalog_manager.cc:5697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.169307 21147 raft_consensus.cc:2955] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:05:58.178423 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ffee3ff6d914b9a8bf36e54cc2da28d with delete_type TABLET_DATA_TOMBSTONED (TS 5e5e65640c8b4f5ca7eadc6a14312c55 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49704
I20260709 13:05:58.201293 21995 tablet_replica.cc:333] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:05:58.204824 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet d4a635811aa248b599c9361b7abf9f32 with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:05:58.205967 21993 tablet_replica.cc:333] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:05:58.207060 21993 raft_consensus.cc:2243] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.207762 21993 raft_consensus.cc:2272] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.211498 21993 ts_tablet_manager.cc:1916] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:58.221622 21995 raft_consensus.cc:2243] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.222607 21995 raft_consensus.cc:2272] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.227421 21995 ts_tablet_manager.cc:1916] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:58.246212 21993 ts_tablet_manager.cc:1929] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:58.246824 21993 log.cc:1199] T d4a635811aa248b599c9361b7abf9f32 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/d4a635811aa248b599c9361b7abf9f32
I20260709 13:05:58.248529 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet d4a635811aa248b599c9361b7abf9f32 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:05:58.251765 21995 ts_tablet_manager.cc:1929] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:58.252321 21995 log.cc:1199] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/wals/5ffee3ff6d914b9a8bf36e54cc2da28d
I20260709 13:05:58.254209 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet 5ffee3ff6d914b9a8bf36e54cc2da28d (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:05:59.294291 21221 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:59.306418 21140 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:59.308009 21982 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:59.309640 21184 consensus_peers.cc:597] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d51215a492444539c4905a13bd0c12d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:59.311035 21067 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:59.312280 21920 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.315948 21920 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.322990 21147 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.333748 21067 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.345667 20967 catalog_manager.cc:5697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:59.384406 21140 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:59.396867 21221 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:59.398236 21067 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:59.400309 21996 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.406424 21996 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.464179 21067 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.461982 22008 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.480888 21218 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:05:59.484867 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 48339us
I20260709 13:05:59.524168 20968 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.556804 21067 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:05:59.569882 21035 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9028fa09ccd24f5494e4e5de5a8e6e18. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:59.572722 21140 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:59.574342 21110 consensus_peers.cc:597] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b59f0de1289c4c98ae7ce302ca584a30. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:59.573704 21218 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.577111 22015 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.579002 21981 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.595448 21978 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.598073 21216 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.598861 21140 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.611757 21218 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:59.618381 20968 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 2 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:59.629977 21140 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:59.631628 21920 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.638738 21067 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:59.644541 22014 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
W20260709 13:05:59.653656 21184 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d158af381004432b3f3f21e53fd19ac. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:59.658792 21920 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.662130 21147 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.686519 20967 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.689705 21067 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:05:59.794373 22022 ts_tablet_manager.cc:933] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:05:59.796685 22022 tablet_copy_client.cc:323] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.194:39351
I20260709 13:05:59.816371 21157 tablet_copy_service.cc:140] P 5e5e65640c8b4f5ca7eadc6a14312c55: Received BeginTabletCopySession request for tablet b59f0de1289c4c98ae7ce302ca584a30 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:50996)
I20260709 13:05:59.817075 21157 tablet_copy_service.cc:161] P 5e5e65640c8b4f5ca7eadc6a14312c55: Beginning new tablet copy session on tablet b59f0de1289c4c98ae7ce302ca584a30 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:50996: session id = b70fc04de37e4239a8336534f0c9c0fb-b59f0de1289c4c98ae7ce302ca584a30
I20260709 13:05:59.826329 21157 tablet_copy_source_session.cc:215] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:59.830457 22022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59f0de1289c4c98ae7ce302ca584a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:59.851300 22022 tablet_copy_client.cc:806] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:59.852023 22022 tablet_copy_client.cc:670] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:59.856211 22022 tablet_copy_client.cc:538] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:59.865584 22022 tablet_bootstrap.cc:492] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:05:59.888626 22022 tablet_bootstrap.cc:492] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:59.889427 22022 tablet_bootstrap.cc:492] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:05:59.890334 22022 ts_tablet_manager.cc:1403] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.025s	user 0.023s	sys 0.000s
I20260709 13:05:59.893257 22022 raft_consensus.cc:359] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:59.894490 22022 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:05:59.895155 22022 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:05:59.923920 22022 ts_tablet_manager.cc:1434] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.033s	user 0.021s	sys 0.005s
I20260709 13:05:59.925782 21157 tablet_copy_service.cc:342] P 5e5e65640c8b4f5ca7eadc6a14312c55: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b59f0de1289c4c98ae7ce302ca584a30 received from {username='slave'} at 127.0.0.1:50996
I20260709 13:05:59.926316 21157 tablet_copy_service.cc:434] P 5e5e65640c8b4f5ca7eadc6a14312c55: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b59f0de1289c4c98ae7ce302ca584a30 on tablet b59f0de1289c4c98ae7ce302ca584a30 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:05:59.933679 22022 ts_tablet_manager.cc:933] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:05:59.943630 22022 tablet_copy_client.cc:323] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:05:59.948994 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 3d51215a492444539c4905a13bd0c12d from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:05:59.949699 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 3d51215a492444539c4905a13bd0c12d from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-3d51215a492444539c4905a13bd0c12d
I20260709 13:05:59.956741 21231 tablet_copy_source_session.cc:215] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:59.959653 22022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d51215a492444539c4905a13bd0c12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:59.976475 22022 tablet_copy_client.cc:806] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:59.977129 22022 tablet_copy_client.cc:670] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:59.981884 22022 tablet_copy_client.cc:538] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:59.991641 22022 tablet_bootstrap.cc:492] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:00.015255 22022 tablet_bootstrap.cc:492] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:00.016654 22022 tablet_bootstrap.cc:492] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:00.017371 22022 ts_tablet_manager.cc:1403] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.004s
I20260709 13:06:00.020114 22022 raft_consensus.cc:359] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:00.020957 22022 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:00.021730 22022 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:00.026526 22022 ts_tablet_manager.cc:1434] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.005s
I20260709 13:06:00.028643 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-3d51215a492444539c4905a13bd0c12d received from {username='slave'} at 127.0.0.1:54620
I20260709 13:06:00.029053 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-3d51215a492444539c4905a13bd0c12d on tablet 3d51215a492444539c4905a13bd0c12d with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:00.107878 22022 ts_tablet_manager.cc:933] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:00.110221 22022 tablet_copy_client.cc:323] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:00.113049 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet 9028fa09ccd24f5494e4e5de5a8e6e18 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:00.113768 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet 9028fa09ccd24f5494e4e5de5a8e6e18 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-9028fa09ccd24f5494e4e5de5a8e6e18
I20260709 13:06:00.119271 21082 tablet_copy_source_session.cc:215] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:00.123487 22022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9028fa09ccd24f5494e4e5de5a8e6e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:00.143807 22028 ts_tablet_manager.cc:933] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:00.145816 22028 tablet_copy_client.cc:323] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:06:00.147662 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 3d158af381004432b3f3f21e53fd19ac from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:06:00.148241 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 3d158af381004432b3f3f21e53fd19ac from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-3d158af381004432b3f3f21e53fd19ac
I20260709 13:06:00.150744 22022 tablet_copy_client.cc:806] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:00.151342 22022 tablet_copy_client.cc:670] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:00.158190 22022 tablet_copy_client.cc:538] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:00.160277 21231 tablet_copy_source_session.cc:215] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:00.163666 22028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d158af381004432b3f3f21e53fd19ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:00.168167 22022 tablet_bootstrap.cc:492] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:00.191318 22028 tablet_copy_client.cc:806] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:00.191890 22028 tablet_copy_client.cc:670] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:00.201776 22022 tablet_bootstrap.cc:492] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:00.201977 22028 tablet_copy_client.cc:538] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:00.203137 22022 tablet_bootstrap.cc:492] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:00.203903 22022 ts_tablet_manager.cc:1403] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.005s
I20260709 13:06:00.206408 22022 raft_consensus.cc:359] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:00.207015 22022 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:00.207544 22022 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:00.209690 22022 ts_tablet_manager.cc:1434] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:00.211867 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-9028fa09ccd24f5494e4e5de5a8e6e18 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:00.212317 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-9028fa09ccd24f5494e4e5de5a8e6e18 on tablet 9028fa09ccd24f5494e4e5de5a8e6e18 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:00.214296 22028 tablet_bootstrap.cc:492] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:00.235770 22028 tablet_bootstrap.cc:492] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:00.236804 22028 tablet_bootstrap.cc:492] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:00.237794 22028 ts_tablet_manager.cc:1403] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.008s
I20260709 13:06:00.240867 22028 raft_consensus.cc:359] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:00.241894 22028 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:00.242575 22028 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:00.246219 22028 ts_tablet_manager.cc:1434] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:06:00.248977 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-3d158af381004432b3f3f21e53fd19ac received from {username='slave'} at 127.0.0.1:54620
I20260709 13:06:00.249495 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-3d158af381004432b3f3f21e53fd19ac on tablet 3d158af381004432b3f3f21e53fd19ac with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:00.276340 21801 raft_consensus.cc:1217] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:00.328975 21801 raft_consensus.cc:1217] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:00.597126 21801 raft_consensus.cc:1217] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:00.681627 21801 raft_consensus.cc:1217] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:06:00.880975 22038 raft_consensus.cc:1064] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:00.883343 22038 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:00.890580 21801 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:00.892284 22039 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:00.890663 21216 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:00.897827 22039 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:00.903151 21069 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:00.904803 22038 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:00.914433 22039 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:00.917181 21997 raft_consensus.cc:1064] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:00.920109 21997 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:00.925496 21218 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:00.962044 21801 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:00.965039 21071 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:00.968283 22014 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:00.967948 21801 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:00.973302 20967 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:00.962044 21140 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:00.976120 21997 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:00.985642 22014 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:01.001065 21069 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.031985 21997 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.037753 21799 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.045318 21140 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.056042 21069 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.073212 21147 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:01.079852 21799 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:01.080571 21216 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:01.082051 22035 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:01.084442 22035 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:01.083917 20968 catalog_manager.cc:5697] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.092903 22039 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.107918 21216 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.108382 21799 raft_consensus.cc:2955] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.124226 21218 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:01.127509 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b59f0de1289c4c98ae7ce302ca584a30 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:01.133062 21140 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:01.134500 21801 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:01.136112 22014 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:01.140970 21997 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:01.142149 20970 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 1 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.150732 21920 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.158598 21145 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.153338 21801 raft_consensus.cc:2955] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.174196 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3d51215a492444539c4905a13bd0c12d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:01.198671 21056 tablet_service.cc:1558] Processing DeleteTablet for tablet b59f0de1289c4c98ae7ce302ca584a30 with delete_type TABLET_DATA_TOMBSTONED (TS 9384358ce43f4e3fac2ae44bdf11c389 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20260709 13:06:01.204165 20969 catalog_manager.cc:5697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:01.207105 21997 raft_consensus.cc:1064] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:01.215831 21997 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:01.221215 21055 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d51215a492444539c4905a13bd0c12d with delete_type TABLET_DATA_TOMBSTONED (TS 9384358ce43f4e3fac2ae44bdf11c389 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20260709 13:06:01.237295 22025 raft_consensus.cc:1064] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:01.240505 22052 tablet_replica.cc:333] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:06:01.241065 22025 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:01.242060 22052 raft_consensus.cc:2243] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.244514 22052 raft_consensus.cc:2272] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.245537 21145 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:01.246805 21801 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:01.248373 22014 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:01.250921 22014 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:06:01.259848 21069 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:01.263561 22014 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:01.272428 22052 ts_tablet_manager.cc:1916] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:01.287642 21801 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:01.290763 21983 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:01.310376 21216 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:01.311761 22052 ts_tablet_manager.cc:1929] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:01.312335 22052 log.cc:1199] T b59f0de1289c4c98ae7ce302ca584a30 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/wals/b59f0de1289c4c98ae7ce302ca584a30
I20260709 13:06:01.314471 22052 tablet_replica.cc:333] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:06:01.314621 20957 catalog_manager.cc:5028] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): tablet b59f0de1289c4c98ae7ce302ca584a30 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:01.315552 22052 raft_consensus.cc:2243] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.316327 22052 raft_consensus.cc:2272] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.320574 21140 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:01.317889 21982 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.321854 22052 ts_tablet_manager.cc:1916] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:01.321712 21069 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.322734 21983 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:01.326272 21981 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:01.333215 21145 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.360808 21801 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.367120 20970 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:01.374811 22053 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.391332 21138 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.399199 21218 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.410148 21221 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:01.410635 21799 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.416004 22052 ts_tablet_manager.cc:1929] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:01.419517 22052 log.cc:1199] T 3d51215a492444539c4905a13bd0c12d P 9384358ce43f4e3fac2ae44bdf11c389: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/wals/3d51215a492444539c4905a13bd0c12d
I20260709 13:06:01.421212 20957 catalog_manager.cc:5028] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): tablet 3d51215a492444539c4905a13bd0c12d (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:01.425410 20969 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:01.452045 21067 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:01.450582 21069 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:01.454216 22056 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:01.455809 21801 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:01.460001 21145 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:06:01.461900 22063 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:01.463428 22014 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:01.472426 22014 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.478237 21801 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:06:01.481251 22053 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:01.484969 21067 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.495749 21799 raft_consensus.cc:2955] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.503069 22025 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.508998 21801 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.515533 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9028fa09ccd24f5494e4e5de5a8e6e18 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:01.532239 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3d158af381004432b3f3f21e53fd19ac with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:01.535741 20970 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.543203 21138 raft_consensus.cc:2955] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.560992 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d158af381004432b3f3f21e53fd19ac with delete_type TABLET_DATA_TOMBSTONED (TS 5e5e65640c8b4f5ca7eadc6a14312c55 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49704
I20260709 13:06:01.573663 22066 tablet_replica.cc:333] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:06:01.568441 20970 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 2 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:01.581881 22066 raft_consensus.cc:2243] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.583004 22066 raft_consensus.cc:2272] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.587034 22066 ts_tablet_manager.cc:1916] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:01.591012 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet 9028fa09ccd24f5494e4e5de5a8e6e18 with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:06:01.599584 22067 tablet_replica.cc:333] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:06:01.600900 22067 raft_consensus.cc:2243] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:01.601773 22067 raft_consensus.cc:2272] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:01.606544 22067 ts_tablet_manager.cc:1916] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:01.612798 22066 ts_tablet_manager.cc:1929] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:01.613438 22066 log.cc:1199] T 3d158af381004432b3f3f21e53fd19ac P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/wals/3d158af381004432b3f3f21e53fd19ac
I20260709 13:06:01.615899 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet 3d158af381004432b3f3f21e53fd19ac (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:01.624431 22067 ts_tablet_manager.cc:1929] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:06:01.624898 22067 log.cc:1199] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/9028fa09ccd24f5494e4e5de5a8e6e18
I20260709 13:06:01.626268 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet 9028fa09ccd24f5494e4e5de5a8e6e18 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:02.673748 21069 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:02.681586 21140 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:02.681982 21035 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 223e82a164f445c2b46667d58a5d2d1f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.682735 21221 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:02.683509 22064 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:02.687394 22025 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:02.696920 21140 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.702221 21221 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.694890 22064 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.723191 21069 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:02.723992 20970 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.731878 21140 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:02.732173 21221 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:02.732249 21035 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e8c9b46a490540a2bb0a90f7bfbd1e53. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.733723 22064 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:02.734812 22068 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:02.746924 22064 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.755927 21140 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.763070 21221 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.766106 21218 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:02.784026 20969 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.798708 21069 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:02.801401 21184 consensus_peers.cc:597] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53c918605cdf4c39b832a695968308a3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.802974 22069 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:02.806375 21140 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:02.807968 21982 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:02.817286 22069 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.819111 21138 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.822062 21069 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.831692 21221 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:02.836416 20970 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:06:02.844805 21184 consensus_peers.cc:597] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce55364f10c348819e7c63c08633b184. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:02.846347 21138 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:02.848480 22069 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:02.860574 21069 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:02.860574 22014 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.862671 21982 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:02.862270 21140 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.894115 20968 catalog_manager.cc:5697] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 2 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:02.900722 21069 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:03.262136 22089 ts_tablet_manager.cc:933] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:03.264120 22089 tablet_copy_client.cc:323] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:06:03.265760 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 53c918605cdf4c39b832a695968308a3 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:06:03.266322 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 53c918605cdf4c39b832a695968308a3 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-53c918605cdf4c39b832a695968308a3
I20260709 13:06:03.269827 22091 ts_tablet_manager.cc:933] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:03.275274 21231 tablet_copy_source_session.cc:215] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:03.278321 22091 tablet_copy_client.cc:323] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:06:03.280108 22089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c918605cdf4c39b832a695968308a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.280953 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet ce55364f10c348819e7c63c08633b184 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:06:03.281831 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet ce55364f10c348819e7c63c08633b184 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-ce55364f10c348819e7c63c08633b184
I20260709 13:06:03.289855 21231 tablet_copy_source_session.cc:215] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:03.295783 22091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce55364f10c348819e7c63c08633b184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.305102 22089 tablet_copy_client.cc:806] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:03.305907 22089 tablet_copy_client.cc:670] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:03.310899 22089 tablet_copy_client.cc:538] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:03.334391 22089 tablet_bootstrap.cc:492] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:03.337493 22093 ts_tablet_manager.cc:933] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:03.345903 22094 ts_tablet_manager.cc:933] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:03.355741 22093 tablet_copy_client.cc:323] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:03.357895 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet 223e82a164f445c2b46667d58a5d2d1f from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:03.358713 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet 223e82a164f445c2b46667d58a5d2d1f from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-223e82a164f445c2b46667d58a5d2d1f
I20260709 13:06:03.369307 22091 tablet_copy_client.cc:806] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:03.377956 22091 tablet_copy_client.cc:670] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:03.382561 22089 tablet_bootstrap.cc:492] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:03.383781 22089 tablet_bootstrap.cc:492] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:03.384508 22089 ts_tablet_manager.cc:1403] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.014s
I20260709 13:06:03.389101 22089 raft_consensus.cc:359] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:03.390081 22089 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:03.390795 22089 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:03.402026 22089 ts_tablet_manager.cc:1434] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.004s
I20260709 13:06:03.417480 22091 tablet_copy_client.cc:538] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:03.423664 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-53c918605cdf4c39b832a695968308a3 received from {username='slave'} at 127.0.0.1:54620
I20260709 13:06:03.424183 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-53c918605cdf4c39b832a695968308a3 on tablet 53c918605cdf4c39b832a695968308a3 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:03.432675 22091 tablet_bootstrap.cc:492] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:03.439702 21082 tablet_copy_source_session.cc:215] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:03.463193 22091 tablet_bootstrap.cc:492] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:03.471477 22093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223e82a164f445c2b46667d58a5d2d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.486598 22091 tablet_bootstrap.cc:492] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:03.496318 22091 ts_tablet_manager.cc:1403] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.064s	user 0.023s	sys 0.007s
I20260709 13:06:03.499744 22091 raft_consensus.cc:359] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:03.500931 22091 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:03.501694 22091 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:03.524425 22094 tablet_copy_client.cc:323] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:03.526294 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet e8c9b46a490540a2bb0a90f7bfbd1e53 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:03.526870 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet e8c9b46a490540a2bb0a90f7bfbd1e53 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-e8c9b46a490540a2bb0a90f7bfbd1e53
I20260709 13:06:03.531198 22091 ts_tablet_manager.cc:1434] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.005s
I20260709 13:06:03.535835 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-ce55364f10c348819e7c63c08633b184 received from {username='slave'} at 127.0.0.1:54620
I20260709 13:06:03.536363 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-ce55364f10c348819e7c63c08633b184 on tablet ce55364f10c348819e7c63c08633b184 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:03.537493 21082 tablet_copy_source_session.cc:215] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:03.541841 22094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8c9b46a490540a2bb0a90f7bfbd1e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.544754 22093 tablet_copy_client.cc:806] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:03.545476 22093 tablet_copy_client.cc:670] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:03.549446 22093 tablet_copy_client.cc:538] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:03.557523 22093 tablet_bootstrap.cc:492] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:03.561196 22094 tablet_copy_client.cc:806] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:03.561970 22094 tablet_copy_client.cc:670] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:03.566987 22094 tablet_copy_client.cc:538] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:03.579346 22094 tablet_bootstrap.cc:492] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:03.592100 22093 tablet_bootstrap.cc:492] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:03.593101 22093 tablet_bootstrap.cc:492] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:03.593851 22093 ts_tablet_manager.cc:1403] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.020s
I20260709 13:06:03.597524 22093 raft_consensus.cc:359] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:03.598528 22093 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:03.599309 22093 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:03.603055 22093 ts_tablet_manager.cc:1434] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:06:03.605891 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-223e82a164f445c2b46667d58a5d2d1f received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:03.606326 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-223e82a164f445c2b46667d58a5d2d1f on tablet 223e82a164f445c2b46667d58a5d2d1f with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:03.614010 22094 tablet_bootstrap.cc:492] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:03.615757 22094 tablet_bootstrap.cc:492] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:03.618024 22094 ts_tablet_manager.cc:1403] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.006s
I20260709 13:06:03.621033 22094 raft_consensus.cc:359] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:03.621953 22094 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:03.622699 22094 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:03.625090 22094 ts_tablet_manager.cc:1434] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:03.631999 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-e8c9b46a490540a2bb0a90f7bfbd1e53 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:03.632481 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-e8c9b46a490540a2bb0a90f7bfbd1e53 on tablet e8c9b46a490540a2bb0a90f7bfbd1e53 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:03.702445 21801 raft_consensus.cc:1217] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:03.719420 21801 raft_consensus.cc:1217] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:03.812203 21801 raft_consensus.cc:1217] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:06:03.816599 21801 raft_consensus.cc:1217] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:04.119143 21982 raft_consensus.cc:1064] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:04.121690 21982 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:04.128222 21801 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:04.128222 21145 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:04.130095 22069 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:04.132546 22069 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:04.135300 21067 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:04.137909 22014 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:04.172111 21982 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.175169 21067 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.179152 21145 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.195303 21801 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.204180 20969 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.240702 21218 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:04.245173 22025 raft_consensus.cc:1064] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:04.249645 21801 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:04.251650 21145 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:04.251924 22069 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:04.247550 22025 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:04.254731 22069 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:04.262550 22106 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.268571 21218 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:04.269704 21145 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.271118 21799 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:04.274237 22068 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:04.275980 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53c918605cdf4c39b832a695968308a3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:04.280999 22068 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:04.285938 21801 raft_consensus.cc:2955] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.301205 20967 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.328685 21138 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:04.332199 22104 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.340794 21056 tablet_service.cc:1558] Processing DeleteTablet for tablet 53c918605cdf4c39b832a695968308a3 with delete_type TABLET_DATA_TOMBSTONED (TS 9384358ce43f4e3fac2ae44bdf11c389 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20260709 13:06:04.348722 22121 tablet_replica.cc:333] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:06:04.351432 22121 raft_consensus.cc:2243] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:04.352321 22121 raft_consensus.cc:2272] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:04.351940 21799 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.356950 22121 ts_tablet_manager.cc:1916] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:04.370692 21221 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.374190 20967 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.380638 22121 ts_tablet_manager.cc:1929] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:04.381276 22121 log.cc:1199] T 53c918605cdf4c39b832a695968308a3 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/wals/53c918605cdf4c39b832a695968308a3
I20260709 13:06:04.383050 20957 catalog_manager.cc:5028] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): tablet 53c918605cdf4c39b832a695968308a3 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:04.384289 22068 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:04.404832 21138 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.427237 22118 raft_consensus.cc:1064] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:04.432176 22118 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:04.434891 22106 raft_consensus.cc:1064] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:04.437610 22106 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:04.452492 21140 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:04.453913 21067 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:04.456988 22014 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:04.460944 22014 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:04.462137 21799 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:04.464406 22069 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:04.493832 21801 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:04.497756 21145 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:04.498725 22119 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:04.500923 22118 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:04.502627 21218 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:04.509831 22119 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:04.528136 21067 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.524899 22069 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.543440 21799 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.550084 22104 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.567098 20967 catalog_manager.cc:5697] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.565551 21138 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.575402 21140 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.582772 21799 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.601521 21218 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:04.606781 20967 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.617551 21799 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:06:04.619719 22014 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:06:04.627489 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50232.3us
I20260709 13:06:04.639518 21138 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:06:04.646610 22069 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:04.661705 21799 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.664484 22014 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.664901 21138 raft_consensus.cc:2955] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.716146 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ce55364f10c348819e7c63c08633b184 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:04.716790 20967 catalog_manager.cc:5697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 2 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.724648 21067 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:04.725860 21220 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.730034 21799 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:04.734583 22068 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:04.751643 22119 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.760846 21056 tablet_service.cc:1558] Processing DeleteTablet for tablet ce55364f10c348819e7c63c08633b184 with delete_type TABLET_DATA_TOMBSTONED (TS 9384358ce43f4e3fac2ae44bdf11c389 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20260709 13:06:04.762300 22121 tablet_replica.cc:333] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:06:04.763366 22121 raft_consensus.cc:2243] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:04.762842 21801 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.764160 22121 raft_consensus.cc:2272] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:04.766031 21221 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:04.771261 22121 ts_tablet_manager.cc:1916] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:04.776850 22025 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:04.782732 20967 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.797731 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet e8c9b46a490540a2bb0a90f7bfbd1e53 with delete_type TABLET_DATA_TOMBSTONED (TS 5e5e65640c8b4f5ca7eadc6a14312c55 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49704
I20260709 13:06:04.800098 21220 raft_consensus.cc:2955] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:04.803786 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e8c9b46a490540a2bb0a90f7bfbd1e53 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:04.808395 22121 ts_tablet_manager.cc:1929] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:06:04.809113 22121 log.cc:1199] T ce55364f10c348819e7c63c08633b184 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/wals/ce55364f10c348819e7c63c08633b184
I20260709 13:06:04.810920 20957 catalog_manager.cc:5028] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): tablet ce55364f10c348819e7c63c08633b184 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:04.828522 22131 tablet_replica.cc:333] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:06:04.829981 22131 raft_consensus.cc:2243] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:04.830886 22131 raft_consensus.cc:2272] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:04.834437 22131 ts_tablet_manager.cc:1916] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:04.952857 22131 ts_tablet_manager.cc:1929] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:04.954164 22131 log.cc:1199] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/wals/e8c9b46a490540a2bb0a90f7bfbd1e53
I20260709 13:06:04.957044 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet e8c9b46a490540a2bb0a90f7bfbd1e53 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:04.979022 21071 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:04.992164 21801 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:04.994334 22025 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:05.011883 21145 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:05.014238 22104 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:05.017628 21799 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:05.046020 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 223e82a164f445c2b46667d58a5d2d1f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:05.047732 20967 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:05.053551 22120 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:05.069067 21138 raft_consensus.cc:2955] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:05.104735 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet 223e82a164f445c2b46667d58a5d2d1f with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:06:05.117005 22134 tablet_replica.cc:333] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:06:05.118541 22134 raft_consensus.cc:2243] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.119798 22134 raft_consensus.cc:2272] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.124046 22134 ts_tablet_manager.cc:1916] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:05.140727 22134 ts_tablet_manager.cc:1929] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:05.141211 22134 log.cc:1199] T 223e82a164f445c2b46667d58a5d2d1f P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/223e82a164f445c2b46667d58a5d2d1f
I20260709 13:06:05.142971 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet 223e82a164f445c2b46667d58a5d2d1f (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:06.189231 21138 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:06:06.196659 21110 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0631ac443bb4e6dbbbc599dd4b67d2e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.197140 21221 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:06.198341 21067 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:06.198537 22138 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.199718 22135 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.208060 22138 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.215453 21221 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.212137 21067 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.228543 21071 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:06.234061 20968 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:06:06.248334 21035 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fdb8a7df9f774ce0afbd5b26fa2a88d2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.253798 21221 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:06.254432 21138 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:06.258052 22133 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.260146 22118 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:06.271667 21145 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.272969 22133 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.278414 21220 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.298211 21071 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:06.308197 20967 catalog_manager.cc:5697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:06:06.317224 21035 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0de4eb0c8e649f09be36671e6dcc4cb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.318778 21220 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:06.320029 21145 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:06.320341 22133 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:06.321432 22118 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.339242 22118 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.340458 21145 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.357945 21221 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.359354 20968 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.360719 21220 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:06.381347 21145 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:06.382066 21067 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:06.383611 22106 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:06.385869 22069 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:06:06.388118 21184 consensus_peers.cc:597] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12e3a58d24d64816bce2752c6adb7e33. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.400115 22136 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.403983 21067 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.420519 20970 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:06.429937 21138 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:06.634822 22158 ts_tablet_manager.cc:933] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:06.637259 22158 tablet_copy_client.cc:323] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:06.639575 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:06.640196 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet fdb8a7df9f774ce0afbd5b26fa2a88d2 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-fdb8a7df9f774ce0afbd5b26fa2a88d2
I20260709 13:06:06.648232 21082 tablet_copy_source_session.cc:215] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:06.652205 22158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:06.672724 22160 ts_tablet_manager.cc:933] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:06.676784 22158 tablet_copy_client.cc:806] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:06.677523 22158 tablet_copy_client.cc:670] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:06.685853 22158 tablet_copy_client.cc:538] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:06.696069 22158 tablet_bootstrap.cc:492] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:06.700153 22160 tablet_copy_client.cc:323] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.194:39351
I20260709 13:06:06.702497 21157 tablet_copy_service.cc:140] P 5e5e65640c8b4f5ca7eadc6a14312c55: Received BeginTabletCopySession request for tablet e0631ac443bb4e6dbbbc599dd4b67d2e from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:50996)
I20260709 13:06:06.703130 21157 tablet_copy_service.cc:161] P 5e5e65640c8b4f5ca7eadc6a14312c55: Beginning new tablet copy session on tablet e0631ac443bb4e6dbbbc599dd4b67d2e from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:50996: session id = b70fc04de37e4239a8336534f0c9c0fb-e0631ac443bb4e6dbbbc599dd4b67d2e
I20260709 13:06:06.712555 21157 tablet_copy_source_session.cc:215] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:06.718091 22160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0631ac443bb4e6dbbbc599dd4b67d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:06.731896 22158 tablet_bootstrap.cc:492] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:06.733014 22158 tablet_bootstrap.cc:492] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:06.733812 22158 ts_tablet_manager.cc:1403] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.038s	user 0.034s	sys 0.000s
I20260709 13:06:06.737680 22160 tablet_copy_client.cc:806] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:06.738370 22160 tablet_copy_client.cc:670] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:06.737964 22158 raft_consensus.cc:359] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:06.739090 22158 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:06.739990 22158 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:06.743609 22158 ts_tablet_manager.cc:1434] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 13:06:06.749187 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-fdb8a7df9f774ce0afbd5b26fa2a88d2 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:06.749725 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-fdb8a7df9f774ce0afbd5b26fa2a88d2 on tablet fdb8a7df9f774ce0afbd5b26fa2a88d2 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:06.751026 22160 tablet_copy_client.cc:538] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:06.773937 22160 tablet_bootstrap.cc:492] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:06.795888 22160 tablet_bootstrap.cc:492] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:06.796934 22160 tablet_bootstrap.cc:492] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:06.797684 22160 ts_tablet_manager.cc:1403] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.005s
I20260709 13:06:06.800836 22160 raft_consensus.cc:359] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:06.801757 22160 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:06.802425 22160 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:06.808120 22160 ts_tablet_manager.cc:1434] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:06:06.809849 21157 tablet_copy_service.cc:342] P 5e5e65640c8b4f5ca7eadc6a14312c55: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-e0631ac443bb4e6dbbbc599dd4b67d2e received from {username='slave'} at 127.0.0.1:50996
I20260709 13:06:06.810351 21157 tablet_copy_service.cc:434] P 5e5e65640c8b4f5ca7eadc6a14312c55: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-e0631ac443bb4e6dbbbc599dd4b67d2e on tablet e0631ac443bb4e6dbbbc599dd4b67d2e with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:06.911921 22160 ts_tablet_manager.cc:933] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:06.913606 22160 tablet_copy_client.cc:323] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:06.915490 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet a0de4eb0c8e649f09be36671e6dcc4cb from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:06.916049 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet a0de4eb0c8e649f09be36671e6dcc4cb from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-a0de4eb0c8e649f09be36671e6dcc4cb
I20260709 13:06:06.923713 21082 tablet_copy_source_session.cc:215] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:06.926740 22160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0de4eb0c8e649f09be36671e6dcc4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:06.933772 22158 ts_tablet_manager.cc:933] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:06.941869 22158 tablet_copy_client.cc:323] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:06:06.944465 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 12e3a58d24d64816bce2752c6adb7e33 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:06:06.945293 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 12e3a58d24d64816bce2752c6adb7e33 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-12e3a58d24d64816bce2752c6adb7e33
I20260709 13:06:06.948079 22160 tablet_copy_client.cc:806] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:06.948792 22160 tablet_copy_client.cc:670] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:06.964926 21231 tablet_copy_source_session.cc:215] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:06.971537 22160 tablet_copy_client.cc:538] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:06.975230 22158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12e3a58d24d64816bce2752c6adb7e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:06.981523 22160 tablet_bootstrap.cc:492] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:07.007256 22158 tablet_copy_client.cc:806] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:07.007884 22158 tablet_copy_client.cc:670] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:07.010246 22160 tablet_bootstrap.cc:492] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:07.011261 22160 tablet_bootstrap.cc:492] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:07.012032 22160 ts_tablet_manager.cc:1403] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.031s	user 0.025s	sys 0.006s
I20260709 13:06:07.015264 22158 tablet_copy_client.cc:538] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:07.015403 22160 raft_consensus.cc:359] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:07.016433 22160 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:07.017645 22160 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:07.029242 22160 ts_tablet_manager.cc:1434] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.001s
I20260709 13:06:07.036796 22158 tablet_bootstrap.cc:492] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:07.042459 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-a0de4eb0c8e649f09be36671e6dcc4cb received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:07.042990 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-a0de4eb0c8e649f09be36671e6dcc4cb on tablet a0de4eb0c8e649f09be36671e6dcc4cb with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:07.069157 22158 tablet_bootstrap.cc:492] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:07.070257 22158 tablet_bootstrap.cc:492] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:07.070989 22158 ts_tablet_manager.cc:1403] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.034s	user 0.022s	sys 0.004s
I20260709 13:06:07.074476 22158 raft_consensus.cc:359] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:07.075448 22158 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:07.076146 22158 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:07.079236 22158 ts_tablet_manager.cc:1434] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:06:07.081175 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-12e3a58d24d64816bce2752c6adb7e33 received from {username='slave'} at 127.0.0.1:54620
I20260709 13:06:07.081656 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-12e3a58d24d64816bce2752c6adb7e33 on tablet 12e3a58d24d64816bce2752c6adb7e33 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:07.141089 21801 raft_consensus.cc:1217] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:07.199826 21801 raft_consensus.cc:1217] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:07.462729 21801 raft_consensus.cc:1217] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:07.535764 21801 raft_consensus.cc:1217] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:07.769821 22170 raft_consensus.cc:1064] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:07.770876 22153 raft_consensus.cc:1064] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:07.773599 22170 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:07.775207 22153 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:07.785835 21801 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:07.786679 21067 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:07.787526 22172 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:07.788434 22173 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:07.789471 21220 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:07.794862 22172 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:07.807045 21799 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:07.811488 22118 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:07.820235 21216 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:07.823947 22152 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:07.824285 21147 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:07.826274 22118 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:07.837082 22170 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.842149 21071 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.839484 21801 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.857225 21220 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.850064 21216 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.851765 22118 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.879581 20968 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:07.892203 20967 catalog_manager.cc:5697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.924602 21801 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.924479 21138 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.934235 22177 raft_consensus.cc:1064] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:07.936854 22177 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:07.947419 21801 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:07.949880 22153 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:07.954317 21220 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:07.951934 21138 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:07.959664 22180 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:07.963188 22180 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:06:07.971344 22106 raft_consensus.cc:1064] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:07.973888 22106 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:07.986757 22177 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.991509 21799 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:07.997066 21145 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:08.001801 21147 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.005069 21220 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.011983 21801 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:08.013753 22171 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:08.021569 21138 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:08.024986 21071 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:08.025154 22136 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.035063 20969 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.041096 22171 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:08.045864 21801 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:08.046579 21071 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:08.050941 22172 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.052860 22172 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.063599 22170 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.068181 21800 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.069175 21067 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:08.069646 22171 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.081187 21069 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:08.088792 21799 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:08.090426 22152 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.108042 20969 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 1 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.109017 21800 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:08.111079 22180 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.116088 21220 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:08.119932 21221 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:08.121519 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet e0631ac443bb4e6dbbbc599dd4b67d2e with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:06:08.135931 22152 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.141068 22200 tablet_replica.cc:333] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:06:08.149184 22200 raft_consensus.cc:2243] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.150627 22200 raft_consensus.cc:2272] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.140795 21799 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.143543 22118 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.145231 22153 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:08.159147 22200 ts_tablet_manager.cc:1916] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:08.164505 21145 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.175341 21069 raft_consensus.cc:2955] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.182463 22153 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.189102 20969 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.195935 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a0de4eb0c8e649f09be36671e6dcc4cb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:08.186159 21801 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.211361 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e0631ac443bb4e6dbbbc599dd4b67d2e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:08.232928 20967 catalog_manager.cc:5697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:06:08.239843 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 94436.1us
I20260709 13:06:08.246474 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2 with delete_type TABLET_DATA_TOMBSTONED (TS 5e5e65640c8b4f5ca7eadc6a14312c55 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49704
I20260709 13:06:08.261847 22200 ts_tablet_manager.cc:1929] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:08.261695 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fdb8a7df9f774ce0afbd5b26fa2a88d2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:08.268718 22200 log.cc:1199] T e0631ac443bb4e6dbbbc599dd4b67d2e P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/e0631ac443bb4e6dbbbc599dd4b67d2e
I20260709 13:06:08.272882 21221 raft_consensus.cc:2955] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.271586 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet e0631ac443bb4e6dbbbc599dd4b67d2e (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:08.287443 20967 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:08.280671 21801 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.299245 22202 tablet_replica.cc:333] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:06:08.314061 22202 raft_consensus.cc:2243] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.315310 22202 raft_consensus.cc:2272] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.316481 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet a0de4eb0c8e649f09be36671e6dcc4cb with delete_type TABLET_DATA_TOMBSTONED (TS 5e5e65640c8b4f5ca7eadc6a14312c55 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49704
I20260709 13:06:08.321197 22202 ts_tablet_manager.cc:1916] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:08.334358 21069 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.345021 22202 ts_tablet_manager.cc:1929] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:08.346920 22202 log.cc:1199] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/wals/fdb8a7df9f774ce0afbd5b26fa2a88d2
I20260709 13:06:08.348686 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet fdb8a7df9f774ce0afbd5b26fa2a88d2 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:08.351233 21216 raft_consensus.cc:2955] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.348805 22202 tablet_replica.cc:333] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:06:08.353557 22202 raft_consensus.cc:2243] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.354535 22202 raft_consensus.cc:2272] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.361744 22202 ts_tablet_manager.cc:1916] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:08.370551 21216 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:08.390398 22202 ts_tablet_manager.cc:1929] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:08.390980 22202 log.cc:1199] T a0de4eb0c8e649f09be36671e6dcc4cb P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/wals/a0de4eb0c8e649f09be36671e6dcc4cb
I20260709 13:06:08.392804 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet a0de4eb0c8e649f09be36671e6dcc4cb (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:08.398283 21801 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:08.400867 22106 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.408507 21145 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:08.408947 22203 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.413327 22171 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.419005 21801 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.419188 21145 raft_consensus.cc:2955] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.432008 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 12e3a58d24d64816bce2752c6adb7e33 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:08.436775 20967 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:08.447609 21056 tablet_service.cc:1558] Processing DeleteTablet for tablet 12e3a58d24d64816bce2752c6adb7e33 with delete_type TABLET_DATA_TOMBSTONED (TS 9384358ce43f4e3fac2ae44bdf11c389 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20260709 13:06:08.451067 22206 tablet_replica.cc:333] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:06:08.453218 22206 raft_consensus.cc:2243] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.454946 22206 raft_consensus.cc:2272] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.458261 22206 ts_tablet_manager.cc:1916] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:08.470404 22206 ts_tablet_manager.cc:1929] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:08.470729 22206 log.cc:1199] T 12e3a58d24d64816bce2752c6adb7e33 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/wals/12e3a58d24d64816bce2752c6adb7e33
I20260709 13:06:08.472563 20957 catalog_manager.cc:5028] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): tablet 12e3a58d24d64816bce2752c6adb7e33 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:09.517947 21071 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:09.525738 21216 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:09.526419 21035 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c80e14318ee44feba007795498031464. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:09.526468 21139 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:09.527892 22201 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:09.528751 22204 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:09.537344 22177 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.538846 21216 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.539738 21139 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.551906 21221 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:06:09.562711 21184 consensus_peers.cc:597] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 293026cede06406bad9e7b5fc490abe2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:09.563563 21069 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:09.564468 21139 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:09.557664 20968 catalog_manager.cc:5697] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.565037 22199 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:09.570276 22199 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:09.592334 22199 raft_consensus.cc:2955] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.593539 21069 raft_consensus.cc:2955] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.596328 21139 raft_consensus.cc:2955] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.650089 21069 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:09.657068 20969 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.667644 21221 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:09.668823 21035 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2875dc050a244109a0287cc6dafa8431. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:09.670552 21139 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:09.670563 22218 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:06:09.672971 22217 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:09.697441 22217 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.699306 21139 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.699527 21221 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.712536 21069 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:09.721666 20967 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:06:09.727047 21035 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9833b15c99c34c269ca3afa7dac3f175. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:09.727928 21139 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:09.729529 21221 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:09.729493 22217 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:09.731220 22204 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:09.749423 22218 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.751456 21221 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.754014 21139 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:09.770989 20969 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.954360 22225 ts_tablet_manager.cc:933] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:09.956629 22225 tablet_copy_client.cc:323] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:09.958585 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet c80e14318ee44feba007795498031464 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:09.959137 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet c80e14318ee44feba007795498031464 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-c80e14318ee44feba007795498031464
I20260709 13:06:09.967420 21082 tablet_copy_source_session.cc:215] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:09.970988 22225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c80e14318ee44feba007795498031464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:09.990906 22225 tablet_copy_client.cc:806] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:09.991670 22225 tablet_copy_client.cc:670] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:09.993283 22227 ts_tablet_manager.cc:933] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:09.998523 22225 tablet_copy_client.cc:538] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:09.998590 22227 tablet_copy_client.cc:323] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:06:10.003648 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 293026cede06406bad9e7b5fc490abe2 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:06:10.004396 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 293026cede06406bad9e7b5fc490abe2 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-293026cede06406bad9e7b5fc490abe2
I20260709 13:06:10.008886 22225 tablet_bootstrap.cc:492] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:10.009910 21231 tablet_copy_source_session.cc:215] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:10.013070 22227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293026cede06406bad9e7b5fc490abe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:10.027777 22227 tablet_copy_client.cc:806] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:10.028405 22227 tablet_copy_client.cc:670] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:10.031613 22225 tablet_bootstrap.cc:492] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.033041 22225 tablet_bootstrap.cc:492] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:10.033730 22225 ts_tablet_manager.cc:1403] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.004s
I20260709 13:06:10.037747 22227 tablet_copy_client.cc:538] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:10.036934 22225 raft_consensus.cc:359] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:10.038676 22225 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:10.039364 22225 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:10.042629 22225 ts_tablet_manager.cc:1434] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:10.044881 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-c80e14318ee44feba007795498031464 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:10.045300 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-c80e14318ee44feba007795498031464 on tablet c80e14318ee44feba007795498031464 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:10.049177 22227 tablet_bootstrap.cc:492] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:10.057036 22225 ts_tablet_manager.cc:933] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:10.060779 22225 tablet_copy_client.cc:323] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:10.063079 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet 2875dc050a244109a0287cc6dafa8431 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:10.063690 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet 2875dc050a244109a0287cc6dafa8431 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-2875dc050a244109a0287cc6dafa8431
I20260709 13:06:10.072022 21082 tablet_copy_source_session.cc:215] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:10.075858 22225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2875dc050a244109a0287cc6dafa8431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:10.090610 22227 tablet_bootstrap.cc:492] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.091665 22227 tablet_bootstrap.cc:492] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:10.092375 22227 ts_tablet_manager.cc:1403] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.017s
I20260709 13:06:10.096560 22227 raft_consensus.cc:359] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:10.097369 22227 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:10.098040 22227 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:10.100622 22227 ts_tablet_manager.cc:1434] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:06:10.103840 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-293026cede06406bad9e7b5fc490abe2 received from {username='slave'} at 127.0.0.1:54620
I20260709 13:06:10.104269 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-293026cede06406bad9e7b5fc490abe2 on tablet 293026cede06406bad9e7b5fc490abe2 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:10.107123 22225 tablet_copy_client.cc:806] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:10.107774 22225 tablet_copy_client.cc:670] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:10.116537 22225 tablet_copy_client.cc:538] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:10.129940 22225 tablet_bootstrap.cc:492] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:10.156654 22225 tablet_bootstrap.cc:492] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.158430 22225 tablet_bootstrap.cc:492] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:10.159427 22225 ts_tablet_manager.cc:1403] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.030s	user 0.029s	sys 0.001s
I20260709 13:06:10.168481 22225 raft_consensus.cc:359] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:10.169697 22225 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:10.170425 22225 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:10.173363 22225 ts_tablet_manager.cc:1434] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:06:10.175096 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-2875dc050a244109a0287cc6dafa8431 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:10.175561 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-2875dc050a244109a0287cc6dafa8431 on tablet 2875dc050a244109a0287cc6dafa8431 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:10.237884 22225 ts_tablet_manager.cc:933] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:10.239564 22225 tablet_copy_client.cc:323] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:10.241151 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet 9833b15c99c34c269ca3afa7dac3f175 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:10.241691 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet 9833b15c99c34c269ca3afa7dac3f175 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-9833b15c99c34c269ca3afa7dac3f175
I20260709 13:06:10.252908 21082 tablet_copy_source_session.cc:215] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:10.256227 22225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9833b15c99c34c269ca3afa7dac3f175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:10.275745 22225 tablet_copy_client.cc:806] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:10.276292 22225 tablet_copy_client.cc:670] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:10.282176 22225 tablet_copy_client.cc:538] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:10.295328 22225 tablet_bootstrap.cc:492] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:10.315419 22225 tablet_bootstrap.cc:492] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:10.316380 22225 tablet_bootstrap.cc:492] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:10.317119 22225 ts_tablet_manager.cc:1403] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.000s
I20260709 13:06:10.320263 22225 raft_consensus.cc:359] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:10.321197 22225 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:10.321838 22225 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:10.324245 22225 ts_tablet_manager.cc:1434] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:10.326779 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-9833b15c99c34c269ca3afa7dac3f175 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:10.327196 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-9833b15c99c34c269ca3afa7dac3f175 on tablet 9833b15c99c34c269ca3afa7dac3f175 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:10.377899 21801 raft_consensus.cc:1217] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:10.485706 21801 raft_consensus.cc:1217] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:10.566398 21801 raft_consensus.cc:1217] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:10.726727 21799 raft_consensus.cc:1217] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:10.869894 22201 raft_consensus.cc:1064] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:10.872500 22201 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:10.882165 21799 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:10.883591 21216 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:10.884595 22204 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:10.885603 22218 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:10.891507 21145 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:10.893126 22217 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:10.911036 21145 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:10.920149 22204 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:10.925805 21221 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:10.940999 20967 catalog_manager.cc:5697] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:10.959257 21799 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:10.995010 21069 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:11.005756 21799 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:11.007421 21138 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:11.009227 22241 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:11.012177 22201 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.019961 22201 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.023736 21138 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.025480 21801 raft_consensus.cc:2955] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.033823 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c80e14318ee44feba007795498031464 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:11.041962 22106 raft_consensus.cc:1064] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:11.042049 20969 catalog_manager.cc:5697] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.044478 22106 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:11.054548 21799 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:11.056137 22236 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.057493 21145 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:11.061012 22236 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.064949 21069 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:11.067119 22199 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.083830 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet c80e14318ee44feba007795498031464 with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:06:11.087774 22171 raft_consensus.cc:2955] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.096431 21801 raft_consensus.cc:2955] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.105707 22250 tablet_replica.cc:333] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:06:11.106889 22204 raft_consensus.cc:1064] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:11.109309 22204 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:11.113238 22250 raft_consensus.cc:2243] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.119616 22250 raft_consensus.cc:2272] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.117213 20968 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.124289 21138 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:11.125903 21216 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:11.126137 22204 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.113415 21145 raft_consensus.cc:2955] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.122383 21800 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:11.127650 22218 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.134267 22250 ts_tablet_manager.cc:1916] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:11.135414 22204 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.152149 22250 ts_tablet_manager.cc:1929] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:11.152895 22250 log.cc:1199] T c80e14318ee44feba007795498031464 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/c80e14318ee44feba007795498031464
I20260709 13:06:11.162238 21221 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:11.163475 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet c80e14318ee44feba007795498031464 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:11.167514 22218 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.172756 21800 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:11.173907 21139 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:11.175067 21145 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.176656 22171 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.185878 22236 raft_consensus.cc:2955] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.189702 22241 raft_consensus.cc:1064] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:11.192628 22241 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:11.195149 22171 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.205170 21800 raft_consensus.cc:2955] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.216636 21139 raft_consensus.cc:2955] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.222393 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 293026cede06406bad9e7b5fc490abe2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:11.238035 20968 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.256091 21056 tablet_service.cc:1558] Processing DeleteTablet for tablet 293026cede06406bad9e7b5fc490abe2 with delete_type TABLET_DATA_TOMBSTONED (TS 9384358ce43f4e3fac2ae44bdf11c389 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20260709 13:06:11.258939 21800 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.260692 20969 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.302774 22259 tablet_replica.cc:333] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:06:11.305557 22259 raft_consensus.cc:2243] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.306356 22259 pending_rounds.cc:70] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:06:11.309736 21800 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:11.314002 21069 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:11.314010 21221 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:11.316293 22260 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:11.306766 22259 pending_rounds.cc:77] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305635311792644096 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "293026cede06406bad9e7b5fc490abe2" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } } }
I20260709 13:06:11.318006 22259 raft_consensus.cc:2917] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:06:11.318698 22259 raft_consensus.cc:2889] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:06:11.319305 22259 raft_consensus.cc:2272] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.323119 21139 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:11.327373 22259 ts_tablet_manager.cc:1916] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:11.329550 21221 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.332496 21801 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:11.334131 22217 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.341506 21139 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:11.341456 22217 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.349336 21801 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.352463 22201 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.358117 22204 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.359884 22204 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.365971 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9833b15c99c34c269ca3afa7dac3f175 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:11.379055 21138 raft_consensus.cc:2955] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.386255 20968 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.397895 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet 9833b15c99c34c269ca3afa7dac3f175 with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:06:11.398943 22250 tablet_replica.cc:333] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:06:11.399978 22250 raft_consensus.cc:2243] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.400770 22250 raft_consensus.cc:2272] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.405900 22250 ts_tablet_manager.cc:1916] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:11.429404 22259 ts_tablet_manager.cc:1929] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:11.430004 22259 log.cc:1199] T 293026cede06406bad9e7b5fc490abe2 P 9384358ce43f4e3fac2ae44bdf11c389: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/wals/293026cede06406bad9e7b5fc490abe2
I20260709 13:06:11.433480 20957 catalog_manager.cc:5028] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): tablet 293026cede06406bad9e7b5fc490abe2 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:11.437263 22204 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.457116 21216 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.470180 21138 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.478269 21801 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.483479 20969 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.502048 22250 ts_tablet_manager.cc:1929] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:11.502599 22250 log.cc:1199] T 9833b15c99c34c269ca3afa7dac3f175 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/9833b15c99c34c269ca3afa7dac3f175
I20260709 13:06:11.506256 21069 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:11.509001 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet 9833b15c99c34c269ca3afa7dac3f175 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:11.513543 21216 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:11.513904 21801 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:11.515368 22217 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.517378 22260 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.524922 22204 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.528113 21801 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.533116 21216 raft_consensus.cc:2955] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.535360 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2875dc050a244109a0287cc6dafa8431 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:11.543728 20967 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:11.554670 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet 2875dc050a244109a0287cc6dafa8431 with delete_type TABLET_DATA_TOMBSTONED (TS 5e5e65640c8b4f5ca7eadc6a14312c55 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49704
I20260709 13:06:11.558638 22265 tablet_replica.cc:333] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:06:11.559767 22265 raft_consensus.cc:2243] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.560986 22265 raft_consensus.cc:2272] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.565565 22265 ts_tablet_manager.cc:1916] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:11.579849 22265 ts_tablet_manager.cc:1929] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:11.580335 22265 log.cc:1199] T 2875dc050a244109a0287cc6dafa8431 P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/wals/2875dc050a244109a0287cc6dafa8431
I20260709 13:06:11.582998 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet 2875dc050a244109a0287cc6dafa8431 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:11.825376 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:11.890059 21221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:11.890664 21221 raft_consensus.cc:606] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:11.896509 21221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c918605cdf4c39b832a695968308a3"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:11.897104 21221 raft_consensus.cc:606] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:11.899789 21221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:11.900238 21221 raft_consensus.cc:606] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:11.911432 21069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:11.912022 21069 raft_consensus.cc:606] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:11.915140 21221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:11.915767 21221 raft_consensus.cc:606] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:11.918118 21221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:11.918633 21221 raft_consensus.cc:606] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:11.920120 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:06:11.922195 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:11.924867 22255 raft_consensus.cc:993] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:11.925448 22236 raft_consensus.cc:1081] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:11.926836 21138 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49724
I20260709 13:06:11.927407 21138 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.927690 21138 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.933880 21138 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:11.936184 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:11.936543 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:11.936924 21069 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.937026 21221 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:11.937378 21221 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:11.938133 21221 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:11.939121 21221 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.935926 21138 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:11.944914 21069 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:11.946215 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:11.946979 22170 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:11.947418 21221 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:11.948383 22170 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:11.949385 22170 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:11.960000 20969 catalog_manager.cc:5697] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:11.987694 22236 raft_consensus.cc:993] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:11.988294 22256 raft_consensus.cc:1081] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Signalling peer b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:11.991153 21801 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:54634
I20260709 13:06:11.991786 21801 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.992142 22256 raft_consensus.cc:993] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:11.992162 21801 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.993295 22256 raft_consensus.cc:1081] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:11.994141 22271 raft_consensus.cc:993] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:11.994691 22271 raft_consensus.cc:1081] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Signalling peer b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:11.994971 21138 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c918605cdf4c39b832a695968308a3"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49724
I20260709 13:06:11.995628 21138 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.995946 21138 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:11.996348 21800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:54634
I20260709 13:06:11.996742 21800 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.996938 21800 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.000286 21801 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:12.010715 21138 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:12.013559 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:12.013970 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:12.014205 21069 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.014557 21216 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:12.014626 21138 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:12.015622 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:12.016057 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:12.016189 21220 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:12.016602 21799 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.016571 21220 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:12.017532 21220 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:12.018617 21220 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.021533 21069 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:12.014904 21216 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:12.022565 21767 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:12.023845 22230 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:12.023748 21216 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:12.024931 21216 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.025892 21220 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:12.027163 22230 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:12.028295 21801 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:12.028185 22230 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:12.033079 21799 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:12.034278 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:12.035097 22170 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:12.035641 22170 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:12.022544 21800 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:12.036628 22170 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:12.040198 20969 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:12.041880 21800 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:12.047762 21216 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:12.049396 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:12.050055 21221 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:12.050388 21221 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:12.052687 21221 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:12.054096 21221 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.055370 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:12.055982 21143 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.062695 20969 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:12.063805 21221 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:12.065165 21143 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:12.065289 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:12.066201 22230 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:12.066699 22230 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:12.067646 22230 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:12.077606 20969 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:12.157102 22271 raft_consensus.cc:993] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:12.157620 22255 raft_consensus.cc:1081] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:12.159128 21143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49724
I20260709 13:06:12.159641 21143 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:12.160010 21143 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.165469 21143 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:12.167011 21143 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:12.167855 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:12.167982 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:12.168421 21069 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.168529 21216 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:12.168921 21216 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:12.169754 21216 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:12.170934 21216 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.175500 21069 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:12.176584 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:12.177409 21216 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:12.177418 22170 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:12.178082 22170 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:12.178937 22170 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:12.188733 20969 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:12.204937 22204 raft_consensus.cc:993] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:12.205372 22260 raft_consensus.cc:1081] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:12.206629 21138 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49718
I20260709 13:06:12.207083 21138 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:12.207422 21138 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.212314 21138 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:12.214413 21138 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:12.214689 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:12.214895 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:12.215440 21071 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:12.215586 21216 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.215761 21071 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:12.216497 21071 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:12.217595 21071 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:12.222658 21216 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:12.223933 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:12.224766 22170 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:12.225255 22170 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:12.224594 21071 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:12.225989 22170 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:12.235060 20969 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:12.399748 21216 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:12.400909 22170 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:12.416543 21071 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:12.419392 22275 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:12.484534 21138 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:12.485818 22230 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:12.496562 21221 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:12.500345 22230 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:12.565047 21221 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:12.566416 22287 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:12.576074 21071 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:12.577435 22273 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:12.582536 21800 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:12.584033 21221 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:12.584394 22275 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:12.587355 22275 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:12.602420 21071 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:12.620329 22275 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:06:12.696015 21221 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:12.699685 22269 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:12.714215 21216 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:12.717502 21221 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:12.725991 22269 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:12.730052 21139 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:12.730085 21071 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:12.735574 22296 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:12.737385 22255 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:12.741415 21184 consensus_peers.cc:597] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8a4c442e73243eda44d43ba0eaf0d2a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.755784 21139 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.755836 22255 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.762607 21069 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.775955 21071 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:12.779029 21070 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:12.781060 22170 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:12.791862 21035 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8d00e740dba47b2b43be9e4ef1ecb55. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.793244 21138 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:12.795710 22260 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:12.799324 21221 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:12.801527 22293 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:12.819522 20969 catalog_manager.cc:5697] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.834357 22260 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.836597 21221 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.853293 21069 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:12.853035 21138 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.862129 21143 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:12.862831 21035 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d78b35ac92d402db8a5e3fe45be52b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.864409 22260 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:12.867769 20968 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:12.871613 21221 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:12.874609 22293 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:12.883172 22260 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.888258 21138 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.904358 20967 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:12.893352 21221 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:12.909806 21069 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:12.923348 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:12.937177 21035 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc53d52fdeaa441b9e9d329e69d94440. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.942085 21143 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:12.944645 22294 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:12.957082 21221 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:12.972939 22293 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:12.979537 22217 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:13.013432 21143 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:13.016892 20968 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.020799 21221 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:13.041007 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:13.110329 21216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:13.111133 21216 raft_consensus.cc:606] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:13.114133 21069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:13.114719 21069 raft_consensus.cc:606] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:13.121774 21216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:13.122414 21216 raft_consensus.cc:606] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:13.125689 21069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:13.126282 21069 raft_consensus.cc:606] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:13.132102 21069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:13.132705 21069 raft_consensus.cc:606] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:13.134021 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:13.135917 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:13.197075 22260 raft_consensus.cc:993] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:13.197695 22260 raft_consensus.cc:1081] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:13.199553 21143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49718
I20260709 13:06:13.200129 21143 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.200560 21143 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.205004 21143 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:13.207257 21143 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:13.207477 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:13.208119 21071 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:13.208424 21071 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:13.207477 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:13.209719 21216 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.209169 21071 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:13.212313 21071 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.223165 21071 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:13.224443 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:13.225450 22269 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:13.225919 22269 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:13.226729 22269 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:13.228827 21216 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:13.257257 22320 ts_tablet_manager.cc:933] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:13.273542 20969 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.278736 22320 tablet_copy_client.cc:323] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:13.280565 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet 0d78b35ac92d402db8a5e3fe45be52b5 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:13.281160 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet 0d78b35ac92d402db8a5e3fe45be52b5 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-0d78b35ac92d402db8a5e3fe45be52b5
I20260709 13:06:13.287632 22316 raft_consensus.cc:993] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:13.288188 22316 raft_consensus.cc:1081] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Signalling peer b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:13.289095 21082 tablet_copy_source_session.cc:215] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:13.290243 21799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:54648
I20260709 13:06:13.290853 21799 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.291216 21799 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.293088 22320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d78b35ac92d402db8a5e3fe45be52b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.298250 21799 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:13.302220 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:13.302429 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:13.302867 21069 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:13.301440 21799 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:13.303210 21069 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:13.304087 21069 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:13.302876 21143 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.305220 21069 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.306380 22322 ts_tablet_manager.cc:933] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:13.312698 21143 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:13.314996 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:13.315994 22230 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:13.316481 22230 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:13.317371 22230 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:13.324700 22320 tablet_copy_client.cc:806] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:13.325424 22320 tablet_copy_client.cc:670] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:13.331332 22320 tablet_copy_client.cc:538] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:13.337488 22322 tablet_copy_client.cc:323] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:06:13.339387 21069 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:13.340122 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet d8a4c442e73243eda44d43ba0eaf0d2a from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:06:13.340672 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet d8a4c442e73243eda44d43ba0eaf0d2a from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-d8a4c442e73243eda44d43ba0eaf0d2a
I20260709 13:06:13.342082 22260 raft_consensus.cc:993] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:13.342326 22320 tablet_bootstrap.cc:492] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:13.342552 22260 raft_consensus.cc:1081] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:13.344435 21138 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49718
I20260709 13:06:13.344997 21138 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.345333 21138 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.347517 20969 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 2 to 3, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 3 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.354141 21231 tablet_copy_source_session.cc:215] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:13.358608 22325 ts_tablet_manager.cc:933] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:13.365182 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a4c442e73243eda44d43ba0eaf0d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.370409 22320 tablet_bootstrap.cc:492] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:13.371515 22320 tablet_bootstrap.cc:492] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:13.372319 22320 ts_tablet_manager.cc:1403] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.014s
I20260709 13:06:13.359943 21138 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:13.375727 22320 raft_consensus.cc:359] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:13.377641 22320 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:13.378446 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:13.378783 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:13.379021 21069 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:13.379135 21138 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:13.379307 21216 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.379446 21069 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:13.380230 21069 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:13.381260 21069 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.381943 22322 tablet_copy_client.cc:806] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:13.382488 22322 tablet_copy_client.cc:670] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:13.386613 22322 tablet_copy_client.cc:538] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:13.390249 21069 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:13.378427 22320 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:13.393817 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:13.394769 22269 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:13.395304 22269 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:13.396212 22269 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:13.396829 22320 ts_tablet_manager.cc:1434] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.024s	user 0.001s	sys 0.006s
I20260709 13:06:13.398803 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-0d78b35ac92d402db8a5e3fe45be52b5 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:13.399235 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-0d78b35ac92d402db8a5e3fe45be52b5 on tablet 0d78b35ac92d402db8a5e3fe45be52b5 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:13.409005 22296 raft_consensus.cc:993] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:13.414191 22296 raft_consensus.cc:1081] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Signalling peer b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:13.411885 22325 tablet_copy_client.cc:323] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:13.415841 21800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:54634
I20260709 13:06:13.416407 21800 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.412266 21216 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:13.416751 21800 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.419186 22322 tablet_bootstrap.cc:492] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:13.420868 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet b8d00e740dba47b2b43be9e4ef1ecb55 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:13.421983 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet b8d00e740dba47b2b43be9e4ef1ecb55 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-b8d00e740dba47b2b43be9e4ef1ecb55
I20260709 13:06:13.429929 21082 tablet_copy_source_session.cc:215] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:13.434191 22325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8d00e740dba47b2b43be9e4ef1ecb55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.434396 21800 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:13.436260 20969 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.444615 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:13.445226 21143 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.456151 21800 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:13.459271 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:13.459919 21216 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:13.460220 21216 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:13.460995 21216 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:13.462194 21216 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.470840 22325 tablet_copy_client.cc:806] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:13.471648 22325 tablet_copy_client.cc:670] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:13.477280 22325 tablet_copy_client.cc:538] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:13.478436 22322 tablet_bootstrap.cc:492] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:13.479429 22322 tablet_bootstrap.cc:492] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:13.480123 22322 ts_tablet_manager.cc:1403] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.061s	user 0.026s	sys 0.016s
I20260709 13:06:13.486210 22322 raft_consensus.cc:359] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:13.487720 22322 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:13.488449 22322 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:13.488998 22325 tablet_bootstrap.cc:492] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:13.489930 21216 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:13.491621 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:13.492533 22230 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:13.493099 22230 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:13.493970 22230 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:13.511152 21143 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:13.534965 22322 ts_tablet_manager.cc:1434] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 13:06:13.537405 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-d8a4c442e73243eda44d43ba0eaf0d2a received from {username='slave'} at 127.0.0.1:54620
I20260709 13:06:13.537868 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-d8a4c442e73243eda44d43ba0eaf0d2a on tablet d8a4c442e73243eda44d43ba0eaf0d2a with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:13.539374 22325 tablet_bootstrap.cc:492] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:13.540356 22325 tablet_bootstrap.cc:492] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:13.541049 22325 ts_tablet_manager.cc:1403] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.052s	user 0.022s	sys 0.009s
I20260709 13:06:13.540566 20969 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.545722 22322 ts_tablet_manager.cc:933] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:13.546527 22325 raft_consensus.cc:359] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:13.547370 22325 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:13.548075 22325 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:13.550686 22325 ts_tablet_manager.cc:1434] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:06:13.552392 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b8d00e740dba47b2b43be9e4ef1ecb55 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:13.552951 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b8d00e740dba47b2b43be9e4ef1ecb55 on tablet b8d00e740dba47b2b43be9e4ef1ecb55 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:13.557322 22322 tablet_copy_client.cc:323] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.193:42843
I20260709 13:06:13.560209 21082 tablet_copy_service.cc:140] P 9384358ce43f4e3fac2ae44bdf11c389: Received BeginTabletCopySession request for tablet cc53d52fdeaa441b9e9d329e69d94440 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:60302)
I20260709 13:06:13.560778 21082 tablet_copy_service.cc:161] P 9384358ce43f4e3fac2ae44bdf11c389: Beginning new tablet copy session on tablet cc53d52fdeaa441b9e9d329e69d94440 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:60302: session id = b70fc04de37e4239a8336534f0c9c0fb-cc53d52fdeaa441b9e9d329e69d94440
I20260709 13:06:13.567667 21082 tablet_copy_source_session.cc:215] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:13.570420 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc53d52fdeaa441b9e9d329e69d94440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.583225 22322 tablet_copy_client.cc:806] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:13.583851 22322 tablet_copy_client.cc:670] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:13.588807 22322 tablet_copy_client.cc:538] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:13.598477 22322 tablet_bootstrap.cc:492] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:13.605552 22296 raft_consensus.cc:993] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:13.606060 22295 raft_consensus.cc:1081] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:13.607545 21143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49724
I20260709 13:06:13.608073 21143 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.608400 21143 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.614173 21143 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:13.616849 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:13.617421 21069 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.620736 21143 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:13.620934 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:13.621462 21216 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:13.621678 21216 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:13.622406 21216 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:13.623132 21216 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.623396 22322 tablet_bootstrap.cc:492] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:13.624349 22322 tablet_bootstrap.cc:492] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:13.625007 22322 ts_tablet_manager.cc:1403] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.008s
I20260709 13:06:13.627981 22322 raft_consensus.cc:359] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:13.628877 21216 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:13.628896 22322 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:13.629055 21069 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:13.630335 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:13.630021 22322 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:13.631172 22269 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:13.631726 22269 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:13.632851 22322 ts_tablet_manager.cc:1434] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 13:06:13.632524 22269 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:13.640430 21082 tablet_copy_service.cc:342] P 9384358ce43f4e3fac2ae44bdf11c389: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-cc53d52fdeaa441b9e9d329e69d94440 received from {username='slave'} at 127.0.0.1:60302
I20260709 13:06:13.640946 21082 tablet_copy_service.cc:434] P 9384358ce43f4e3fac2ae44bdf11c389: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-cc53d52fdeaa441b9e9d329e69d94440 on tablet cc53d52fdeaa441b9e9d329e69d94440 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:13.666975 21216 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:13.668627 22269 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:13.674952 20969 catalog_manager.cc:5697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.691170 21800 raft_consensus.cc:1217] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:13.715498 21143 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:13.717909 21069 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:13.718058 21071 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:13.719313 22230 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:13.721697 22170 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:13.723624 22230 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:13.749470 21800 raft_consensus.cc:1217] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:13.855041 21216 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:13.856458 22170 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:13.859645 21800 raft_consensus.cc:1217] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:13.875385 21069 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:13.880241 22170 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:13.992970 21800 raft_consensus.cc:1217] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:14.007268 21216 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:14.009266 22273 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:14.040098 21138 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:14.043138 21216 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.044322 22318 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:14.045604 22269 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:14.063679 21069 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.069152 22170 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.136945 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:14.184580 22295 raft_consensus.cc:1064] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:14.188264 22295 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.190191 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:14.198290 21800 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.199847 22255 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.203649 21138 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.205219 22255 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.213596 21069 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.215368 22295 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.227581 22296 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.229676 21800 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.244917 22204 raft_consensus.cc:1064] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:14.246863 21069 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.250221 22204 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.257139 22260 raft_consensus.cc:1064] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:14.261377 20967 catalog_manager.cc:5697] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.268688 22260 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.273577 21138 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.304126 21070 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:14.305919 21220 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.312806 21800 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.307735 21143 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.315541 22217 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.324301 21070 raft_consensus.cc:606] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:14.333638 21801 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.335350 22204 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.338572 22293 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.341572 22217 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.345103 21220 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.345180 21143 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.368014 22260 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.372448 21070 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:14.374383 21070 raft_consensus.cc:606] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:14.369107 22217 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:06:14.387696 21070 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:14.388355 21070 raft_consensus.cc:606] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:14.389082 22204 raft_consensus.cc:1064] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:06:14.391661 22204 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.395874 21070 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:14.396486 21070 raft_consensus.cc:606] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:14.401835 21221 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.403527 22217 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.420143 21143 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:14.420317 21139 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.423743 22296 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:14.425351 22293 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.434929 22295 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.436513 21145 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.442272 21801 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:14.448060 22296 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:14.451709 21221 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.460462 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d8a4c442e73243eda44d43ba0eaf0d2a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:14.462795 20968 catalog_manager.cc:5697] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 1 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.480168 21216 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.482993 21070 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c20ca56847f64504a85694d10f47558e"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:14.483700 21070 raft_consensus.cc:606] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:14.487638 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:14.489339 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:14.493317 21798 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.494822 22260 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.477267 21801 raft_consensus.cc:2955] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.516003 21800 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.519331 21220 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.523144 21145 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.524351 22260 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.539640 20969 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.545771 21799 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.516026 22293 raft_consensus.cc:993] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:14.557974 21069 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.558928 22368 raft_consensus.cc:993] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:14.559444 22365 raft_consensus.cc:993] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:14.559794 21138 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:14.565737 22365 raft_consensus.cc:1081] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:14.565871 22368 raft_consensus.cc:1081] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:14.568207 21056 tablet_service.cc:1558] Processing DeleteTablet for tablet d8a4c442e73243eda44d43ba0eaf0d2a with delete_type TABLET_DATA_TOMBSTONED (TS 9384358ce43f4e3fac2ae44bdf11c389 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20260709 13:06:14.565276 22293 raft_consensus.cc:1081] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:14.570765 21797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:54648
I20260709 13:06:14.571359 21798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:54648
I20260709 13:06:14.572026 21798 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.572443 21798 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.571494 21797 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.573079 21797 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.574064 21800 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:14.576398 22293 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:14.584961 22260 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.596160 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8d00e740dba47b2b43be9e4ef1ecb55 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:14.594764 20970 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.596694 21800 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.600037 22260 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.629863 21797 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.632682 21797 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:14.633081 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:14.633968 21220 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.637159 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:14.637773 21070 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.638059 21070 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:14.638767 21070 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.639760 21070 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.641206 21220 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:14.642436 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:14.643271 22318 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.643731 22318 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:14.644470 22318 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.649956 20969 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.664798 21143 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:14.666788 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet b8d00e740dba47b2b43be9e4ef1ecb55 with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:06:14.676699 21216 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.684170 21221 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:14.685945 21143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49718
I20260709 13:06:14.686533 21143 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.686893 21143 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.689164 21798 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.695238 21143 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.697520 21798 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:14.697805 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:14.698436 21138 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.701354 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:14.702026 21069 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.702340 21069 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:14.703120 21069 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.704468 21069 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.711452 21138 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:14.712731 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:14.713550 22273 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.714033 22273 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:14.714854 22273 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.674857 21070 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:14.729739 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:14.730320 21067 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.730605 21067 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:14.731320 21067 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.732841 21069 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:14.732939 21067 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.748309 22372 tablet_replica.cc:333] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:06:14.739053 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:14.752434 21212 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.761773 21212 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:14.764392 22204 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.765342 21067 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:14.768285 22204 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:14.769781 22370 tablet_replica.cc:333] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:06:14.770812 22370 raft_consensus.cc:2243] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.771481 22370 raft_consensus.cc:2272] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.768955 20969 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.774499 20968 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.782261 21143 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:14.783272 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:14.783951 21798 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.802301 20969 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.807585 22370 ts_tablet_manager.cc:1916] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:14.812391 22372 raft_consensus.cc:2243] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.813303 22372 raft_consensus.cc:2272] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.817268 22372 ts_tablet_manager.cc:1916] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:14.818534 21113 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:14.819320 22275 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.819793 22275 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:14.820711 22275 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:14.823451 21797 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.838793 21145 raft_consensus.cc:2955] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:14.872963 22293 raft_consensus.cc:993] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:14.873286 22260 raft_consensus.cc:993] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:14.874600 22217 raft_consensus.cc:1081] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:14.875810 22217 raft_consensus.cc:1081] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:14.852415 21798 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
W20260709 13:06:14.927029 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53585.5us
I20260709 13:06:14.968438 22370 ts_tablet_manager.cc:1929] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:14.990229 22370 log.cc:1199] T d8a4c442e73243eda44d43ba0eaf0d2a P 9384358ce43f4e3fac2ae44bdf11c389: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/wals/d8a4c442e73243eda44d43ba0eaf0d2a
I20260709 13:06:15.011129 21143 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:15.017745 21145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49718
I20260709 13:06:15.019168 21139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c20ca56847f64504a85694d10f47558e"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49718
I20260709 13:06:15.033578 21145 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:15.033627 21139 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:15.034246 21139 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.045114 21139 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:15.045961 22372 ts_tablet_manager.cc:1929] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:15.046419 22372 log.cc:1199] T b8d00e740dba47b2b43be9e4ef1ecb55 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/b8d00e740dba47b2b43be9e4ef1ecb55
I20260709 13:06:15.047183 21139 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:15.053918 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:15.034153 21145 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.054549 21216 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.064569 21216 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:15.064687 21145 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:15.065771 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:15.066574 22275 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:15.067061 22275 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:15.068253 22275 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:15.081708 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:15.082572 21220 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.089990 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:15.090857 21067 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:15.091120 21067 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:15.091964 21067 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:15.092897 21220 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:15.093004 21067 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.099014 21145 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:15.102075 21067 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:15.111876 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:15.114002 22170 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:15.114472 22170 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:15.115322 22170 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:15.145757 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:15.146637 21070 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:15.146943 21070 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:15.147739 21070 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:15.148780 21070 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:15.160346 21219 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:15.162179 22273 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:06:16.249740 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.040s	sys 0.102s
W20260709 13:06:16.269559 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.040s	sys 0.103s
W20260709 13:06:16.264094 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.018s	sys 0.087s
W20260709 13:06:16.312755 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.018s	sys 0.088s
W20260709 13:06:16.352169 19972 debug-util.cc:398] Leaking SignalData structure 0x7b080000f740 after lost signal to thread 21185
W20260709 13:06:16.375337 21001 debug-util.cc:398] Leaking SignalData structure 0x7b080003b2a0 after lost signal to thread 19970
W20260709 13:06:16.388470 21070 consensus_meta.cc:287] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.239s	user 0.002s	sys 0.018s
I20260709 13:06:16.389257 21070 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:16.390602 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 248) took 1336 ms. Trace:
I20260709 13:06:16.391172 21070 rpcz_store.cc:276] 0709 13:06:15.054367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.145402 (+ 91035us) service_pool.cc:224] Handling call
0709 13:06:16.390435 (+1245033us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:16.353148 19972 kernel_stack_watchdog.cc:198] Thread 21185 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 227ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260709 13:06:16.420413 21038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.015s	sys 0.072s
W20260709 13:06:16.424316 21038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.016s	sys 0.073s
I20260709 13:06:16.427461 22378 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.427505 22395 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.430972 22378 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.451521 22394 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:16.456678 22389 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.452714 22394 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.457885 22389 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.436331 22395 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.503356 22379 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.557636 22393 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.504354 22379 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.562143 22397 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.578564 22392 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:16.654153 22374 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.655532 22394 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:16.577997 22395 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:16.564900 22378 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:16.562898 22393 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.666067 22378 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.668009 22378 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.665123 22392 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.655478 22374 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.680773 22389 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:16.681883 22374 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:16.682689 22397 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.658757 22394 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:16.692894 22397 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:16.693638 22397 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.695580 22394 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.698625 22394 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:16.702139 22379 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:16.702930 22379 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.703608 22379 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.706269 22379 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:16.709297 22392 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:16.665100 22395 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.710417 22395 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.713320 22395 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:16.713941 22395 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.714634 22395 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.717242 22395 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:16.717835 22395 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.718854 22391 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.719640 22391 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.762460 22393 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:16.767561 22378 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:16.780817 22391 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:16.821637 22401 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:16.822587 22401 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.825573 22401 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:16.826243 22401 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:16.827015 22401 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.829860 22401 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:16.830467 22401 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.694567 22397 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.842268 22397 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:16.831110 22401 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:16.872709 21185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.870s	user 0.033s	sys 0.057s
W20260709 13:06:16.873303 21185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.870s	user 0.034s	sys 0.057s
W20260709 13:06:16.873762 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 1.87085e+06us
I20260709 13:06:16.876410 22393 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.877434 22393 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.878925 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet b8d00e740dba47b2b43be9e4ef1ecb55 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:16.879997 22400 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:16.880841 22400 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.706892 22379 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.885144 22400 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:16.886299 22379 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.905436 22393 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:16.872635 22401 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:16.913587 22404 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.917474 22293 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:16.918319 22293 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.920465 22293 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:16.921002 22293 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:16.917440 22365 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:16.921576 22293 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.925319 22365 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:16.927522 21143 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.893s	user 0.032s	sys 0.025s
I20260709 13:06:16.929605 22366 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.930751 22389 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:16.934257 22389 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.937639 22389 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:16.927187 22379 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:16.916217 22395 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:16.952579 22204 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:16.953434 22204 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.953773 21070 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:06:16.956182 21187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.010s	sys 0.015s
I20260709 13:06:16.930377 22366 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:16.966459 21187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.010s	sys 0.015s
I20260709 13:06:16.983503 22390 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:16.983911 22365 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:16.984395 22390 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:16.985625 22405 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:16.946383 22378 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.916801 22404 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.088845 22293 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.089716 22293 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.090226 22293 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.091724 22394 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.088719 22398 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:16.956645 22393 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.144482 21070 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:17.172266 22293 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.184665 22204 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.092319 22394 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.220472 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.221496 22396 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.221639 22402 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.222671 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2079) took 2385 ms. Trace:
I20260709 13:06:17.222311 22378 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.220351 22410 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.224484 22395 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.162420 22405 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.261149 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.329540 22382 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:17.330726 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.328855 21070 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:17.329344 21143 rpcz_store.cc:276] 0709 13:06:14.836723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.836823 (+   100us) service_pool.cc:224] Handling call
0709 13:06:14.842070 (+  5247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.842333 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.842349 (+    16us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:14.848524 (+  6175us) log.cc:844] Serialized 572 byte log entry
0709 13:06:15.010767 (+162243us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b18000c6340. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1
0709 13:06:15.010960 (+   193us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:16.929800 (+1918840us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.221939 (+292139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.222082 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.222182 (+   100us) raft_consensus.cc:1685] finished
0709 13:06:17.222300 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.222523 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21109,"raft.queue_time_us":149148,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":277,"spinlock_wait_cycles":39394560,"thread_start_us":246701,"threads_started":2,"wal-append.queue_time_us":257976,"wal-append.run_cpu_time_us":2305,"wal-append.run_wall_time_us":1348943}
I20260709 13:06:17.298738 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.342552 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.224812 22366 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.270241 22204 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:17.331130 22394 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.351585 21071 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:17.302788 22397 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.226089 22390 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.262300 22398 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.321991 22296 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.330472 22382 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.360486 22399 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.335098 22395 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.234918 22393 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.331331 22374 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:17.265146 22269 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.312503 22408 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.341718 22379 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.371786 22409 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:17.371711 22296 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:17.372501 22397 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.372797 22204 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.320397 22406 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.249035 22407 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.356729 22365 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.363984 22400 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:06:17.381049 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 318072us
I20260709 13:06:17.371624 22217 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.366942 22366 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.368337 22394 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.363687 22402 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.393990 22401 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.398006 22401 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.400173 22293 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:17.401171 19972 debug-util.cc:398] Leaking SignalData structure 0x7b080000f8a0 after lost signal to thread 21185
I20260709 13:06:17.359323 22396 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.417124 22412 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.415560 22365 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.410938 22396 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.391366 22390 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:17.400187 22402 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.419034 22217 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.433059 22293 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.392475 22407 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.438653 22407 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.415236 22399 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.417970 22412 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.401680 22395 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.439623 22394 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.379592 22410 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.447369 22204 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.449347 22365 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.450034 22394 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.450479 22410 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.460270 21070 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:17.419135 22374 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.428241 22366 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.466463 20957 catalog_manager.cc:5028] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): tablet d8a4c442e73243eda44d43ba0eaf0d2a (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:06:17.466796 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.421393 22379 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.446853 22396 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.477053 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.478838 22415 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:17.480219 21146 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:17.422729 22269 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.434482 22400 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.439838 22390 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.491197 22293 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.500854 22412 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.490654 22217 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.464349 22404 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.421512 22408 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.504741 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.468016 22413 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.441449 22409 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.479810 22415 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.509126 22414 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.520632 21069 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:17.510519 22378 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.522250 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.442706 22406 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.522552 22374 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.524801 21140 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:17.507030 22396 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.510923 22404 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.520654 22393 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.530575 22393 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.531248 22400 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.532688 22269 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.533214 22269 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.533710 22269 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.535390 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.522114 22407 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.536906 22407 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.538938 22407 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.540787 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.538751 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.545337 21146 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:17.531313 22393 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.546237 22397 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.547000 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.547762 21140 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:17.548062 22393 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.542692 22401 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.549973 22395 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.550925 22390 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.545394 22422 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.529681 22296 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.552433 22404 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.553247 22366 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.517815 22405 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.529042 22382 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.531268 22396 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.566809 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.527506 22408 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.542166 22398 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.571769 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.555945 22409 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.574514 21147 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:17.572729 21144 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:17.566651 22404 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.525887 22413 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.555577 22422 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.584098 22269 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.575347 22366 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:17.584975 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.585016 22366 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.554497 22414 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.587037 22422 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.587071 22269 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:17.587844 22422 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.588515 22422 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.590958 22406 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.590777 22398 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:17.592590 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:17.580848 22399 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.596153 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:17.596544 22399 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:17.597018 22399 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.597131 22404 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:17.597519 22399 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:17.597580 22404 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:17.598097 22404 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:17.598233 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:17.601598 22415 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.602301 22415 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:17.602375 22296 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:17.603124 22415 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.603348 22296 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.604979 22420 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:17.605872 22415 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.606763 22417 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:17.606997 22415 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.607831 22415 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.607669 22417 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.606158 22420 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.609988 22395 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.610859 22395 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.613370 22395 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.613981 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.614719 21070 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:17.600289 22406 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.615968 22406 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.616835 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.606048 22296 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.615983 22422 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.606406 22382 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.629801 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.630249 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.631114 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.610158 22415 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.628477 22410 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.637645 22422 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.645888 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.643332 22420 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.649313 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:17.633203 22382 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.648643 22413 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.662101 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.650774 22417 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.669394 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.663234 22421 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.670930 22398 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:17.676270 22398 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.676805 22398 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:17.652462 22410 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.655902 22414 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.668725 22382 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.663822 22422 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.690290 22366 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.662215 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.698421 22427 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:17.685331 22423 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.772454 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.830683 22275 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.831048 21219 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:17.831411 22275 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.833624 22275 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:17.835215 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.698868 22410 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:17.857435 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.857901 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.860387 21144 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:17.867617 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.868534 21144 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:17.869694 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.871124 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.690990 22421 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.831418 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.872057 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.881248 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.698431 22425 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.884472 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.885196 21221 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:17.871801 22423 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:17.884986 22422 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.898684 22428 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:17.899398 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.901358 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.897451 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.885602 21065 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:17.903926 21216 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:17.904860 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.885881 21219 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:17.904320 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.908813 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.911096 21221 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:06:17.912083 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.912103 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.914841 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.904716 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.910284 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.917963 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.919667 22423 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:17.920468 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.921074 21216 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:17.914875 21221 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:17.899502 22428 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.905171 22422 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:17.884635 22425 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.897524 22427 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.927783 20968 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.935560 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.940234 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.940902 21072 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:17.936944 21145 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:17.944269 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.944713 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.945348 22422 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:17.946766 21072 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:06:17.948781 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2097) took 1040 ms. Trace:
I20260709 13:06:17.951135 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.949970 21139 rpcz_store.cc:276] 0709 13:06:16.906875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.945765 (+1038890us) service_pool.cc:224] Handling call
0709 13:06:17.946071 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:17.946247 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.946274 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:17.947196 (+   922us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:17.947238 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.947324 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.947439 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:17.949265 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.935879 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:17.951781 21143 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:17.955394 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.955603 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:18.001658 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.002697 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:17.977221 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:17.977437 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.013736 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.014788 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.015117 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.006186 22398 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.001230 22425 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:18.021127 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 67335us
W20260709 13:06:18.030725 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50069.5us
I20260709 13:06:18.031657 21767 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:18.032441 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.033327 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.033708 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.034489 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.035205 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.043907 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.026572 22382 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.046501 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.056526 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.056710 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.039165 22428 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.030056 22399 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:18.131975 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 67943.9us
I20260709 13:06:18.161573 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.161638 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.055020 22382 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:18.168478 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 106328us
I20260709 13:06:18.169035 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.042852 22430 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.129s
I20260709 13:06:18.060335 22431 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.175583 21140 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
W20260709 13:06:18.178419 22432 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:06:18.179814 22425 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.180401 22425 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.185209 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.186200 21139 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:18.167354 22382 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.167564 22398 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:18.192613 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.174605 22428 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.184748 22422 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:18.196688 22404 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.003s
I20260709 13:06:18.226660 22421 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:18.229079 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.231524 21146 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:18.198139 22428 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.184906 22433 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:06:18.243788 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.244019 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.229635 22435 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.235378 22428 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.198918 22431 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.245227 22425 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:18.244530 21214 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:18.230649 22404 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.267480 22426 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:06:18.282605 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.283557 21214 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:18.284504 22404 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:18.284890 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.285148 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.286700 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.287075 22273 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.265552 22399 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.271838 22425 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:18.296036 22431 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:18.296576 22425 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.290136 21146 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:18.295375 22273 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.277518 22435 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.303735 22273 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:18.289986 22426 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.291759 22433 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:18.308934 22433 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.301075 22425 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:18.313215 22433 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.315876 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.317579 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.313043 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.318976 22398 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.322239 22398 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.322782 22398 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.320039 22399 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.344247 22382 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:18.344798 22399 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.345273 22399 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.345907 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.346463 22428 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.347445 22433 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:18.348026 22433 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.348769 22428 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.348870 22440 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.349431 22428 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.349786 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.349664 22440 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.350575 22433 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.350183 22428 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.351238 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.351722 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.352149 22379 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:18.339880 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 166734us
I20260709 13:06:18.352384 22440 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.353655 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.307013 22435 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.354530 22426 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.354861 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:18.355005 22398 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:18.355527 22382 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:18.352555 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.356343 21214 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:18.356834 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:18.357542 22398 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.357893 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.358728 21214 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:18.360113 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.354025 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:18.361435 22368 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.361800 22368 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.362188 22426 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:18.362728 22426 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:18.363077 22426 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:18.365046 22399 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:18.362305 22368 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.367427 22399 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.367457 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.368395 21212 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:18.361828 20969 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 246) took 1470 ms. Trace:
I20260709 13:06:18.369019 20969 rpcz_store.cc:276] 0709 13:06:16.891756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.401590 (+509834us) service_pool.cc:224] Handling call
0709 13:06:18.361654 (+960064us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:06:18.369391 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.370325 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.371549 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.371803 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.372326 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:18.372745 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.373714 21212 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.374562 21214 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:18.376024 21221 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:18.376720 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1306) took 1013 ms. Trace:
I20260709 13:06:18.377099 21071 rpcz_store.cc:276] 0709 13:06:17.363360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.363495 (+   135us) service_pool.cc:224] Handling call
0709 13:06:17.365304 (+  1809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:17.430641 (+ 65337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.430672 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:17.435104 (+  4432us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:17.435145 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.376481 (+941336us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b54000dd254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:18.376537 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:18.376668 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139508096}
I20260709 13:06:18.380386 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:18.380257 22368 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:18.381493 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:18.371819 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800034940 after lost signal to thread 21004
I20260709 13:06:18.382228 22437 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:18.382593 22437 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:18.382954 22437 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:18.378999 22426 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:18.383198 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08003477a0 after lost signal to thread 21096
W20260709 13:06:18.386544 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800347440 after lost signal to thread 21097
I20260709 13:06:18.386926 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:18.387821 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08003200c0 after lost signal to thread 21171
I20260709 13:06:18.387677 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.388319 21070 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:18.384392 22260 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.389254 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.389420 22260 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.389804 22260 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.391779 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.393086 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:18.394819 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800323cc0 after lost signal to thread 21172
I20260709 13:06:18.395179 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.395627 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.395991 22406 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:18.396027 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08002db080 after lost signal to thread 21245
I20260709 13:06:18.396421 21216 raft_consensus.cc:2410] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:18.396802 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.397012 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.397459 22427 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:18.397629 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.383651 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.399096 22427 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.399542 22427 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.399919 22427 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.403899 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.405229 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.405336 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:18.408053 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.409044 22275 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.410956 22435 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.411396 22382 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:18.411921 22382 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:18.411649 22435 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.412354 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.412385 22382 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:18.413862 22435 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.417440 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.418319 21217 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:18.420195 22435 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.420871 22438 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.421645 22433 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.421536 22438 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.422281 22433 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.423930 22438 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:18.424311 22433 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.424503 22438 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.424881 22433 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.425148 22438 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.425427 22433 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.427371 22438 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.420820 22435 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.431478 22435 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.431550 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:18.432308 22433 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.433794 22170 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.434257 22170 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.434604 22170 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.383185 22368 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.435475 22396 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:18.439440 22436 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.443585 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.443842 22428 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.445467 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.446035 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.447844 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.449885 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:18.450181 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.450484 21069 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.450804 21071 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:18.409991 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.453783 22396 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.454015 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:18.455708 21001 debug-util.cc:398] Leaking SignalData structure 0x7b080024eea0 after lost signal to thread 21825
W20260709 13:06:18.456190 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800316d80 after lost signal to thread 21826
I20260709 13:06:18.458616 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.460466 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.460968 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.460968 21147 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:18.453230 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.463713 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.440045 22436 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.467955 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.469499 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:18.470021 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:18.483129 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.491197 22260 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:18.492677 22170 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:18.492759 21146 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:18.496455 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.516213 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:06:18.522415 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 53018.7us
I20260709 13:06:18.525372 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.526577 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.527459 22260 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.386011 22426 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:18.466933 22396 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.493009 22414 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.532903 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:18.537348 21184 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.596s (SENT)
I20260709 13:06:18.535785 22382 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.532703 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:18.551924 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 140691us
I20260709 13:06:18.534340 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.496114 22407 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:18.551051 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:18.546397 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.538343 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.554189 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:18.467209 22427 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:18.560051 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.534809 22404 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.579679 22404 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.580221 22404 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.580328 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:18.581552 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:18.534977 22418 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.578804 22170 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.583539 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:18.528187 22398 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:06:18.545902 22414 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:18.589308 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52160.8us
I20260709 13:06:18.590479 22260 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.555058 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:18.555559 22407 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:18.598515 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.599222 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.613018 21069 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:18.601210 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.605010 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.577531 22392 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.528865 22437 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.600152 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.638499 22260 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.636443 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.619689 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.687252 22260 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.686203 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.582482 22418 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:18.578173 22408 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260709 13:06:18.688946 21069 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:18.697875 22410 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.608136 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.698395 22410 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:18.699039 22428 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.614560 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:18.700901 21220 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:18.608362 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:18.702399 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 115945us
I20260709 13:06:18.690224 22407 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:18.587023 22433 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:06:18.710813 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.711555 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.607529 22295 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:18.620810 22436 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:18.715555 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.725675 21072 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:18.597791 22414 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.716019 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.722034 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.686813 22392 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:18.717677 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 129703us
I20260709 13:06:18.733872 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.619025 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.624128 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.623559 22368 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:18.619411 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.621234 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.702001 22418 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.731515 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.716344 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.619465 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.714821 22428 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:06:18.737937 22392 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.617362 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.728327 21214 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:18.715258 21217 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.739480 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:18.737560 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 201970us
I20260709 13:06:18.742313 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.742796 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.729275 21070 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:18.742727 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:18.763458 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.745523 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.725162 22295 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:18.727933 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.764766 22295 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:18.758304 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:18.740887 22368 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:18.766419 22368 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.767836 21765 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.733781 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.623926 22382 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:18.728001 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.711089 21767 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:18.724795 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:18.748525 22440 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.726015 22435 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:06:18.770565 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:18.816543 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.768703 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.820690 21799 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:18.748116 21794 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:18.814878 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.828192 21220 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:18.768957 22425 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.820384 21793 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:18.829432 22427 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:18.830940 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 60412.4us
I20260709 13:06:18.838222 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.857632 22260 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:18.858698 21072 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:18.862247 22260 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:18.862548 21069 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:18.863526 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.863826 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.864782 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.865296 21064 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:18.867158 21066 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:18.825866 22433 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.843894 22439 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:18.822573 21792 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:18.871712 22440 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.883363 22437 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:18.893921 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.900344 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.901997 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.891619 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.903311 21796 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:18.904264 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.889987 22440 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:18.905506 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.906287 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.906633 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.908921 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.909008 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.893538 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.911339 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:18.911657 21063 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:18.906312 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.911013 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.914670 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.914572 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.917899 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:18.916323 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 154648us
I20260709 13:06:18.919569 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.906561 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.905164 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.920833 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:18.917075 22452 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:06:18.940187 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:18.941183 21141 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:18.943763 21144 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:18.947273 21145 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:18.927070 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.951946 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:18.932858 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.939822 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.945876 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.935946 22368 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:18.953361 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.981693 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:18.981273 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.001289 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.981259 22440 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.073303 22414 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.023011 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.003785 22295 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.050221 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.077555 22414 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.080247 22295 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:18.999879 22436 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:19.003499 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.081133 22436 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:19.082113 22436 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.084223 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.084764 22436 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.018245 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.051971 22421 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.052910 22275 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.053402 22399 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.054190 22457 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.087812 22414 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.037436 22426 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.093544 22457 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.094317 22414 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:19.037451 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 44173.2us
I20260709 13:06:19.095944 22457 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:19.096664 22414 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:19.013062 22396 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.097558 22396 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.098929 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:19.099727 22396 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:18.992726 21071 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:19.014088 22407 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:18.971851 22463 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:19.054344 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.054268 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:19.023211 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:18.995939 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.054271 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.017751 22438 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:19.012107 21035 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:19.017006 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.013698 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.117405 21142 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:19.000176 22255 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:19.020166 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:19.057380 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 76827.6us
I20260709 13:06:18.995347 22410 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.058933 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.059855 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.054437 22413 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:06:19.064973 22392 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.066076 21797 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:19.066591 22418 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.067759 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.068178 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.068665 20968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 233) took 1652 ms. Trace:
I20260709 13:06:19.068828 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.068647 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.071779 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:18.996241 22404 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.048267 22428 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:19.085031 22433 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.085300 21213 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:19.085856 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.087026 22379 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:19.087162 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64249.5us
I20260709 13:06:19.092624 22421 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.094432 22426 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:06:19.087188 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 386612us
I20260709 13:06:18.994704 22368 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:19.111011 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 99116.2us
W20260709 13:06:19.124127 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 65282.7us
I20260709 13:06:19.104753 22463 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:19.164501 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.128793 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.125711 22295 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:06:19.145207 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.161326 21068 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:19.147440 22404 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.139688 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.148257 21147 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:19.103510 22399 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.109066 22396 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.103375 22275 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.181950 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:19.182806 21139 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:19.182739 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.116539 22438 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:19.184132 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:19.183966 22438 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.185169 22438 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.121811 22456 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:19.191572 22456 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:19.191658 21141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49704
I20260709 13:06:19.192273 22456 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.192562 21141 raft_consensus.cc:606] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:06:19.155085 22379 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:19.158576 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:19.159068 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 73284us
I20260709 13:06:19.160717 21069 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:19.198158 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.152050 21065 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:19.128506 22414 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.167634 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.167826 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.168869 21801 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:06:19.146610 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:19.202168 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.202411 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.202211 22375 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:19.203233 22375 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:19.203627 22375 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.169479 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:19.168704 21035 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.167734 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.149241 22428 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:19.174749 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.177448 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:19.178937 21143 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:19.173203 21793 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:19.185191 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.188702 20970 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 215) took 2203 ms. Trace:
I20260709 13:06:19.189385 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:19.189662 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:19.170053 22463 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:19.193794 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54182.4us
I20260709 13:06:19.158491 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:19.212831 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 54401.5us
I20260709 13:06:19.212953 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.213647 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.211046 20970 rpcz_store.cc:276] 0709 13:06:16.886648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.007769 (+1121121us) service_pool.cc:224] Handling call
0709 13:06:19.089267 (+1081498us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":198732,"reported_tablets":5}
I20260709 13:06:19.214511 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.215788 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.216102 22255 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.186628 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.199927 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:19.201346 22275 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:19.168623 22469 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.206653 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.207413 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.197055 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:19.208359 22439 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.209599 21142 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:19.212154 21145 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 2: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:19.216970 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.218129 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.219575 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.141558 20968 rpcz_store.cc:276] 0709 13:06:17.417012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.425725 (+  8713us) service_pool.cc:224] Handling call
0709 13:06:19.068568 (+1642843us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:18.192144 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:18.253458 (+ 61314us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:18.253516 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:06:18.257742 (+  4226us) tablet.cc:662] Decoding operations
0709 13:06:18.301765 (+ 44023us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:18.301907 (+   142us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:18.301953 (+    46us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:18.302352 (+   399us) tablet.cc:701] Row locks acquired
0709 13:06:18.302412 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:06:18.302653 (+   241us) write_op.cc:270] Start()
0709 13:06:18.302851 (+   198us) write_op.cc:276] Timestamp: P: 1783602378302575 usec, L: 0
0709 13:06:18.302908 (+    57us) op_driver.cc:348] REPLICATION: starting
0709 13:06:18.314440 (+ 11532us) log.cc:844] Serialized 603 byte log entry
0709 13:06:18.625751 (+311311us) op_driver.cc:464] REPLICATION: finished
0709 13:06:18.877647 (+251896us) write_op.cc:301] APPLY: starting
0709 13:06:18.877779 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:18.877950 (+   171us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:18.878181 (+   231us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:18.905811 (+ 27630us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:18.906057 (+   246us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:06:18.906127 (+    70us) write_op.cc:312] APPLY: finished
0709 13:06:18.906411 (+   284us) log.cc:844] Serialized 23 byte log entry
0709 13:06:18.906630 (+   219us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:18.907801 (+  1171us) write_op.cc:454] Released schema lock
0709 13:06:18.907858 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:18.908227 (+   369us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:18.908278 (+    51us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":197030,"apply.run_cpu_time_us":2751,"apply.run_wall_time_us":35971,"num_ops":1,"prepare.queue_time_us":236620,"prepare.run_cpu_time_us":8624,"prepare.run_wall_time_us":187088,"raft.queue_time_us":264165,"raft.run_cpu_time_us":34812,"raft.run_wall_time_us":132035,"replication_time_us":322646,"thread_start_us":411273,"threads_started":4,"wal-append.queue_time_us":44067}]]}
I20260709 13:06:19.199535 22379 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.225669 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.239640 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.206243 22440 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.240816 22458 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.227799 22445 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.227679 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.229035 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.221776 22366 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.229665 22433 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.229868 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.221666 22382 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:19.230486 22466 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.236439 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:19.231034 22425 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:19.240985 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:19.242034 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.224731 22465 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:19.317811 22465 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:19.318367 22465 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.240648 22459 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:19.320542 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:19.245689 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:19.224453 22446 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:19.240319 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:19.332373 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 92113.5us
I20260709 13:06:19.333575 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.206048 22396 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.308024 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.306563 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.238974 22457 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:06:19.330660 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 88712.5us
I20260709 13:06:19.245213 22435 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.339498 22459 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:19.349984 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.345728 22255 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.345932 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.354593 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.247545 22439 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.309736 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.316900 22382 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:19.315989 22418 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.368606 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.341738 22469 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.342355 22458 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.351599 22459 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.350044 22446 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:19.394090 22446 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.394119 21798 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:19.346900 22413 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.358711 22433 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.362355 22437 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.358337 22408 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.388617 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.389099 22421 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.370438 22382 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.411660 21799 raft_consensus.cc:2410] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:19.376160 22392 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.399175 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:19.408010 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.369395 22466 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.406672 22455 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:19.416225 22396 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.414520 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.405870 22426 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
W20260709 13:06:19.422821 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 68321.6us
I20260709 13:06:19.423892 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.412256 22437 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.426769 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.428558 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.431744 22417 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.433099 22417 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:19.433818 22379 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.434283 22255 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.434648 22255 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:19.435534 22255 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.428333 22408 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.437626 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.427953 22295 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.424036 22455 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:19.440244 22255 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.427168 22466 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:06:19.443934 22455 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.444931 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.446256 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.446395 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.448057 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.448303 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.446576 22474 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:19.451086 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:19.451954 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 226992us
I20260709 13:06:19.445834 22437 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.438356 22368 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:06:19.446245 22445 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.458199 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.449245 22392 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.451067 22413 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.463639 22476 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:06:19.452461 22389 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.450827 22462 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.464246 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.476028 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.454650 22474 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:19.481678 22474 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.467180 22458 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.477598 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.470937 22379 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:19.474116 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.466812 22425 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:06:19.488018 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.490610 22406 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:19.491962 22398 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.493026 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.493306 22406 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.493759 22398 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.494930 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.495692 22398 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.497242 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.491644 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.487818 22423 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.498970 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.499667 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.500696 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.499727 22392 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.501539 22467 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.501999 22467 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:19.503229 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.505900 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.502056 22426 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:19.488786 22466 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.507973 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.515957 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.500463 22398 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.500306 22413 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.500895 22456 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.504624 22396 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.506182 22458 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.508131 22418 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.506631 22392 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.505565 22421 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.525240 21071 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:19.526741 22389 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:19.527999 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:19.504024 22467 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.530321 21138 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:19.531301 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:19.529595 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:19.527424 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:19.533769 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.548584 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.549404 21212 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:19.544337 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:19.550016 21219 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:19.550247 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.533423 22456 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:19.535193 22410 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.537487 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.544322 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.545820 22459 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.524576 22423 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.524689 22458 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:19.528400 22463 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.567606 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.568204 21214 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:19.569109 22463 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.569274 22458 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.575897 22458 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.575714 22375 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.561911 22459 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.589409 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:19.591295 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.560990 22475 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.592406 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:19.593119 21220 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:19.595573 22375 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.557893 22465 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.547075 22455 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.591137 22467 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.615619 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:19.616312 21220 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:19.601359 21214 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:19.626317 22446 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.613926 22382 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.646135 21072 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:19.650179 21217 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:19.655654 22407 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:19.657924 22465 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:19.659170 22455 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.672413 22428 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.674013 22428 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:19.700093 22399 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.704365 22474 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.706418 21794 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:19.706766 22474 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.712708 22408 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.713140 22408 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:19.714098 22408 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.715612 22446 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:19.716239 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.724612 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.726888 21215 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:19.729965 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.731753 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.733319 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:19.735993 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 271861us
W20260709 13:06:19.736997 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 219075us
I20260709 13:06:19.741014 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.741225 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:19.742347 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.743660 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.745779 22480 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.745908 22379 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:19.737978 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.735774 22458 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.746578 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:19.735992 22466 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.747484 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.756613 21184 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.756889 22485 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.757551 22485 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.758224 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.758248 22485 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:19.759996 22485 consensus_queue.cc:1243] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:19.760681 22482 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.735081 22410 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.750674 20969 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:06:19.765683 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.772956 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:19.774554 22413 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:19.774837 22438 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:19.775004 22413 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.775256 22438 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:19.775472 22413 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.776197 22438 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.748484 22379 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:19.777455 22438 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.777683 22379 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.778666 21065 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:19.780213 22437 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.780673 22437 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:19.781538 22437 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.781975 22456 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:19.782727 22437 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.783270 22475 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.783730 22475 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:19.784592 22475 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.785054 22413 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:19.787487 22413 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:19.789449 22382 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:19.791077 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.795074 21063 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:19.795573 21070 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:19.796499 21069 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:19.797510 22469 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:06:19.801447 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:19.801652 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:19.801734 22428 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:19.803709 22469 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:19.804102 22469 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:19.804968 22469 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.808799 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:19.797799 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:19.811184 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:19.813382 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:19.813562 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.814368 22275 raft_consensus.cc:993] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:06:19.806972 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.817076 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.817550 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.818135 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.818447 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.819340 22436 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.820072 22436 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.820756 22436 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.810402 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:19.824966 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.826334 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.829175 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:19.831236 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:19.833151 22418 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.833559 22423 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.841521 22469 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.842092 22413 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:06:19.845428 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 98027.8us
I20260709 13:06:19.846604 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.847457 22366 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.847858 22275 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.848131 22366 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.848487 22275 raft_consensus.cc:1081] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:06:19.848731 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.849632 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.835770 22379 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.836607 21797 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:19.837196 22408 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.822055 20970 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:19.840030 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:19.840319 22406 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.855901 22255 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.858898 22406 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:19.859828 22406 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:19.861191 22406 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.856441 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.857493 22463 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:06:19.863170 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.865006 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.865262 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.865861 22255 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.866727 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.848093 22475 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.869913 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:19.880093 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:19.881037 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.865530 22455 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.883450 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:19.884728 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.885039 22255 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:19.869439 22366 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.866478 22421 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:06:19.882258 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.868745 22423 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.871460 22382 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.877660 22413 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.877264 22295 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.882735 22396 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.892370 22482 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.886889 22455 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:19.901585 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.911520 22255 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:19.915721 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.916805 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.918008 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.891794 22407 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:19.919131 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.888635 22426 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.924043 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.920917 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:19.920480 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:19.929740 22446 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.933173 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.934142 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.891614 22399 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:06:19.937088 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.911264 22410 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.941037 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.944701 22260 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.942344 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.920507 22421 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.948284 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.953024 21215 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:19.940613 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:19.929019 22447 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.925395 22407 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.921657 22417 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:06:19.960053 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:19.930696 22413 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:19.954584 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.968142 21212 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 2: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:19.949739 22440 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.947386 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:19.923167 22467 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:06:19.947227 22433 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.947021 22410 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:19.967648 22413 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:19.998296 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:20.000779 22479 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.963914 22366 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.966303 22407 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.006935 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:19.971004 22463 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:20.002187 22413 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:20.017338 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.027387 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:20.029619 22467 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:20.065920 21065 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.066411 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:20.067420 21217 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:20.071342 22396 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.071990 22396 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:20.073145 22396 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.129334 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:20.037285 22463 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.124447 22410 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.132371 21216 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:20.075055 22396 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:20.135048 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 251694us
W20260709 13:06:20.147657 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 266973us
I20260709 13:06:20.148433 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:20.015930 22455 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:20.167666 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.159135 22441 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:20.156461 22399 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:20.192384 21217 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:20.204628 21800 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:20.163558 22494 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:06:20.164038 22421 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:06:20.185367 22399 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:20.214087 21143 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.197190 22418 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:06:20.234045 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:20.205086 22455 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.215400 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.247965 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.167052 22275 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:06:20.244141 21216 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.262701 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.241027 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.238514 22399 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.253886 21140 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 2: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:20.202076 21795 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:20.254590 21141 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
W20260709 13:06:20.255915 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44054.1us
I20260709 13:06:20.245147 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.265015 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.269887 21140 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.269981 21143 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.271682 21140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49704
I20260709 13:06:20.274464 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.246786 21069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39510
I20260709 13:06:20.278199 21069 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:20.278625 21069 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.269526 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:20.212983 22463 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.287957 22421 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:20.287420 22495 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
W20260709 13:06:20.344144 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 116921us
I20260709 13:06:20.345141 21140 raft_consensus.cc:606] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:06:20.347251 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.351518 22446 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:20.352712 22446 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.353088 22446 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.362376 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.363507 22260 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:20.370474 21143 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.367974 22392 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:20.328984 22421 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:20.413749 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.415374 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:20.375422 21216 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:20.379166 21069 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.429759 22260 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:20.386099 22504 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.437152 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.439565 21069 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:20.440147 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.353936 22446 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.440742 22437 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:20.441040 22456 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:06:20.381906 22413 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.446061 22444 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:20.458029 21217 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:20.458621 22413 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:20.459224 22421 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:20.446341 22504 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:20.460273 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:20.453950 22446 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.448836 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:20.454506 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:20.457338 21138 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 2. Current term is 3. Ops: []
W20260709 13:06:20.456073 22475 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
I20260709 13:06:20.448207 21213 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:20.448153 22482 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.469609 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:20.491005 22482 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:20.491550 21793 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.467851 20969 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 248) took 1301 ms. Trace:
I20260709 13:06:20.491838 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:20.494602 22482 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.495925 22482 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.502302 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:20.504086 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:20.504812 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:20.505837 22476 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.510232 21069 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.510774 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.511859 21072 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.507951 21145 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:20.502188 22504 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.518220 22504 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.508559 22476 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:20.500355 21793 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:20.519436 22476 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.520514 22476 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.516707 22440 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:20.457280 22456 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.508703 22469 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:20.440109 22407 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:20.534406 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.534754 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 94721.5us
I20260709 13:06:20.530813 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:20.537276 21219 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.516561 22379 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:20.493541 20969 rpcz_store.cc:276] 0709 13:06:19.166086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.166246 (+   160us) service_pool.cc:224] Handling call
0709 13:06:20.467791 (+1301545us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:19.826178 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:19.826788 (+   610us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:19.826823 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:06:19.826843 (+    20us) tablet.cc:662] Decoding operations
0709 13:06:19.827163 (+   320us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:19.827239 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:19.827265 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:19.827492 (+   227us) tablet.cc:701] Row locks acquired
0709 13:06:19.827508 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:06:19.827647 (+   139us) write_op.cc:270] Start()
0709 13:06:19.827749 (+   102us) write_op.cc:276] Timestamp: P: 1783602379827605 usec, L: 0
0709 13:06:19.827769 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:06:19.828369 (+   600us) log.cc:844] Serialized 544 byte log entry
0709 13:06:19.853983 (+ 25614us) op_driver.cc:464] REPLICATION: finished
0709 13:06:20.147538 (+293555us) write_op.cc:301] APPLY: starting
0709 13:06:20.152596 (+  5058us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:20.161778 (+  9182us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:20.190130 (+ 28352us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:20.207655 (+ 17525us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:20.207807 (+   152us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:06:20.207849 (+    42us) write_op.cc:312] APPLY: finished
0709 13:06:20.263950 (+ 56101us) log.cc:844] Serialized 23 byte log entry
0709 13:06:20.275569 (+ 11619us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:20.284105 (+  8536us) write_op.cc:454] Released schema lock
0709 13:06:20.289395 (+  5290us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:20.329888 (+ 40493us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:20.333961 (+  4073us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":206346,"apply.run_cpu_time_us":3222,"apply.run_wall_time_us":185338,"num_ops":1,"prepare.queue_time_us":46220,"prepare.run_cpu_time_us":2468,"prepare.run_wall_time_us":2467,"replication_time_us":26055,"thread_start_us":159626,"threads_started":2}]]}
I20260709 13:06:20.540599 21146 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
W20260709 13:06:20.543642 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.541742 22275 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:20.551635 22275 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:20.552759 22275 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.554152 22275 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.556875 22407 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:20.558601 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:20.522065 22433 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:06:20.516314 22399 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:20.561297 21138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49704
I20260709 13:06:20.561866 22399 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:20.562430 21072 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:20.562973 21065 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.563093 22457 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:20.563553 22457 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:20.528326 22456 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.564358 22457 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.564867 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:20.564810 22456 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.565665 21072 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:20.566135 22456 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.565635 22457 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.567685 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.562530 21144 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.571592 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:20.572883 21035 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:20.570794 20968 catalog_manager.cc:5697] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.530345 22469 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:20.579249 22469 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.579824 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:20.577688 22366 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:20.581858 21069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:20.584164 21068 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:20.586527 22375 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:20.583186 22407 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:06:20.592406 22423 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:20.592855 22423 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:20.583321 22455 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:20.601354 22455 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.599054 22423 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.601980 22375 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.602252 22423 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.602414 22375 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.603446 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:20.604053 22375 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.605202 22375 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.605863 22410 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.606817 22504 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:20.607374 22504 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:20.611250 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:20.608250 22504 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.621601 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:20.622712 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:20.624323 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:20.628391 21065 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:20.629868 22379 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:06:20.633834 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.636234 22426 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:20.637127 22407 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:20.637669 22426 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:20.638063 22426 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:20.642289 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.643401 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:20.644236 21216 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:20.647337 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:06:20.618132 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.656497 20968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 234) took 1121 ms. Trace:
I20260709 13:06:20.646811 22477 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:06:20.656899 20968 rpcz_store.cc:276] 0709 13:06:19.534796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.534955 (+   159us) service_pool.cc:224] Handling call
0709 13:06:20.656441 (+1121486us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:20.579519 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:20.580084 (+   565us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:20.580119 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:06:20.580135 (+    16us) tablet.cc:662] Decoding operations
0709 13:06:20.580469 (+   334us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:20.580534 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:20.580559 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:20.580754 (+   195us) tablet.cc:701] Row locks acquired
0709 13:06:20.580768 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:06:20.580895 (+   127us) write_op.cc:270] Start()
0709 13:06:20.581055 (+   160us) write_op.cc:276] Timestamp: P: 1783602380580856 usec, L: 0
0709 13:06:20.581071 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:06:20.586000 (+  4929us) log.cc:844] Serialized 544 byte log entry
0709 13:06:20.588115 (+  2115us) op_driver.cc:464] REPLICATION: finished
0709 13:06:20.588666 (+   551us) write_op.cc:301] APPLY: starting
0709 13:06:20.588756 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:20.588878 (+   122us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:20.589802 (+   924us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:20.590090 (+   288us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:20.590253 (+   163us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:06:20.590300 (+    47us) write_op.cc:312] APPLY: finished
0709 13:06:20.591500 (+  1200us) log.cc:844] Serialized 23 byte log entry
0709 13:06:20.591655 (+   155us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:20.591739 (+    84us) write_op.cc:454] Released schema lock
0709 13:06:20.591775 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:20.595138 (+  3363us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:20.595171 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":109501,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":298,"apply.run_cpu_time_us":2057,"apply.run_wall_time_us":3443,"num_ops":1,"prepare.queue_time_us":364,"prepare.run_cpu_time_us":2546,"prepare.run_wall_time_us":6744,"replication_time_us":6887}]]}
I20260709 13:06:20.670358 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.671593 21138 raft_consensus.cc:606] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Received request to transfer leadership
I20260709 13:06:20.672258 21138 raft_consensus.cc:614] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:20.671124 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:20.646498 22503 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:20.656931 22502 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:20.648763 22426 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.675321 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:20.688156 22260 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:20.688886 22260 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.689990 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:20.683624 21140 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:20.684502 21069 raft_consensus.cc:606] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.690865 21069 raft_consensus.cc:614] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:20.692878 22469 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.693555 22485 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.694018 21800 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:20.694072 22485 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.695401 22502 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:20.696382 22502 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.697072 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:20.697257 22485 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:20.697696 21069 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:20.706046 21069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:20.721127 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:20.721510 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:20.726682 21069 raft_consensus.cc:606] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.721542 22260 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.722756 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:20.704152 22459 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:20.640153 22455 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.689008 22407 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:20.735451 22407 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.736235 22469 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:20.683574 22504 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:20.736384 21138 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:20.737286 22459 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.737718 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:20.738272 22459 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.739468 22375 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:20.746088 22407 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:20.722754 22426 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:20.752266 21018 auto_rebalancer.cc:893] Removing replace marker failed: Replica 5e5e65640c8b4f5ca7eadc6a14312c55 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:20.752363 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:20.752956 21018 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:06:20.753240 21018 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:06:20.753248 21072 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:06:20.725039 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49826us
I20260709 13:06:20.698936 22446 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:06:20.726516 22457 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:20.734719 22482 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:20.775851 22482 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.776252 22482 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.734826 22410 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:20.779543 22410 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.782006 22410 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:20.782677 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:20.782949 21767 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:20.783424 21068 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:20.784291 22469 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:20.775306 22446 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.784642 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:20.785008 22446 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.785949 22446 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.786407 22410 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:20.786870 22410 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:20.787145 22446 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.787266 22410 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.787853 22459 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.788218 22482 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.790506 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:20.791438 21140 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 2: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:20.798939 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:20.801599 21219 raft_consensus.cc:2368] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:20.803959 21068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a473266451804da089da06e9c927e7ee"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:20.804682 21068 raft_consensus.cc:606] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.805578 21068 raft_consensus.cc:614] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:20.813028 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54404.4us
I20260709 13:06:20.814572 21111 leader_election.cc:400] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:20.815481 21111 leader_election.cc:403] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:20.817376 21069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:20.818023 21069 raft_consensus.cc:606] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:20.818372 22504 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.827168 22465 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:20.828286 22477 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.828666 22477 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.828806 22465 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:20.829097 21069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:20.829780 21069 raft_consensus.cc:606] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.829591 22477 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:20.829954 22375 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:20.830467 22375 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.831346 22375 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.830139 21069 raft_consensus.cc:614] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:20.833642 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:20.835450 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:20.835800 22504 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:20.831091 22477 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.836970 22375 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.845664 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:20.846446 22260 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:20.847182 22260 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:20.847074 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.852561 22410 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.854871 22410 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:20.855989 22482 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:20.856319 22465 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:20.857479 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.859599 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:20.859599 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.860489 21140 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 2: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:20.861335 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:20.861937 21068 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.858722 22260 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.874423 22428 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.875002 22428 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.883080 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:20.883800 21072 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:20.884169 22428 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:20.883080 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:20.891083 21140 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:20.891965 21068 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:20.885339 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:20.887123 22426 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.893674 22426 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.897068 22426 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:20.898535 21138 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:20.899775 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:20.900348 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:20.900704 22485 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:20.901144 21217 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:20.901281 22485 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:20.902189 22485 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.902678 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:20.894321 21069 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:20.894843 22428 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:20.894456 20970 catalog_manager.cc:5697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.903692 22428 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.904177 22428 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:20.904619 20970 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 3, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 3 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.910118 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:20.910941 21072 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:20.916957 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:20.919595 22410 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:20.919581 22502 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:20.920418 22410 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:20.920482 22502 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.921370 22410 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.921947 22502 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.922619 21217 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:20.941058 22502 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.944247 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:20.944895 21069 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.920523 22428 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:20.954861 21069 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:20.956283 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:20.955874 22389 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:06:20.966406 22484 raft_consensus.cc:993] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:20.966879 22475 raft_consensus.cc:1081] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer 5e5e65640c8b4f5ca7eadc6a14312c55 to start an election
I20260709 13:06:20.967078 22457 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:20.968779 22502 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:20.971969 21144 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:20.972110 21142 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:49718
I20260709 13:06:20.972826 21142 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:20.973266 21142 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:20.968546 22457 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:20.975971 20968 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 2 to 3, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 3 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.981418 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:20.984634 21138 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:20.969231 22428 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:20.988481 22465 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:20.984390 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:20.992398 20968 catalog_manager.cc:5697] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:21.002854 20968 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 1 to 2, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:20.999987 21142 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:20.996942 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:21.013494 21138 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:21.017889 21216 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.015924 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:21.020098 21069 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:21.021245 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:21.022343 22469 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:21.023736 21069 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:21.009110 21219 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.024892 22504 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:21.022855 22469 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:21.030541 20970 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:21.028264 21069 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.033607 21069 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.036885 21142 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.025072 21068 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:21.032503 20970 catalog_manager.cc:5697] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:21.038089 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:21.040385 21216 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.047591 22426 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.016121 22457 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.048350 22426 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.056252 21069 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:21.059124 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:21.059937 21213 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:21.061177 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:21.065831 21216 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:21.067150 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:21.057325 22426 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:21.067266 22275 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:21.071354 22504 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:21.085995 22407 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.057443 22469 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.091333 22485 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.083248 22275 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.092420 22275 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:21.092120 22485 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.074430 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:21.095925 22504 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:21.096728 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:21.096911 22504 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.097375 21140 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:21.099771 22407 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.101946 22407 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.104087 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:21.107336 22410 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:21.109661 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:21.109784 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:21.110318 21072 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:21.107860 22410 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.139593 21212 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:21.141216 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:21.141839 21220 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:21.119799 22485 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:21.095953 21797 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:21.135321 22410 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.154309 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:21.162213 20969 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:21.164003 21069 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.173400 22275 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.181941 21068 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:21.181289 20968 catalog_manager.cc:5697] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 1 to 2, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:21.183193 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:21.183828 21215 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:21.189389 22414 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.190279 22399 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:21.190711 22399 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.191061 22399 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.209736 21215 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:21.173156 20967 catalog_manager.cc:5697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 3, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 3 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:21.219317 22275 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:21.220112 21217 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.226542 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:21.227311 21798 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:21.228778 21072 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.192188 21219 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:21.219485 21797 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:21.161816 22407 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:06:21.236657 22469 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.237196 22469 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.239368 22414 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:21.246840 22456 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.247457 22456 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.249531 22456 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.250312 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:21.250999 21068 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:21.251678 21798 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:21.252017 22399 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.253072 21140 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:21.254357 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:21.256191 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:21.256810 21138 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:21.258880 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:21.259760 21220 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:21.257906 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:21.263432 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 61696.1us
I20260709 13:06:21.269438 22485 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.277081 22485 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.282094 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:21.285825 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:21.292860 21138 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:21.284505 22469 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:21.291272 22410 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.301755 22389 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.269294 22503 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:21.270020 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:21.271756 22459 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:21.315253 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:21.317590 21793 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:21.320242 22455 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:21.320572 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:21.316259 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 62297.4us
I20260709 13:06:21.313819 22503 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:21.322638 22503 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.320492 22260 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:21.324929 22260 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:21.308171 22485 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:21.328686 22457 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:21.329381 22457 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.325706 22260 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.313522 22459 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.334841 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:21.311381 21068 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:21.315196 21217 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:21.336210 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:21.357880 21215 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:21.332917 20970 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:21.359083 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:21.335902 22260 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.342602 22399 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.345366 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:21.362675 21220 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.347368 22459 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.348035 22477 raft_consensus.cc:993] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:21.324537 22469 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:21.377218 22465 raft_consensus.cc:1081] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Signalling peer b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:21.377506 22469 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.377948 22469 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:21.379031 21142 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.380820 22485 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:21.352015 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:21.383301 21797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:54648
I20260709 13:06:21.383975 21797 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.384202 22482 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:21.384620 22482 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.384505 21797 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.384974 22482 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.332178 22457 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.360782 22410 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:21.388422 22465 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:21.388860 21066 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.368019 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:21.390929 21140 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.370564 22426 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:21.371651 21138 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:21.341203 21072 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:21.390463 22410 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:06:21.388458 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:21.395370 21068 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.393070 22426 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:21.396095 22426 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.399917 21140 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:21.400825 22465 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:21.415658 21800 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:21.404049 21220 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:21.409381 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:21.400751 22469 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.418574 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:21.419152 22260 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:21.429388 21140 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.425169 22485 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:21.426312 22414 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:21.447674 22414 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:21.460637 22482 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.468390 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:21.469301 21138 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.454670 22459 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.497915 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:21.498569 21140 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:21.435098 22389 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:06:21.499735 21220 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.485181 22482 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:21.520802 22260 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.523829 22260 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.525866 22260 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.526973 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:21.448697 22414 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.529960 21144 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:21.533556 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:21.540987 22260 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:21.541448 22260 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.541762 22260 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.547187 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:21.529277 22469 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:21.545223 21138 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:21.511052 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:21.567682 21066 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:21.511471 21068 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:21.568063 21138 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:21.570181 22410 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:21.570946 22410 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.571832 21140 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:21.542431 21797 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.575273 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:21.511050 22477 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:06:21.540539 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:21.581745 21069 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:21.583457 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:21.509541 22426 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.584900 22260 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.598416 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:21.600543 21140 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.604404 22260 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.605727 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:21.611356 21140 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:21.593619 21065 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.614384 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:21.595429 22428 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:21.617511 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:21.616583 21142 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.623848 22463 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:21.618016 21217 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:21.624424 22463 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:21.593838 22433 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:06:21.625272 22463 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:21.635485 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52202.7us
W20260709 13:06:21.616541 22539 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:06:21.627000 22459 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.643482 21138 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:21.593509 22476 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:06:21.682137 21142 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:21.684120 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:21.636956 22428 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:21.643812 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:21.709486 21797 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:21.687489 22548 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:21.717167 22538 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:06:21.639834 22275 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 13:06:21.652123 22540 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.698087 22475 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:21.691511 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:21.736966 22482 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:21.746803 21219 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:21.780499 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:21.781229 21144 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:21.746496 22475 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:06:21.761731 22534 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:06:21.744154 22410 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:21.739629 22548 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:21.758649 21064 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:21.839838 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:21.776155 21214 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:21.742933 22540 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.812937 22482 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.878238 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:21.766965 22428 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.876425 21066 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:21.878927 22426 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.908221 21792 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:21.886475 22433 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:21.923309 22433 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.925434 22433 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:21.926982 22446 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:21.928267 22504 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.928783 22504 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:21.914168 22545 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:06:21.922060 22548 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:21.890283 22475 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.935076 22540 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:21.939909 20969 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:21.975227 22540 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:21.976001 22540 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:21.975416 22433 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:21.978053 22433 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:21.992820 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 239114us
I20260709 13:06:21.994974 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:21.996121 21066 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:22.004375 22465 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.982940 22548 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.005146 22465 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.005745 22548 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:21.969539 22423 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.018381 21184 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:22.024138 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:21.921974 21065 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
W20260709 13:06:22.030212 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 309618us
I20260709 13:06:22.026753 22375 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.028425 22459 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.056547 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:22.036212 22485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.070s	sys 0.127s
I20260709 13:06:22.076018 22502 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:22.085595 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.087986 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:22.037889 22414 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.088831 21069 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:22.037067 22410 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:22.038506 22457 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:22.038231 22554 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.096113 21066 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:22.053488 21797 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:22.032657 22423 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.106333 22260 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:22.071305 22485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.070s	sys 0.127s
I20260709 13:06:22.107745 22260 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.109935 21185 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:22.102456 22433 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.114347 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.098572 21219 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:22.115345 22399 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:22.116680 21138 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:22.134832 22551 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:22.135984 22558 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:22.136693 22551 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:22.136588 22558 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.137245 22551 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:22.102480 22554 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.138119 22551 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.104470 22465 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.069593 22375 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.103971 22414 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.140898 22375 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:22.141203 22414 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.107442 21064 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:22.071504 22459 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.110623 22485 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:22.113722 22547 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:22.130460 22540 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:22.181985 22446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:39484 (request call id 81) took 1014 ms. Trace:
W20260709 13:06:22.182065 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 83617.6us
I20260709 13:06:22.205727 21144 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:22.207844 22260 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.170636 22562 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.148310 22504 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.214865 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:22.107348 22410 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.148106 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:22.183391 22399 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.111441 22457 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.257149 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:22.171597 22414 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:22.183379 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.169059 22482 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.185222 22558 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.181324 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:22.219450 22423 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.187644 22548 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.220441 22410 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.220244 22485 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.220937 22433 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:22.222218 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.222522 22566 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.222919 22546 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:22.204695 22550 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:06:22.223639 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.210811 22554 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.211462 22540 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:22.212250 22446 rpcz_store.cc:276] 0709 13:06:21.193648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.198622 (+  4974us) service_pool.cc:224] Handling call
0709 13:06:21.253406 (+ 54784us) log.cc:844] Serialized 508 byte log entry
0709 13:06:22.181844 (+928438us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7178,"raft.queue_time_us":1866,"raft.run_cpu_time_us":1332,"raft.run_wall_time_us":31181,"thread_start_us":21203,"threads_started":1,"wal-append.queue_time_us":76782}
I20260709 13:06:22.204661 22547 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:22.225432 22559 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:06:22.229984 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:22.321185 21070 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:22.157047 20968 catalog_manager.cc:5697] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.237097 21212 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
W20260709 13:06:22.239367 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 245350us
I20260709 13:06:22.266569 22407 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:06:22.326846 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:22.327593 21071 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
W20260709 13:06:22.261130 22539 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:06:22.320811 22540 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.216205 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:22.261168 22533 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:06:22.271548 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.330703 20957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cc53d52fdeaa441b9e9d329e69d94440 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:22.331462 22540 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.297998 21146 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 2: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:22.299437 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.262115 22414 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.305070 21145 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:22.299121 21066 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:22.302543 21139 raft_consensus.cc:2955] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:22.258100 21067 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:22.333753 21144 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:22.333912 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.297315 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:22.262641 22399 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.338371 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.342237 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.342821 21144 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:22.344233 22547 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.338405 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.310595 22433 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:22.370814 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.371521 21215 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:22.371637 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.372404 21212 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:22.372725 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.373335 21216 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:22.309693 22423 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:06:22.217445 22562 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:22.337406 22414 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:22.377035 21141 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:22.341463 22463 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.341768 22446 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.405592 22469 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.408743 22446 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.305688 22566 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.411942 21065 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:22.369755 21064 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:22.377198 22547 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:22.309763 22546 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.423710 21220 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:22.404352 22463 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.419684 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:22.429746 22457 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.431816 22562 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.418524 22459 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.433588 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:22.411656 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44439.5us
I20260709 13:06:22.421240 22446 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.436167 21069 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:22.427002 22547 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.417471 22469 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.433722 22463 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.441499 22433 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.450886 22566 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:22.433364 22546 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.457445 22399 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:22.459458 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:22.460214 21800 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:22.447809 22547 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:22.462272 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:22.463271 22504 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.463663 22504 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.464026 22504 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.465863 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:22.466639 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 122243us
I20260709 13:06:22.447492 22423 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.471851 22423 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.473819 22504 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.474226 22423 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.477221 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.482539 22561 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.482797 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.483554 22476 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:22.454128 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 480073us
I20260709 13:06:22.488502 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:22.500490 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.490672 21221 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:22.514253 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.515327 21142 raft_consensus.cc:2368] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:22.516736 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.517480 21139 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:22.518558 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.519336 21145 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:22.521724 21221 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:22.522982 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.523705 21221 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:22.524682 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.525365 21221 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:22.526443 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.529032 22469 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.529654 22469 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:22.532168 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.535827 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.536775 21145 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 2: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:22.498909 22561 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.543678 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.548264 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.545908 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:22.551600 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.553248 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.554198 21139 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 2: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:22.552224 21218 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:22.556483 21212 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:22.537873 22544 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.545099 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.530997 22555 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:22.544278 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.531507 22504 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:22.544832 21797 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:22.550689 22561 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.520774 22476 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.559862 22549 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:22.563534 21221 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:22.565303 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:22.530439 21214 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:22.541006 22477 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:22.554942 22565 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.529050 22414 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.574796 21185 leader_election.cc:400] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:22.704568 21185 leader_election.cc:403] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:22.705289 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 132128us
I20260709 13:06:22.543124 22459 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.567761 22555 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:22.563089 22544 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.540637 22469 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.630582 22457 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:22.587570 22549 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:22.571735 22503 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 13:06:22.640251 22578 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:06:22.682865 21216 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:22.714380 21145 raft_consensus.cc:2368] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:22.632159 22477 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:22.629709 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 64487.9us
I20260709 13:06:22.699393 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:22.702167 21140 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:22.706248 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:22.717360 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:22.706941 22428 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.707711 22459 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.707201 22399 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.709954 22504 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.709887 22482 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.708690 22555 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:22.710388 22544 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.722038 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:22.722559 21799 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:22.711269 22457 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.744699 22477 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:22.724572 22459 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.725096 22485 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.777521 22485 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.777983 22485 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:22.778416 22568 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.439s	user 0.008s	sys 0.006s
I20260709 13:06:22.784650 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.785619 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.786938 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:22.747421 22457 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:22.734061 22504 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.788010 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.747788 22426 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.733371 22551 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.726218 22399 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:22.711365 22565 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:22.767604 22583 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:06:22.794559 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:22.727365 22428 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.796855 22537 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:22.817418 22564 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:22.799520 22407 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.797895 22426 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.805794 22410 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:22.799214 22540 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:22.775331 22423 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:06:22.800889 22551 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:22.829557 21141 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:22.820592 21142 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:22.751643 22570 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:06:22.825417 22537 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:22.826252 22407 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.832291 22426 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.804589 22572 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:22.830664 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2096) took 1192 ms. Trace:
W20260709 13:06:22.867908 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 118466us
I20260709 13:06:22.865198 22485 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.844502 22551 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.831935 22564 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.827984 22547 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.836195 22410 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:22.855497 22537 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:22.835587 22540 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:22.854292 22407 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.894420 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:22.885252 22410 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:22.895149 21144 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.897008 21215 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:22.881891 22564 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:22.874689 22414 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.804749 22549 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.852283 22561 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.895431 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.884277 22469 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.871338 22572 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.912542 22428 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.913159 22428 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:22.913652 22428 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.889663 22540 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.910403 22571 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:22.867177 21217 rpcz_store.cc:276] 0709 13:06:21.629872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.636011 (+  6139us) service_pool.cc:224] Handling call
0709 13:06:21.669361 (+ 33350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.095613 (+426252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.099406 (+  3793us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.102539 (+  3133us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.453042 (+350503us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:22.456195 (+  3153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.493702 (+ 37507us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.803137 (+309435us) raft_consensus.cc:1685] finished
0709 13:06:22.804457 (+  1320us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.822466 (+ 18009us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102888,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":141,"thread_start_us":740333,"threads_started":2,"wal-append.queue_time_us":455321}
I20260709 13:06:22.901098 22475 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:22.904071 22399 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:22.898835 22485 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.940728 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:22.940901 21144 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:22.942566 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:22.902597 22594 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:06:22.951213 22428 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:22.915149 22572 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:22.925539 22565 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.932881 22482 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:22.953882 22546 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:22.969130 21144 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:22.959601 22571 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.947880 22475 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:22.987442 22389 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.988000 22389 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.990541 22389 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:22.973804 22456 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.996876 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:22.997903 21067 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:22.975126 22546 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:22.980242 22572 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:06:23.021637 22504 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:23.007928 22456 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.030763 22456 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:23.030352 22407 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:22.985692 22410 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:06:23.036147 22586 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.149s	user 0.001s	sys 0.001s
I20260709 13:06:23.031062 22476 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:23.031692 22463 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.003s
I20260709 13:06:23.043742 21072 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:06:23.055184 21072 pending_rounds.cc:85] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 1
I20260709 13:06:23.045027 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.056005 21069 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:23.057004 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.046991 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:23.040500 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 229002us
I20260709 13:06:23.058586 21071 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:23.053648 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:23.067068 22582 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.067699 22582 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:23.070751 22582 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:23.072314 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:23.073099 21799 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:23.055552 21072 pending_rounds.cc:107] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:06:23.074568 21072 raft_consensus.cc:2889] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:23.096722 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 360819us
W20260709 13:06:23.082494 22446 consensus_meta.cc:287] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.634s	user 0.011s	sys 0.001s
I20260709 13:06:23.101732 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:23.102464 21141 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:23.075799 22561 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:23.105242 21147 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:23.063975 22457 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.066399 22463 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.094395 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.111410 21113 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:23.117019 21113 leader_election.cc:403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:23.077183 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:23.104228 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:23.087076 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:23.110211 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:23.104975 22597 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.108557 22582 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:23.066584 22546 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.105261 22446 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.127575 21066 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:23.147847 21071 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:23.128273 21068 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:23.129367 21217 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:23.153156 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.135074 22582 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.170377 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:23.204116 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:23.204990 21801 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:23.206130 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:23.151132 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:23.152916 22407 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:23.154243 22475 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:23.168031 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.169041 22410 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:23.155256 22549 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:23.174157 22463 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:23.227586 21063 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:23.227795 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:23.230782 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:23.234241 20968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 221) took 1192 ms. Trace:
I20260709 13:06:23.234313 22564 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.238489 22389 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:23.239509 22457 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:23.240166 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:23.240618 22446 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:23.240850 22540 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.166921 22605 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:23.251688 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:23.131503 22597 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:23.248402 22555 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.508s	user 0.000s	sys 0.011s
I20260709 13:06:23.199867 22426 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.263741 22389 consensus_queue.cc:1243] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:23.251447 22601 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:23.259537 20968 rpcz_store.cc:276] 0709 13:06:22.042030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.042207 (+   177us) service_pool.cc:224] Handling call
0709 13:06:23.234168 (+1191961us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:22.325685 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:22.326246 (+   561us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:22.326275 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:06:22.326291 (+    16us) tablet.cc:662] Decoding operations
0709 13:06:22.326609 (+   318us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:22.326649 (+    40us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:22.326673 (+    24us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:22.326856 (+   183us) tablet.cc:701] Row locks acquired
0709 13:06:22.326874 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:06:22.327009 (+   135us) write_op.cc:270] Start()
0709 13:06:22.327114 (+   105us) write_op.cc:276] Timestamp: P: 1783602382326963 usec, L: 0
0709 13:06:22.327153 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:06:22.327816 (+   663us) log.cc:844] Serialized 544 byte log entry
0709 13:06:22.766307 (+438491us) op_driver.cc:464] REPLICATION: finished
0709 13:06:22.976577 (+210270us) write_op.cc:301] APPLY: starting
0709 13:06:22.977353 (+   776us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:22.985891 (+  8538us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:23.009258 (+ 23367us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:23.024070 (+ 14812us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:23.040319 (+ 16249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:06:23.054699 (+ 14380us) write_op.cc:312] APPLY: finished
0709 13:06:23.073809 (+ 19110us) log.cc:844] Serialized 23 byte log entry
0709 13:06:23.093844 (+ 20035us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:23.099131 (+  5287us) write_op.cc:454] Released schema lock
0709 13:06:23.103719 (+  4588us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:23.122194 (+ 18475us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:23.124313 (+  2119us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":197569,"apply.run_cpu_time_us":2556,"apply.run_wall_time_us":138664,"num_ops":1,"prepare.queue_time_us":655,"prepare.run_cpu_time_us":2386,"prepare.run_wall_time_us":2387,"replication_time_us":421534,"thread_start_us":273722,"threads_started":1}]]}
I20260709 13:06:23.282176 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.290807 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.342160 21072 raft_consensus.cc:2368] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
W20260709 13:06:23.336200 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 230733us
I20260709 13:06:23.346298 21071 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:23.341284 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.348773 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.349247 21067 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 2: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:23.265842 21796 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:23.285241 22582 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:23.289633 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.277520 22602 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:23.290278 22547 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:23.359792 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.361420 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:23.363031 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.288178 22537 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:23.280922 22540 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:23.363930 21072 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:23.363677 21800 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:23.344730 22551 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.364609 22579 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:06:23.274183 21214 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:23.382083 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.360411 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:23.341287 22410 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:23.365356 22602 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:23.329972 22555 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:23.285360 22459 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:23.391973 21216 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:23.382875 21067 raft_consensus.cc:2368] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:23.394225 22457 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:23.346319 22605 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:23.395609 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.398506 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.398715 22457 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:23.400799 22584 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.698s	user 0.004s	sys 0.000s
I20260709 13:06:23.403424 22426 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:23.404356 21070 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:23.406189 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:23.406960 21067 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:23.392916 21066 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:23.391037 22602 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:23.402261 22544 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.661s	user 0.011s	sys 0.001s
I20260709 13:06:23.398815 22605 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:23.425163 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 204821us
I20260709 13:06:23.391463 22410 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:23.409682 21797 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:23.352197 22601 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:23.439854 21035 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:23.444100 22375 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:23.435655 22540 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.453644 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:23.437849 22605 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:23.522960 21147 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:23.525317 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:23.526142 21140 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:23.447746 22551 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:23.451795 22582 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:23.446715 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:23.450740 22375 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.435308 22544 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:23.450562 22457 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:23.844554 22504 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:23.990144 21766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.002s	sys 0.009s
I20260709 13:06:23.813632 22561 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:23.873229 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2233) took 1170 ms. Trace:
I20260709 13:06:23.851619 22601 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.486487 22561 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:23.724658 22537 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:24.466478 22543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.021s	sys 0.049s
W20260709 13:06:23.966387 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.007s	sys 0.004s
I20260709 13:06:24.486896 21142 rpcz_store.cc:276] 0709 13:06:22.698777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.698987 (+   210us) service_pool.cc:224] Handling call
0709 13:06:23.868379 (+1169392us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b540036c754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:23.868551 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233604992}
W20260709 13:06:24.488000 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.011s	sys 0.010s
W20260709 13:06:24.487859 22543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.233s	user 0.022s	sys 0.050s
W20260709 13:06:24.488366 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 1.0758e+06us
I20260709 13:06:24.488814 22561 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.468880 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 489) took 1309 ms. Trace:
I20260709 13:06:24.489362 22561 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.490304 21063 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:24.059469 22579 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:24.491466 22375 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.489614 21795 rpcz_store.cc:276] 0709 13:06:23.159296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.169536 (+ 10240us) service_pool.cc:224] Handling call
0709 13:06:24.467816 (+1298280us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:24.468062 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.468333 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:24.468376 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.468443 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:24.468492 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.468581 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.468760 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15264640}
W20260709 13:06:23.875872 22546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.046s	sys 0.092s
I20260709 13:06:23.869762 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:24.522053 21139 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:23.951679 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:23.770560 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:24.493137 22582 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.855908 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:24.549160 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:24.549909 21141 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:23.923879 22562 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.191021 22455 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:24.493287 21184 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.032s	sys 0.027s
W20260709 13:06:24.553496 21184 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.032s	sys 0.027s
I20260709 13:06:23.931634 21068 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:06:24.494302 21799 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.008s	sys 0.040s
I20260709 13:06:23.772257 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:23.989373 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:23.808784 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:23.784397 21185 leader_election.cc:400] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:23.531338 22547 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:24.281702 21145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.000s	sys 0.017s
I20260709 13:06:23.816402 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:24.562383 21145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.000s	sys 0.021s
I20260709 13:06:24.562652 21069 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:23.990002 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.003s	sys 0.008s
I20260709 13:06:23.520865 22564 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:24.332350 22618 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.446s	user 0.000s	sys 0.002s
I20260709 13:06:23.907472 21067 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:24.498088 21066 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:23.820676 22446 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:23.593883 22597 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:24.318341 21110 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.761s (SENT) [suppressed 7 similar messages]
W20260709 13:06:24.492982 21797 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.040s	user 0.020s	sys 0.010s
I20260709 13:06:23.618343 21147 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:24.556648 21068 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:24.559319 22551 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.501422 22637 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:23.532166 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:23.519789 22540 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.489869 22561 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.504010 22544 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:24.506659 22375 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.486212 22504 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:24.486129 21766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.062s	user 0.007s	sys 0.014s
I20260709 13:06:23.529775 22605 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:23.521077 22594 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.800880 22410 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:24.008814 21138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.008s	sys 0.009s
W20260709 13:06:24.647652 21138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.011s	sys 0.010s
W20260709 13:06:23.918841 22477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.019s	sys 0.018s
W20260709 13:06:23.753178 22617 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
I20260709 13:06:24.285795 22275 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:23.792074 22615 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:06:23.786017 22555 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:24.633466 22457 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:24.655900 22477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.392s	user 0.021s	sys 0.019s
I20260709 13:06:23.873209 22613 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:24.274715 22616 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:23.753526 22575 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:24.520562 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.004s	sys 0.003s
W20260709 13:06:24.688694 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.004s	sys 0.003s
W20260709 13:06:24.689186 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 875343us
W20260709 13:06:24.355566 22572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.000s	sys 0.015s
I20260709 13:06:24.523593 21143 raft_consensus.cc:2368] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:24.524039 21212 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
W20260709 13:06:24.388557 21186 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.005s	sys 0.009s
W20260709 13:06:24.690577 21186 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.005s	sys 0.009s
I20260709 13:06:24.556975 21072 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:24.622360 21068 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.568471 22547 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:24.559356 21799 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.009s	sys 0.040s
I20260709 13:06:24.638693 22375 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.492038 22579 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.556744 22562 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.553498 22582 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:24.550096 22601 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.568602 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 378) took 1176 ms. Trace:
W20260709 13:06:24.566129 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.131s	user 0.006s	sys 0.016s
I20260709 13:06:23.651865 22459 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:23.839975 22620 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.557188 22626 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.555699 22628 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.711736 21067 rpcz_store.cc:276] 0709 13:06:23.392535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.393949 (+  1414us) service_pool.cc:224] Handling call
0709 13:06:24.568516 (+1174567us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:24.582975 21069 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:24.624984 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2250) took 1094 ms. Trace:
I20260709 13:06:24.737401 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 380) took 1344 ms. Trace:
I20260709 13:06:24.556809 22455 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.563907 22537 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.575506 21797 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:24.659626 22594 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.575804 22564 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.640008 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:24.570858 22632 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.619067 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 376) took 1187 ms. Trace:
I20260709 13:06:24.563382 22627 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:06:24.643030 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:24.584605 22600 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.645596 21071 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:24.710832 22641 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.583385 22570 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.660271 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2243) took 1241 ms. Trace:
I20260709 13:06:24.628708 22544 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.575999 22446 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:24.647436 21215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 388) took 2123 ms (client timeout 1829 ms). Trace:
I20260709 13:06:24.623610 22571 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:23.616186 22260 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:23.919114 22418 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.018s	sys 0.019s
I20260709 13:06:24.684716 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:24.774688 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 90102.5us
I20260709 13:06:24.623304 22551 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.559258 21185 leader_election.cc:403] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:24.665445 22410 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.662827 22543 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.746630 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2119) took 1545 ms. Trace:
I20260709 13:06:24.696596 21799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 486) took 1538 ms. Trace:
W20260709 13:06:24.548628 22546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.280s	user 0.064s	sys 0.124s
I20260709 13:06:24.626540 22637 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.700091 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2248) took 1179 ms. Trace:
I20260709 13:06:24.702323 22547 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:24.672546 22485 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:06:24.692416 22572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.001s	sys 0.015s
I20260709 13:06:24.690750 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:24.672667 22275 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.691828 22555 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:24.696395 22428 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.698101 22631 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.567821 22633 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.737754 21069 rpcz_store.cc:276] 0709 13:06:23.393061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.816107 (+423046us) service_pool.cc:224] Handling call
0709 13:06:24.737344 (+921237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:24.737584 21140 rpcz_store.cc:276] 0709 13:06:23.530397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.531813 (+  1416us) service_pool.cc:224] Handling call
0709 13:06:24.624884 (+1093071us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:24.687166 22457 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.744304 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:24.746363 21220 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:24.708858 22643 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:24.749718 21066 rpcz_store.cc:276] 0709 13:06:23.391909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.414990 (+ 23081us) service_pool.cc:224] Handling call
0709 13:06:24.579640 (+1164650us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:24.764420 22644 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.764456 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:24.765187 22638 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:25.607555 21216 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:25.607474 22644 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:25.607998 22638 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:25.608667 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:25.609724 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:25.609885 22644 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.698634 22616 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.767648 22260 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:24.772161 22625 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:06:24.772989 22455 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:25.612118 22260 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:25.612542 22616 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:24.760969 21138 rpcz_store.cc:276] 0709 13:06:23.406904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.407364 (+   460us) service_pool.cc:224] Handling call
0709 13:06:23.407769 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:23.408177 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.408213 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:23.415505 (+  7292us) log.cc:844] Serialized 26 byte log entry
0709 13:06:24.648058 (+1232553us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:24.648093 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.648164 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:24.648212 (+    48us) raft_consensus.cc:1685] finished
0709 13:06:24.648223 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.648410 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2704,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":33899,"thread_start_us":1195892,"threads_started":1,"wal-append.queue_time_us":1084379}
I20260709 13:06:24.775418 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:24.767367 21215 rpcz_store.cc:269] 0709 13:06:22.518745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.526152 (+  7407us) service_pool.cc:224] Handling call
0709 13:06:24.642352 (+2116200us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54003b4cd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:24.642505 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850142464}
I20260709 13:06:24.736515 22601 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:25.613869 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 838497us
I20260709 13:06:25.614243 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:25.614571 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:25.614971 21215 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:25.615193 22601 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:25.615768 22601 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:25.615873 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:25.616139 22601 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.808145 22646 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:25.617863 22601 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:25.617928 22646 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:25.618409 22646 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:24.735810 22620 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:25.622155 22620 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:24.800066 22418 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.478s	user 0.020s	sys 0.021s
I20260709 13:06:24.746289 22537 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.625065 22537 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:24.813985 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 495661us
I20260709 13:06:25.625773 22601 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.628072 22601 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.763473 22433 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.750053 22564 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:25.628804 22433 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.629336 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:25.630107 22564 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.812254 21145 rpcz_store.cc:276] 0709 13:06:23.520985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.521147 (+   162us) service_pool.cc:224] Handling call
0709 13:06:23.521441 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.521656 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:23.615437 (+ 93781us) log.cc:844] Serialized 14 byte log entry
0709 13:06:24.699702 (+1084265us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.699754 (+    52us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:24.699796 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.699867 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.700023 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":93940,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":20273,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3274,"thread_start_us":1083812,"threads_started":2,"wal-append.queue_time_us":884491}
I20260709 13:06:24.805799 21799 rpcz_store.cc:276] 0709 13:06:23.158302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.161156 (+  2854us) service_pool.cc:224] Handling call
0709 13:06:23.161572 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.161828 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:23.453008 (+291180us) log.cc:844] Serialized 14 byte log entry
0709 13:06:24.696107 (+1243099us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.696183 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:24.696244 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.696356 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.696521 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4769,"prepare.queue_time_us":188231,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":3641,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":10010,"thread_start_us":1476265,"threads_started":2,"wal-append.queue_time_us":1041320}
I20260709 13:06:24.767066 22641 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:25.630936 22433 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:24.900775 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 1.11649e+06us
I20260709 13:06:24.805413 22571 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.719684 22575 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.773480 22544 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:24.908852 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 164608us
I20260709 13:06:24.851689 22561 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.747030 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:24.711730 22613 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.710261 21068 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:24.852828 21072 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:25.002094 22659 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:25.015627 22660 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:06:25.825552 21072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 381) took 2200 ms (client timeout 1726 ms). Trace:
I20260709 13:06:25.825855 22660 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:25.825716 22659 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:25.825989 21072 rpcz_store.cc:269] 0709 13:06:23.624848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.771890 (+147042us) service_pool.cc:224] Handling call
0709 13:06:25.825474 (+2053584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.827044 22660 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:25.657581 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:25.828444 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1389) took 1150 ms. Trace:
I20260709 13:06:25.828776 21067 rpcz_store.cc:276] 0709 13:06:24.678590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.657337 (+978747us) service_pool.cc:224] Handling call
0709 13:06:25.828376 (+171039us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.829041 22659 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:25.829715 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:25.830493 21067 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:25.663357 22662 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:25.830897 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1388) took 1158 ms. Trace:
I20260709 13:06:25.831236 21067 rpcz_store.cc:276] 0709 13:06:24.672233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.829470 (+1157237us) service_pool.cc:224] Handling call
0709 13:06:25.830830 (+  1360us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.831032 22662 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.832589 21067 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:25.833200 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 98) took 2782 ms. Trace:
I20260709 13:06:25.833549 22662 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:25.833487 21067 rpcz_store.cc:276] 0709 13:06:23.050842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.831988 (+2781146us) service_pool.cc:224] Handling call
0709 13:06:25.832290 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.832965 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.833154 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:25.637017 21035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.926s	user 0.023s	sys 0.058s
W20260709 13:06:25.834985 21035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.024s	sys 0.059s
I20260709 13:06:25.674575 20969 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 238) took 4060 ms. Trace:
I20260709 13:06:25.835211 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 100) took 2783 ms. Trace:
W20260709 13:06:25.682564 21070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 373) took 2633 ms (client timeout 1539 ms). Trace:
I20260709 13:06:25.693130 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:25.568723 21111 connection.cc:256] RPC call timeout handler was delayed by -1.41866638099998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
W20260709 13:06:26.073060 21111 connection.cc:256] RPC call timeout handler was delayed by -1.0782297189999781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
I20260709 13:06:25.658450 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:25.697580 22665 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.076649 22664 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.076375 22665 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.077240 22664 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.078900 22565 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.079350 22665 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:25.636449 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:26.077198 22669 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.081146 21138 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:26.081471 22669 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.082302 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:25.637434 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:25.664315 21142 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:26.087056 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:26.087445 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:26.088217 22663 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:25.641855 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:24.963718 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1487) took 1983 ms. Trace:
I20260709 13:06:25.646206 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:26.092873 21146 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:26.093000 22668 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:26.093824 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:25.645232 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:25.016896 21187 leader_election.cc:400] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:26.109676 21187 leader_election.cc:403] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:26.110606 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 1.09385e+06us
W20260709 13:06:25.003835 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 91893.9us
I20260709 13:06:24.694620 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2118) took 1526 ms. Trace:
I20260709 13:06:26.116340 21212 rpcz_store.cc:276] 0709 13:06:23.163923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.165265 (+  1342us) service_pool.cc:224] Handling call
0709 13:06:23.770347 (+605082us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:24.690340 (+919993us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33805056}
I20260709 13:06:26.117990 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:26.118781 21212 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:25.622308 21214 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:26.120584 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2122) took 1463 ms. Trace:
I20260709 13:06:26.121073 21214 rpcz_store.cc:276] 0709 13:06:24.657442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.661510 (+  4068us) service_pool.cc:224] Handling call
0709 13:06:25.621667 (+960157us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:25.621920 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.120184 (+498264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.120503 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3649280}
I20260709 13:06:25.617224 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:26.124526 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 410) took 1383 ms. Trace:
I20260709 13:06:26.124964 21215 rpcz_store.cc:276] 0709 13:06:24.740905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.616761 (+875856us) service_pool.cc:224] Handling call
0709 13:06:26.124436 (+507675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.126142 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:26.126875 21215 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:25.639546 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:26.128286 21220 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:24.903683 21219 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:24.807217 21217 rpcz_store.cc:276] 0709 13:06:23.200858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.641867 (+1441009us) service_pool.cc:224] Handling call
0709 13:06:24.674691 (+ 32824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.694966 (+ 20275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:24.697451 (+  2485us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.699501 (+  2050us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:24.710714 (+ 11213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.720053 (+  9339us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.745683 (+ 25630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.134496 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:24.981381 21792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 118) took 1922 ms. Trace:
I20260709 13:06:26.136369 21792 rpcz_store.cc:276] 0709 13:06:23.059411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.071971 (+ 12560us) service_pool.cc:224] Handling call
0709 13:06:24.916136 (+1844165us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:24.958412 (+ 42276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.977198 (+ 18786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:24.978588 (+  1390us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.980852 (+  2264us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:24.981022 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.981129 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.981298 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7997824}
I20260709 13:06:26.138616 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:25.664565 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:25.009498 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:25.647102 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:25.654585 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:26.142769 21801 raft_consensus.cc:2376] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3. Re-sending same reply.
I20260709 13:06:26.143832 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:25.634250 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:25.011534 21797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 117) took 1952 ms (client timeout 1761 ms). Trace:
I20260709 13:06:24.919763 22375 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:25.567111 22407 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:26.150408 22407 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.994038 22275 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.809105 22410 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:25.631486 22418 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.157541 22418 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.908290 22428 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.660939 22433 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:26.164383 22433 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.164901 22433 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:25.001363 22446 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:25.005421 22457 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.895224 22485 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.897574 22540 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:24.893391 22543 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.660305 22544 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.177033 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:24.897383 22546 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:26.178903 22546 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.901820 22547 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:24.987998 22562 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.982160 22551 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:24.906447 22555 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.805354 22570 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.185341 21796 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:25.010121 22582 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.190271 22582 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:25.616775 22591 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:26.191857 22591 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.192243 22591 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:24.953089 22594 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:25.633733 22423 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:24.803393 22600 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.195497 22423 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:24.985422 22602 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.541s	user 0.011s	sys 0.004s
I20260709 13:06:24.907923 22605 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:25.660105 22613 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:24.740283 22626 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.917559 22631 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.759531 22632 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.912994 22633 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.984081 22637 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:25.610488 22638 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:25.633538 22641 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:24.914602 22643 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.661353 22644 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:26.216323 22644 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.667027 22646 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.222026 22646 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:24.919699 22648 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.223827 22648 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.982151 22649 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:26.226871 22649 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:24.852083 22651 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:26.229997 22651 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.015103 22476 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.232267 22476 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:24.908267 22504 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:25.662727 22537 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.646381 22564 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.235831 22537 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.236289 22564 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.659220 22571 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:25.658239 22575 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:24.744805 22628 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.244741 21147 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:25.008917 22657 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:06:25.758379 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 371) took 2710 ms. Trace:
I20260709 13:06:25.768193 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:26.247867 21139 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:25.785563 22579 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:25.798264 21071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 374) took 2742 ms (client timeout 3370 ms). Trace:
W20260709 13:06:26.249541 21071 rpcz_store.cc:269] 0709 13:06:23.055477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.808441 (+752964us) service_pool.cc:224] Handling call
0709 13:06:25.798025 (+1989584us) spinlock_profiling.cc:243] Waited 86.1 ms on lock 0x7b54000fb254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:25.798182 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198110080}
I20260709 13:06:26.251955 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 102) took 3198 ms. Trace:
I20260709 13:06:26.252450 21071 rpcz_store.cc:276] 0709 13:06:23.053868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.250793 (+3196925us) service_pool.cc:224] Handling call
0709 13:06:26.251219 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.251462 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.251501 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.251559 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:26.251606 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.251677 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.251823 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.198839 22602 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:25.800398 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 104) took 2704 ms (client timeout 2850 ms). Trace:
I20260709 13:06:26.255136 21213 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
W20260709 13:06:26.255069 21063 rpcz_store.cc:269] 0709 13:06:23.095727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.398084 (+302357us) service_pool.cc:224] Handling call
0709 13:06:25.800134 (+2402050us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540011bcd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:25.800323 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2508645120}
I20260709 13:06:25.610526 21066 raft_consensus.cc:2368] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
W20260709 13:06:26.256896 21066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1393) took 1549 ms (client timeout 1843 ms). Trace:
I20260709 13:06:26.256994 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1384) took 3183 ms. Trace:
W20260709 13:06:26.256562 21213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2117) took 3093 ms (client timeout 1548 ms). Trace:
I20260709 13:06:25.836483 21065 raft_consensus.cc:2376] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3. Re-sending same reply.
I20260709 13:06:26.256541 22602 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.247534 21064 rpcz_store.cc:276] 0709 13:06:23.047992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.050054 (+  2062us) service_pool.cc:224] Handling call
0709 13:06:25.757755 (+2707701us) spinlock_profiling.cc:243] Waited 8.43 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:25.757918 (+   163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.758078 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:25.758096 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:25.758130 (+    34us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:25.758153 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.758193 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.758294 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19383040}
I20260709 13:06:25.836974 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:26.258921 21072 raft_consensus.cc:2368] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
W20260709 13:06:25.837846 21070 rpcz_store.cc:269] 0709 13:06:23.048775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.056739 (+  7964us) service_pool.cc:224] Handling call
0709 13:06:25.682221 (+2625482us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540019d2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:25.682422 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796689152}
W20260709 13:06:26.257629 21213 rpcz_store.cc:269] 0709 13:06:23.163574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.169100 (+  5526us) service_pool.cc:224] Handling call
0709 13:06:23.957672 (+788572us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:26.256390 (+2298718us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54004434d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:26.256498 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3937548416}
I20260709 13:06:26.259469 21064 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:25.835431 20969 rpcz_store.cc:276] 0709 13:06:21.636126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.677329 (+ 41203us) service_pool.cc:224] Handling call
0709 13:06:25.674476 (+3997147us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:22.219113 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:22.219738 (+   625us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:22.219769 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:06:22.219785 (+    16us) tablet.cc:662] Decoding operations
0709 13:06:22.220133 (+   348us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:22.220179 (+    46us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:22.220206 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:22.220376 (+   170us) tablet.cc:701] Row locks acquired
0709 13:06:22.220393 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:06:22.220525 (+   132us) write_op.cc:270] Start()
0709 13:06:22.220653 (+   128us) write_op.cc:276] Timestamp: P: 1783602382220483 usec, L: 0
0709 13:06:22.220674 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:06:22.221307 (+   633us) log.cc:844] Serialized 539 byte log entry
0709 13:06:22.482975 (+261668us) op_driver.cc:464] REPLICATION: finished
0709 13:06:22.841469 (+358494us) write_op.cc:301] APPLY: starting
0709 13:06:22.844836 (+  3367us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:22.867372 (+ 22536us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:22.885488 (+ 18116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:22.902572 (+ 17084us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:22.902732 (+   160us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:06:22.902774 (+    42us) write_op.cc:312] APPLY: finished
0709 13:06:22.909632 (+  6858us) log.cc:844] Serialized 23 byte log entry
0709 13:06:22.927398 (+ 17766us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:22.929786 (+  2388us) write_op.cc:454] Released schema lock
0709 13:06:22.933162 (+  3376us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:22.963929 (+ 30767us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:22.965234 (+  1305us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":210694,"reported_tablets":4,"spinlock_wait_cycles":8576,"child_traces":[["op",{"apply.queue_time_us":328933,"apply.run_cpu_time_us":2442,"apply.run_wall_time_us":121243,"num_ops":1,"prepare.queue_time_us":246759,"prepare.run_cpu_time_us":2466,"prepare.run_wall_time_us":2466,"replication_time_us":262131,"thread_start_us":375027,"threads_started":2}]]}
I20260709 13:06:26.260036 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:25.847384 21068 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:26.249562 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 121) took 1698 ms. Trace:
W20260709 13:06:26.260910 21068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 99) took 3209 ms (client timeout 2046 ms). Trace:
I20260709 13:06:26.038094 20970 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:26.261096 22599 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.261672 22599 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.257537 21063 rpcz_store.cc:276] 0709 13:06:23.073231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.256178 (+3182947us) service_pool.cc:224] Handling call
0709 13:06:26.256481 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.256653 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.256680 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.256723 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:26.256760 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.256821 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.256942 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.843552 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:26.261143 21795 rpcz_store.cc:276] 0709 13:06:24.550847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.553172 (+  2325us) service_pool.cc:224] Handling call
0709 13:06:24.569748 (+ 16576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.248428 (+1678680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.248464 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.248512 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:26.248544 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.249293 (+   749us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54004934d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:26.249332 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.249497 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3855382400}
I20260709 13:06:26.264449 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.265096 21795 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:26.070268 22601 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.266376 22601 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.260692 21213 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:26.269085 21068 rpcz_store.cc:269] 0709 13:06:23.051127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.239830 (+188703us) service_pool.cc:224] Handling call
0709 13:06:23.931441 (+691611us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b540020b5a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42493dc0 00007f7f4251fe18 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6
0709 13:06:26.260828 (+2329387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120781184}
I20260709 13:06:26.260036 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1390) took 1573 ms. Trace:
I20260709 13:06:26.272118 21072 rpcz_store.cc:276] 0709 13:06:24.686490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.827053 (+1140563us) service_pool.cc:224] Handling call
0709 13:06:26.259970 (+432917us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:26.257382 21066 rpcz_store.cc:269] 0709 13:06:24.707711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.609458 (+901747us) service_pool.cc:224] Handling call
0709 13:06:26.256823 (+647365us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:26.277904 22463 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:06:26.079491 22565 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.080209 22664 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:26.080437 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 422038us
I20260709 13:06:26.083642 22662 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:26.283326 22662 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.083950 22669 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.085091 21185 leader_election.cc:400] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:26.085254 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:26.287824 21140 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:26.085495 22548 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.289340 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 379) took 2896 ms. Trace:
I20260709 13:06:26.088539 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:26.290058 21142 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:26.289753 21068 rpcz_store.cc:276] 0709 13:06:23.392793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.270384 (+2877591us) service_pool.cc:224] Handling call
0709 13:06:26.270822 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.271049 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.271096 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.271158 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:26.271205 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.289134 (+ 17929us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.289281 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.088641 22663 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:26.291013 22663 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:26.291441 21068 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:26.089717 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:26.292189 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 384) took 1783 ms. Trace:
I20260709 13:06:26.292563 21068 rpcz_store.cc:276] 0709 13:06:24.508477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.290867 (+1782390us) service_pool.cc:224] Handling call
0709 13:06:26.291232 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.291894 (+   662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.292131 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.293028 21185 leader_election.cc:403] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:06:26.293603 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 208736us
I20260709 13:06:26.294194 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 385) took 1785 ms. Trace:
I20260709 13:06:26.090703 21143 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:26.091785 22660 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.096403 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:26.289008 22548 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.295655 22660 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:26.296033 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 199691us
I20260709 13:06:26.096850 22616 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:26.299786 22616 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.097654 20968 catalog_manager.cc:5697] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:26.092113 21144 rpcz_store.cc:276] 0709 13:06:22.979840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.980806 (+   966us) service_pool.cc:224] Handling call
0709 13:06:22.981090 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.007417 (+ 26327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.007465 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:23.007525 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:23.007573 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.963391 (+1955818us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b54002585d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:24.963484 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.963650 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59928832}
I20260709 13:06:26.303730 22646 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:26.133384 21219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2123) took 1469 ms (client timeout 1916 ms). Trace:
I20260709 13:06:26.304275 22646 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.304811 22646 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:26.304466 21219 rpcz_store.cc:269] 0709 13:06:24.663609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.664191 (+   582us) service_pool.cc:224] Handling call
0709 13:06:26.133290 (+1469099us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.134667 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:26.307678 21217 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:26.308470 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 407) took 1590 ms. Trace:
I20260709 13:06:26.308903 21217 rpcz_store.cc:276] 0709 13:06:24.718539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.134357 (+1415818us) service_pool.cc:224] Handling call
0709 13:06:26.308344 (+173987us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.310621 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 408) took 1591 ms. Trace:
W20260709 13:06:24.898908 21767 connection.cc:256] RPC call timeout handler was delayed by -1.1335817679999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
W20260709 13:06:26.141876 21798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 488) took 2982 ms (client timeout 1757 ms). Trace:
W20260709 13:06:26.313722 21798 rpcz_store.cc:269] 0709 13:06:23.158953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.168970 (+ 10017us) service_pool.cc:224] Handling call
0709 13:06:25.008058 (+1839088us) spinlock_profiling.cc:243] Waited 6.67 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:26.141741 (+1133683us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15329536}
I20260709 13:06:26.302520 20968 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:26.316617 22546 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.319591 22601 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.319772 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:26.320673 21070 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:26.321313 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 105) took 3209 ms. Trace:
I20260709 13:06:26.321916 21767 leader_election.cc:400] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
W20260709 13:06:26.322095 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 139160us
I20260709 13:06:26.197746 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.323611 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:26.323807 21796 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:26.323974 22649 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.324493 22616 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.324572 22649 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:26.324954 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.321616 21070 rpcz_store.cc:276] 0709 13:06:23.112004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.259583 (+3147579us) service_pool.cc:224] Handling call
0709 13:06:26.320292 (+ 60709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.321073 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.321258 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.326844 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 386) took 1817 ms. Trace:
I20260709 13:06:26.327450 21070 rpcz_store.cc:276] 0709 13:06:24.508957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.326103 (+1817146us) service_pool.cc:224] Handling call
0709 13:06:26.326390 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.326539 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.326563 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.326602 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:26.326639 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.326691 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.326792 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.329698 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:26.034888 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.325s	user 0.032s	sys 0.047s
W20260709 13:06:26.146338 21797 rpcz_store.cc:269] 0709 13:06:23.058906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.063824 (+  4918us) service_pool.cc:224] Handling call
0709 13:06:25.011307 (+1947483us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54006c11d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:25.011452 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633674496}
W20260709 13:06:26.375023 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.666s	user 0.034s	sys 0.047s
W20260709 13:06:26.375496 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 1.6662e+06us
I20260709 13:06:26.147254 21800 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:24.951853 22572 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:25.837580 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.517748 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55822.1us
I20260709 13:06:26.260160 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1385) took 3184 ms. Trace:
I20260709 13:06:26.688663 21064 rpcz_store.cc:276] 0709 13:06:23.075744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.258777 (+3183033us) service_pool.cc:224] Handling call
0709 13:06:26.259085 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.259891 (+   806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.260105 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:26.258256 21065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 367) took 3217 ms (client timeout 1601 ms). Trace:
I20260709 13:06:26.691025 22571 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.332482 21069 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:26.624878 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 383) took 2116 ms. Trace:
I20260709 13:06:25.835577 21067 rpcz_store.cc:276] 0709 13:06:23.051427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.834428 (+2783001us) service_pool.cc:224] Handling call
0709 13:06:25.834734 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.834915 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:25.834941 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:25.834980 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:25.835011 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.835066 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.835169 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.294596 21068 rpcz_store.cc:276] 0709 13:06:24.508719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.293360 (+1784641us) service_pool.cc:224] Handling call
0709 13:06:26.293678 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.293853 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.293883 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.293926 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:26.293963 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.294038 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.294146 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:26.695621 21069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1391) took 2002 ms (client timeout 1773 ms). Trace:
I20260709 13:06:26.331383 21070 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
W20260709 13:06:26.696312 21069 rpcz_store.cc:269] 0709 13:06:24.693145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.767882 (+1074737us) service_pool.cc:224] Handling call
0709 13:06:26.694786 (+926904us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.628333 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 377) took 3236 ms. Trace:
I20260709 13:06:26.696638 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:26.543604 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 448323us
I20260709 13:06:26.508642 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1383) took 3436 ms. Trace:
W20260709 13:06:26.372833 21113 connection.cc:256] RPC call timeout handler was delayed by -1.105962489000035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
I20260709 13:06:26.699195 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:26.700879 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:26.339057 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:26.544340 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:26.703943 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:26.402812 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 79270.3us
W20260709 13:06:26.704492 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 160120us
W20260709 13:06:26.395972 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 61910.1us
I20260709 13:06:26.395131 22485 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.414885 21215 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:26.631651 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 401) took 1914 ms. Trace:
I20260709 13:06:26.544483 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 400) took 1828 ms. Trace:
I20260709 13:06:26.211261 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 398) took 1495 ms. Trace:
I20260709 13:06:26.242691 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.554733 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.373368 21796 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:26.459997 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.568231 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.705945 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:26.397774 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 99223us
I20260709 13:06:26.714389 21213 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:26.715396 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 415) took 1824 ms. Trace:
I20260709 13:06:26.716099 21213 rpcz_store.cc:276] 0709 13:06:24.893861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.259772 (+1365911us) service_pool.cc:224] Handling call
0709 13:06:26.715257 (+455485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.717337 21793 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:26.717639 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:26.717818 22613 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:26.718123 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 416) took 1044 ms. Trace:
I20260709 13:06:26.415989 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.340559 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:26.558000 21767 connection.cc:256] RPC call timeout handler was delayed by -1.228573603999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28500000000000003s for a response
I20260709 13:06:26.144789 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 96) took 1128 ms. Trace:
W20260709 13:06:26.715171 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 376041us
I20260709 13:06:26.718564 22613 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:26.495153 22407 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.549846 22410 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.514369 22418 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.523502 22423 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.517004 22428 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.394068 22570 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.562585 22476 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.566016 22591 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.698783 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:26.719264 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:26.733170 21801 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:26.697069 21063 rpcz_store.cc:276] 0709 13:06:23.392245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.262959 (+2870714us) service_pool.cc:224] Handling call
0709 13:06:26.263261 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.263579 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:26.617924 (+354345us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.627697 (+  9773us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.627791 (+    94us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:26.627895 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.627994 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.628170 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":285511,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":8761,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":145,"thread_start_us":258934,"threads_started":2,"wal-append.queue_time_us":6953}
I20260709 13:06:26.325202 22600 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.484815 22601 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:26.735817 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:26.736631 22601 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.407677 22602 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 13:06:26.715459 21221 rpcz_store.cc:276] 0709 13:06:24.715676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.715831 (+   155us) service_pool.cc:224] Handling call
0709 13:06:24.736479 (+ 20648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.210096 (+1473617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.210143 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.210221 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:26.210265 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.210955 (+   690us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540034cbd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:26.211010 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.211184 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3388443520}
I20260709 13:06:26.740883 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:26.396685 22638 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:26.705971 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:26.545831 22644 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.420640 22537 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.745625 21187 leader_election.cc:400] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:26.551909 22548 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.523191 22555 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.562067 22564 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.557330 22565 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.371894 22504 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.509958 22660 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.555653 22543 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.549510 22662 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.718669 21793 rpcz_store.cc:276] 0709 13:06:25.673227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.242325 (+569098us) service_pool.cc:224] Handling call
0709 13:06:26.718019 (+475694us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.181141 22547 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.436421 22626 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.749339 22504 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.697742 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:26.723553 21219 rpcz_store.cc:276] 0709 13:06:24.716407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.305686 (+1589279us) service_pool.cc:224] Handling call
0709 13:06:26.306156 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:26.306675 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.306721 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:26.341225 (+ 34504us) log.cc:844] Serialized 26 byte log entry
0709 13:06:26.541727 (+200502us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:26.541783 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.541882 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:26.543987 (+  2105us) raft_consensus.cc:1685] finished
0709 13:06:26.544055 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.544406 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5335,"raft.queue_time_us":26898,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":56881,"thread_start_us":84317,"threads_started":1,"wal-append.queue_time_us":128188}
I20260709 13:06:26.754298 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:26.580006 22646 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.561479 22628 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.569427 22663 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.435169 22631 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.433856 22633 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.410362 22641 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:26.438776 22643 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.562527 22648 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.545190 22668 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:26.437599 22651 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:26.648728 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 399) took 1932 ms. Trace:
I20260709 13:06:26.575615 22433 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:26.653059 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.725414 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:26.724015 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:26.675721 22657 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:26.682129 22446 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:26.691694 21065 rpcz_store.cc:269] 0709 13:06:23.040833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.050006 (+  9173us) service_pool.cc:224] Handling call
0709 13:06:25.692869 (+2642863us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:26.258187 (+565318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6424832}
I20260709 13:06:26.695569 22597 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.625211 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 382) took 2999 ms. Trace:
I20260709 13:06:26.701984 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:26.698565 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:26.311215 21212 rpcz_store.cc:276] 0709 13:06:24.718852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.117608 (+1398756us) service_pool.cc:224] Handling call
0709 13:06:26.310530 (+192922us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:26.705770 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:26.356734 21220 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:26.712975 21797 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:26.698381 21071 rpcz_store.cc:276] 0709 13:06:23.071751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.253648 (+3181897us) service_pool.cc:224] Handling call
0709 13:06:26.253938 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.254126 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:26.409032 (+154906us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.508298 (+ 99266us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.508349 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:26.508397 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.508464 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.508583 (+   119us) inbound_call.cc:177] Queueing success response
0709 13:06:26.671676 (+163093us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b180004f540. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
Metrics: {"mutex_wait_us":47062,"prepare.queue_time_us":234495,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":3873,"raft.queue_time_us":67907,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":6297,"spinlock_wait_cycles":6016384,"thread_start_us":191072,"threads_started":2,"wal-append.queue_time_us":36875}
I20260709 13:06:26.713891 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.714176 22676 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:26.718621 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:26.537629 22599 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.720058 22616 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:26.731421 22649 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:26.694833 21066 rpcz_store.cc:276] 0709 13:06:24.508163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.274917 (+1766754us) service_pool.cc:224] Handling call
0709 13:06:26.275398 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:26.275895 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.275942 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:26.276542 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 13:06:26.509368 (+232826us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:26.514298 (+  4930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.523400 (+  9102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:26.623932 (+100532us) raft_consensus.cc:1685] finished
0709 13:06:26.624030 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.624524 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1586,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":100,"thread_start_us":181478,"threads_started":1,"wal-append.queue_time_us":215857}
I20260709 13:06:26.732656 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:26.734795 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:26.746083 21187 leader_election.cc:403] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:06:26.752957 22547 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.553004 22632 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.758347 22646 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:26.746003 22575 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:26.742776 22638 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:26.713935 21215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 409) took 1994 ms (client timeout 2554 ms). Trace:
I20260709 13:06:26.763218 21800 raft_consensus.cc:1813] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Handling vote request from an unknown peer 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:26.908761 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:27.131242 22574 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:26.726467 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:26.766865 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 493) took 1092 ms. Trace:
I20260709 13:06:26.764204 22476 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:27.132609 21035 connection.cc:256] RPC call timeout handler was delayed by -1.9537131749999617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 13:06:27.127606 21215 rpcz_store.cc:269] 0709 13:06:24.719191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.125796 (+1406605us) service_pool.cc:224] Handling call
0709 13:06:26.713684 (+587888us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.138617 21800 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:26.720377 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:26.731010 22591 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.738880 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 387) took 2229 ms. Trace:
W20260709 13:06:26.724303 21093 debug-util.cc:398] Leaking SignalData structure 0x7b080014b480 after lost signal to thread 19970
I20260709 13:06:26.722981 22613 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:26.763741 22428 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:27.140945 22476 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:27.149209 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 414604us
I20260709 13:06:27.135977 21801 rpcz_store.cc:276] 0709 13:06:25.674670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.340207 (+665537us) service_pool.cc:224] Handling call
0709 13:06:26.766523 (+426316us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b54005d5f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:26.766787 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29161344}
I20260709 13:06:26.768301 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:26.723613 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 415) took 1050 ms. Trace:
I20260709 13:06:26.769549 22601 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:27.112692 22504 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:26.724669 22540 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:27.128712 22616 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:26.764652 22602 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:26.957213 22562 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.730125 21068 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:27.134641 22570 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:27.154685 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 246110us
I20260709 13:06:27.159466 21799 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:26.955727 22572 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.160063 21141 raft_consensus.cc:1813] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Handling vote request from an unknown peer 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:27.167325 21141 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:06:26.720558 21214 rpcz_store.cc:276] 0709 13:06:25.015882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.122581 (+1106699us) service_pool.cc:224] Handling call
0709 13:06:26.144694 (+ 22113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.141631 22676 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:27.127606 21072 rpcz_store.cc:276] 0709 13:06:23.625171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.272903 (+2647732us) service_pool.cc:224] Handling call
0709 13:06:26.273251 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.273513 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:26.422460 (+148947us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.624335 (+201875us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.624467 (+   132us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:26.624627 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.624778 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.624979 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105421,"prepare.queue_time_us":282716,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":117175,"raft.run_cpu_time_us":517,"raft.run_wall_time_us":522,"thread_start_us":244097,"threads_started":2,"wal-append.queue_time_us":151699}
I20260709 13:06:27.153330 22476 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.157775 22628 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:27.163125 21794 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:27.170043 22570 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:27.177099 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 451029us
I20260709 13:06:27.149933 22428 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:27.182518 22428 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.176765 22628 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:27.184491 22628 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:27.176409 22660 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:27.187417 22601 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:27.188004 22660 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:27.189350 22660 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.128558 21218 rpcz_store.cc:276] 0709 13:06:24.716056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.716182 (+   126us) service_pool.cc:224] Handling call
0709 13:06:24.716466 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.206592 (+1490126us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:26.206932 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.520042 (+313110us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.520121 (+    79us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:26.520184 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.647888 (+127704us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540034c6d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:26.648002 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.648430 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":653,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":616,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":25829,"spinlock_wait_cycles":3426476416,"thread_start_us":312048,"threads_started":1,"wal-append.queue_time_us":330454}
I20260709 13:06:27.162014 22616 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.154398 21795 rpcz_store.cc:276] 0709 13:06:25.672920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.721315 (+1048395us) service_pool.cc:224] Handling call
0709 13:06:26.722207 (+   892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.722699 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.722729 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.722815 (+    86us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:26.722865 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.722924 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.723078 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.202178 21221 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:27.177153 21184 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:27.201260 21220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 411) took 2399 ms (client timeout 1765 ms). Trace:
W20260709 13:06:27.179999 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 434650us
I20260709 13:06:27.196489 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:27.209651 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.210532 21221 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:27.211197 22428 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.214838 22428 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:27.202184 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.217190 21792 raft_consensus.cc:2410] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:06:27.217757 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 402) took 2500 ms. Trace:
I20260709 13:06:27.218295 21215 rpcz_store.cc:276] 0709 13:06:24.717027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.216436 (+2499409us) service_pool.cc:224] Handling call
0709 13:06:27.216876 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.217222 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.217253 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.217344 (+    91us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:27.217399 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.217472 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.217667 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.218945 22433 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:27.255926 21799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 496) took 1570 ms (client timeout 1915 ms). Trace:
W20260709 13:06:27.256392 21799 rpcz_store.cc:269] 0709 13:06:25.685541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.176714 (+491173us) service_pool.cc:224] Handling call
0709 13:06:27.255648 (+1078934us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54006c1e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:27.255857 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219095936}
I20260709 13:06:27.257272 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.258075 22433 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:27.259011 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.161723 21219 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:27.259831 21187 leader_election.cc:400] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:27.260409 21216 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:27.261312 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 413) took 2457 ms. Trace:
I20260709 13:06:27.261435 22485 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:27.261790 22562 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:27.219939 21093 debug-util.cc:398] Leaking SignalData structure 0x7b0800107160 after lost signal to thread 21004
I20260709 13:06:27.264094 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:27.264397 21093 debug-util.cc:398] Leaking SignalData structure 0x7b0800372920 after lost signal to thread 21096
I20260709 13:06:27.264683 22562 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:27.142462 21063 rpcz_store.cc:276] 0709 13:06:24.509220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.737402 (+2228182us) service_pool.cc:224] Handling call
0709 13:06:26.737937 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.738395 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.738434 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.738519 (+    85us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:26.738559 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.738641 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.738810 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.265300 22548 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:27.265890 22548 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:27.222015 22570 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:27.224210 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:27.223596 21070 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:27.220031 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.228780 22632 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:27.235826 21794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 127) took 1571 ms (client timeout 1881 ms). Trace:
I20260709 13:06:27.236596 21793 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:27.238200 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.239078 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.240137 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.242386 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:27.261742 21216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 405) took 2543 ms (client timeout 1900 ms). Trace:
I20260709 13:06:27.243469 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.243927 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 495) took 1558 ms. Trace:
I20260709 13:06:27.245523 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.245575 21792 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:27.247579 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.248736 22565 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:27.248967 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 412) took 2487 ms. Trace:
I20260709 13:06:27.249670 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.164311 22446 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:26.709177 21217 rpcz_store.cc:276] 0709 13:06:24.716738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.309867 (+1593129us) service_pool.cc:224] Handling call
0709 13:06:26.310940 (+  1073us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:26.325299 (+ 14359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.325334 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:26.452803 (+127469us) log.cc:844] Serialized 26 byte log entry
0709 13:06:26.630444 (+177641us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:26.630579 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.630836 (+   257us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:26.630957 (+   121us) raft_consensus.cc:1685] finished
0709 13:06:26.630974 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.631301 (+   327us) inbound_call.cc:177] Queueing success response
0709 13:06:26.681522 (+ 50221us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b4800281958. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
Metrics: {"raft.queue_time_us":30178,"raft.run_cpu_time_us":4778,"raft.run_wall_time_us":310351,"spinlock_wait_cycles":54163712,"thread_start_us":80720,"threads_started":1,"wal-append.queue_time_us":71185}
I20260709 13:06:27.236616 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 403) took 2519 ms. Trace:
I20260709 13:06:27.259768 21139 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:27.260807 22676 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.261765 21219 rpcz_store.cc:276] 0709 13:06:24.804298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.260007 (+2455709us) service_pool.cc:224] Handling call
0709 13:06:27.260495 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.260865 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.260895 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.260963 (+    68us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:27.261040 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.261132 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.261256 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:27.270041 21216 rpcz_store.cc:269] 0709 13:06:24.717945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.608397 (+890452us) service_pool.cc:224] Handling call
0709 13:06:27.261548 (+1653151us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54001252d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:27.261683 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3541218176}
I20260709 13:06:27.219331 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:27.222733 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:27.205163 21220 rpcz_store.cc:269] 0709 13:06:24.745540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.639223 (+893683us) service_pool.cc:224] Handling call
0709 13:06:27.141307 (+1502084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.225085 22668 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.267354 21799 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:27.230012 22628 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:27.264779 21093 debug-util.cc:398] Leaking SignalData structure 0x7b08001b9aa0 after lost signal to thread 21097
W20260709 13:06:27.326911 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 143677us
I20260709 13:06:27.270219 21800 rpcz_store.cc:276] 0709 13:06:25.685181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.237231 (+1552050us) service_pool.cc:224] Handling call
0709 13:06:27.237632 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.237912 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.237936 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.243564 (+  5628us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:27.243638 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.243702 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.243841 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.271111 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:27.224251 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.164039 22597 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.327937 21142 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:27.337751 21800 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:27.338773 21141 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:06:27.313365 21221 rpcz_store.cc:276] 0709 13:06:24.717318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.235371 (+2518053us) service_pool.cc:224] Handling call
0709 13:06:27.235764 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.236309 (+   545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.236333 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.236382 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:27.236411 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.236474 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.236567 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.345168 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 416) took 2440 ms. Trace:
I20260709 13:06:27.348421 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 494) took 1673 ms. Trace:
W20260709 13:06:27.336035 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 179783us
I20260709 13:06:27.347965 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 406) took 2629 ms. Trace:
I20260709 13:06:27.346263 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1392) took 2651 ms. Trace:
I20260709 13:06:27.330452 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 94) took 2315 ms. Trace:
I20260709 13:06:27.330773 21147 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
W20260709 13:06:27.345516 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 81560.7us
I20260709 13:06:27.328171 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:27.327306 21798 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
W20260709 13:06:27.338106 21093 debug-util.cc:398] Leaking SignalData structure 0x7b0800384ac0 after lost signal to thread 21171
I20260709 13:06:27.425424 21801 rpcz_store.cc:276] 0709 13:06:25.684716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.235538 (+1550822us) service_pool.cc:224] Handling call
0709 13:06:27.268394 (+ 32856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.348051 (+ 79657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.348089 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.348139 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:27.348212 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.348269 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.348367 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.339004 21795 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:27.445220 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:27.340638 22648 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:27.425212 21220 rpcz_store.cc:276] 0709 13:06:24.904444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.335377 (+2430933us) service_pool.cc:224] Handling call
0709 13:06:27.344525 (+  9148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.344801 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.344825 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.344871 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:27.344934 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.344987 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.345082 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.329677 22548 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.347860 22663 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:27.458086 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 191492us
I20260709 13:06:27.458833 21796 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:27.314700 21214 rpcz_store.cc:276] 0709 13:06:24.761962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.247806 (+2485844us) service_pool.cc:224] Handling call
0709 13:06:27.248227 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.248639 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.248676 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.248717 (+    41us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:27.248750 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.248807 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.248915 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.462091 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 418) took 1148 ms. Trace:
I20260709 13:06:27.465080 22613 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.452260 21218 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 1.
I20260709 13:06:27.487486 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 426) took 1070 ms. Trace:
I20260709 13:06:27.487977 21218 rpcz_store.cc:276] 0709 13:06:26.416894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.195727 (+778833us) service_pool.cc:224] Handling call
0709 13:06:27.487402 (+291675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.472071 20970 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 161) took 4370 ms. Trace:
I20260709 13:06:27.472178 22572 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:27.494031 21035 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:27.452718 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.425693 21215 rpcz_store.cc:276] 0709 13:06:24.718237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.247227 (+2528990us) service_pool.cc:224] Handling call
0709 13:06:27.326679 (+ 79452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.347582 (+ 20903us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.347610 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.347658 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:27.347698 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.347756 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.347885 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.433511 21212 rpcz_store.cc:276] 0709 13:06:25.015329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.329223 (+2313894us) service_pool.cc:224] Handling call
0709 13:06:27.329663 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.330013 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.330047 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.330116 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:27.330154 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.330218 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.330342 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.334561 21792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 128) took 1263 ms. Trace:
W20260709 13:06:27.268100 21794 rpcz_store.cc:269] 0709 13:06:25.664190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.664320 (+   130us) service_pool.cc:224] Handling call
0709 13:06:27.235561 (+1571241us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002c3654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:27.235742 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2344469248}
I20260709 13:06:27.345168 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 414) took 2456 ms. Trace:
I20260709 13:06:27.434342 22641 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:27.342096 22446 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:27.483565 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 304298us
I20260709 13:06:27.489065 21216 rpcz_store.cc:276] 0709 13:06:26.313876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.461144 (+1147268us) service_pool.cc:224] Handling call
0709 13:06:27.461488 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.461759 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.461793 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.461830 (+    37us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:27.461855 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.461906 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.462017 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.332132 22565 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.491741 22628 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:27.493433 21093 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2600 after lost signal to thread 21245
I20260709 13:06:27.493324 22613 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:27.345628 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 417) took 1672 ms. Trace:
I20260709 13:06:27.463110 22616 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.433287 21069 rpcz_store.cc:276] 0709 13:06:24.694433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.752163 (+2057730us) service_pool.cc:224] Handling call
0709 13:06:27.166376 (+414213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.168540 (+  2164us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:27.168942 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 13:06:27.271700 (+102758us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.271749 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:27.271793 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.271854 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.346092 (+ 74238us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b540011c1d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:27.346185 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2099,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":42582,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10871,"spinlock_wait_cycles":951445248,"thread_start_us":98401,"threads_started":1}
I20260709 13:06:27.479190 22648 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:27.490762 22504 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:27.503750 22572 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:27.525305 22570 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:27.426069 22540 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:27.528074 22540 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.529423 22504 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.529953 22504 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:27.535928 21797 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:27.537015 22616 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:27.537892 20968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 255) took 4106 ms. Trace:
I20260709 13:06:27.539458 21217 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:27.540184 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 419) took 1226 ms. Trace:
I20260709 13:06:27.540601 21217 rpcz_store.cc:276] 0709 13:06:26.314152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.426480 (+1112328us) service_pool.cc:224] Handling call
0709 13:06:27.507215 (+ 80735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.539873 (+ 32658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.540096 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.541181 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 418) took 1867 ms. Trace:
I20260709 13:06:27.541764 21797 rpcz_store.cc:276] 0709 13:06:25.673791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.232014 (+1558223us) service_pool.cc:224] Handling call
0709 13:06:27.232470 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.537008 (+304538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.540832 (+  3824us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b5400700dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:27.541086 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697000320}
I20260709 13:06:27.512228 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:27.518719 21139 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
W20260709 13:06:27.516455 21093 debug-util.cc:398] Leaking SignalData structure 0x7b08003a25c0 after lost signal to thread 21246
I20260709 13:06:27.516451 21793 rpcz_store.cc:276] 0709 13:06:25.673510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.750412 (+1076902us) service_pool.cc:224] Handling call
0709 13:06:27.236353 (+485941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.310587 (+ 74234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.345396 (+ 34809us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b54002c3650. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:27.345547 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116211072}
I20260709 13:06:27.521994 22632 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:27.496682 21792 rpcz_store.cc:276] 0709 13:06:26.071509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.244532 (+1173023us) service_pool.cc:224] Handling call
0709 13:06:27.245118 (+   586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.314396 (+ 69278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.334495 (+ 20099us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.501582 22638 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:27.508018 22641 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:27.546900 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 50893.3us
I20260709 13:06:27.538285 20968 rpcz_store.cc:276] 0709 13:06:23.410480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.410641 (+   161us) service_pool.cc:224] Handling call
0709 13:06:27.511917 (+4101276us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:26.647159 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:26.648402 (+  1243us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:26.648466 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:06:26.648508 (+    42us) tablet.cc:662] Decoding operations
0709 13:06:26.649251 (+   743us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:26.649396 (+   145us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:26.649449 (+    53us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:06:26.650223 (+   774us) tablet.cc:701] Row locks acquired
0709 13:06:26.650298 (+    75us) write_op.cc:260] PREPARE: finished
0709 13:06:26.650676 (+   378us) write_op.cc:270] Start()
0709 13:06:26.650934 (+   258us) write_op.cc:276] Timestamp: P: 1783602386650539 usec, L: 0
0709 13:06:26.650990 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 13:06:26.685167 (+ 34177us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b540039db20. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42499520 00007f7f42522b42 00007f7f42517f67 00007f7f42517da4 00007f7f45f52581 00007f7f45f51d58 00007f7f45f569f1 00007f7f45f569a9 00007f7f45f56939 00007f7f45f56901
0709 13:06:26.686753 (+  1586us) log.cc:844] Serialized 935 byte log entry
0709 13:06:27.252580 (+565827us) op_driver.cc:464] REPLICATION: finished
0709 13:06:27.456190 (+203610us) write_op.cc:301] APPLY: starting
0709 13:06:27.456273 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:27.456396 (+   123us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:27.456419 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:27.456874 (+   455us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:27.457048 (+   174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:06:27.457109 (+    61us) write_op.cc:312] APPLY: finished
0709 13:06:27.457302 (+   193us) log.cc:844] Serialized 29 byte log entry
0709 13:06:27.457468 (+   166us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:27.457558 (+    90us) write_op.cc:454] Released schema lock
0709 13:06:27.457617 (+    59us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":6994,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":203319,"apply.run_cpu_time_us":1742,"apply.run_wall_time_us":1744,"num_ops":2,"prepare.queue_time_us":264509,"prepare.run_cpu_time_us":9182,"prepare.run_wall_time_us":59967,"replication_time_us":601347,"spinlock_wait_cycles":7783168,"threads_started":1}]]}
I20260709 13:06:27.530944 22540 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:27.550093 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 122077us
W20260709 13:06:27.553035 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 101927us
I20260709 13:06:27.544668 22648 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.562937 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:27.489271 20970 rpcz_store.cc:276] 0709 13:06:23.101154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.101284 (+   130us) service_pool.cc:224] Handling call
0709 13:06:27.471821 (+4370537us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:26.577380 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:26.579759 (+  2379us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:26.579796 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:06:26.579813 (+    17us) tablet.cc:662] Decoding operations
0709 13:06:26.580229 (+   416us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:26.580270 (+    41us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:26.580287 (+    17us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:26.580526 (+   239us) tablet.cc:701] Row locks acquired
0709 13:06:26.580547 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:06:26.580693 (+   146us) write_op.cc:270] Start()
0709 13:06:26.580795 (+   102us) write_op.cc:276] Timestamp: P: 1783602386580655 usec, L: 0
0709 13:06:26.580811 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:06:26.581357 (+   546us) log.cc:844] Serialized 539 byte log entry
0709 13:06:26.744898 (+163541us) op_driver.cc:464] REPLICATION: finished
0709 13:06:27.153255 (+408357us) write_op.cc:301] APPLY: starting
0709 13:06:27.163907 (+ 10652us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:27.204878 (+ 40971us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:27.251644 (+ 46766us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b540039d5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251932a 00007f7f4249ef14 00007f7f4249eec3 00007f7f4249ee38 00007f7f4249edf3 00007f7f4249e0dc 00007f7f424c7e16 00007f7f424afcbb 00007f7f4249a39d 00007f7f4249dd75
0709 13:06:27.271343 (+ 19699us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:27.335972 (+ 64629us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:27.434854 (+ 98882us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:06:27.434918 (+    64us) write_op.cc:312] APPLY: finished
0709 13:06:27.454764 (+ 19846us) log.cc:844] Serialized 23 byte log entry
0709 13:06:27.455033 (+   269us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:27.455109 (+    76us) write_op.cc:454] Released schema lock
0709 13:06:27.455167 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:27.455731 (+   564us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:27.455766 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":479915,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":387854,"apply.run_cpu_time_us":3386,"apply.run_wall_time_us":310189,"mutex_wait_us":18902,"num_ops":1,"prepare.queue_time_us":309837,"prepare.run_cpu_time_us":4628,"prepare.run_wall_time_us":7242,"raft.queue_time_us":580027,"raft.run_cpu_time_us":14717,"raft.run_wall_time_us":579616,"replication_time_us":163773,"spinlock_wait_cycles":51226496,"thread_start_us":531273,"threads_started":4,"wal-append.queue_time_us":2108}]]}
I20260709 13:06:27.589679 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.557299 22641 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:27.501979 21219 rpcz_store.cc:276] 0709 13:06:24.888775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.326642 (+2437867us) service_pool.cc:224] Handling call
0709 13:06:27.344351 (+ 17709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.344712 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.344746 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:27.344803 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:27.344877 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.344978 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.345082 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.584846 22574 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:27.604944 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83930.7us
I20260709 13:06:27.609670 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:27.613476 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:27.614548 22477 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:27.614969 21093 debug-util.cc:398] Leaking SignalData structure 0x7b08003aebe0 after lost signal to thread 21825
W20260709 13:06:27.615389 21093 debug-util.cc:398] Leaking SignalData structure 0x7b08003aec00 after lost signal to thread 21826
I20260709 13:06:27.615232 22477 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:27.617316 22698 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.191s	user 0.003s	sys 0.002s
I20260709 13:06:27.617731 22477 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:27.602931 22574 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:27.608551 22632 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:27.620584 22574 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.617370 22562 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
W20260709 13:06:27.627938 21036 connection.cc:256] RPC call timeout handler was delayed by -2.1454296460000251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
W20260709 13:06:27.628417 21036 connection.cc:256] RPC call timeout handler was delayed by -1.814229646000058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 13:06:27.628741 21036 connection.cc:256] RPC call timeout handler was delayed by -1.7566296459999649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.198s for a response
I20260709 13:06:27.629056 22599 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:06:27.630306 22599 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:27.630684 22599 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:27.633898 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:27.634213 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:27.634554 21219 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:27.635206 21217 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:27.636245 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:27.627941 22631 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:27.640095 22631 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:27.640539 22631 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.631932 22599 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.642853 22599 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:27.629077 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5226296460000412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
I20260709 13:06:27.648629 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.643908 22504 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.651335 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.651782 22565 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:27.644749 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 43562.6us
I20260709 13:06:27.652722 21184 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:27.653036 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.653780 22572 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.653864 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:27.622751 21142 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
I20260709 13:06:27.655244 22540 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:27.656013 22632 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:27.656473 22632 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.628937 22562 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:27.657711 22562 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:27.658995 22562 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.660476 22562 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.663401 22504 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:27.645620 22574 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.646585 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:27.665753 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45811.8us
I20260709 13:06:27.664438 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:27.683956 21185 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:06:27.687563 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 70741.8us
I20260709 13:06:27.667876 22485 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:27.692934 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.693778 21216 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:27.694384 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.695060 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.696421 22657 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:27.709470 22657 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:27.724591 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46569.9us
I20260709 13:06:27.737193 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:27.737920 21218 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:27.739657 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 75310.9us
W20260709 13:06:27.738664 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45848.7us
I20260709 13:06:27.740958 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.743077 21215 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:27.724290 22657 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:27.748536 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.746429 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:27.730244 22602 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:27.756451 21139 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:27.757282 22476 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.753830 21069 raft_consensus.cc:2368] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:27.753194 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.739759 22643 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:06:27.764029 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.764858 21069 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:27.765226 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.771484 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.773785 21066 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:27.776136 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.761152 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.746116 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.781989 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.740415 22485 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:06:27.786460 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.756196 22602 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:27.788563 21070 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:06:27.788688 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 48224.2us
I20260709 13:06:27.790273 21218 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
W20260709 13:06:27.791976 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62799us
I20260709 13:06:27.787906 22540 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.793469 22485 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:27.793932 21147 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:27.794565 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.796288 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:27.796491 22648 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.817454 21071 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:27.796986 22643 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:27.819602 22643 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:27.804217 22602 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:27.820562 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.821770 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.822819 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.798379 22572 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:27.799501 22702 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:27.826648 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.805847 22485 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:27.804700 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:06:27.817348 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56479.1us
I20260709 13:06:27.819108 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.785173 22597 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:27.805181 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.825521 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.829388 21795 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:27.830248 22476 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:27.830303 22572 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:27.851228 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.851871 21218 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:27.852144 21219 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:27.853003 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.834775 22676 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:27.833634 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.853761 21070 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:27.833834 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.836376 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1512) took 1103 ms. Trace:
I20260709 13:06:27.836755 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.837265 22485 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.840305 21067 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:27.854774 21144 rpcz_store.cc:276] 0709 13:06:26.732004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.732170 (+   166us) service_pool.cc:224] Handling call
0709 13:06:27.835250 (+1103080us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b540042f754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:27.835419 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588715776}
I20260709 13:06:27.856629 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.842123 21069 raft_consensus.cc:2368] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:27.843811 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 420) took 1529 ms. Trace:
I20260709 13:06:27.844038 21793 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:27.845496 22547 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:27.846062 21213 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:27.846199 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:27.863055 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.863446 22649 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:27.863734 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.864248 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.849890 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:27.864907 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.858879 21141 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:27.854660 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:27.841478 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.853681 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.875186 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.859122 21147 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 2: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:27.838076 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.860575 22485 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.862676 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 425) took 1451 ms. Trace:
I20260709 13:06:27.865134 21800 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:27.856684 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 502) took 1101 ms. Trace:
I20260709 13:06:27.856340 22676 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:27.885833 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.832459 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.857758 21221 rpcz_store.cc:276] 0709 13:06:26.314401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.438955 (+1124554us) service_pool.cc:224] Handling call
0709 13:06:27.494514 (+ 55559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:27.550909 (+ 56395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.550959 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:27.561202 (+ 10243us) log.cc:844] Serialized 26 byte log entry
0709 13:06:27.612504 (+ 51302us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:27.612557 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.612630 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:27.843614 (+230984us) raft_consensus.cc:1685] finished
0709 13:06:27.843634 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.843757 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6176,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":28117,"thread_start_us":14910,"threads_started":1,"wal-append.queue_time_us":136121}
I20260709 13:06:27.882344 22701 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.888432 22597 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:06:27.894155 21221 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:27.894025 22701 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.894703 22597 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:27.895176 22597 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:27.896198 22701 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:27.896277 22597 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.897665 21066 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:27.898102 22597 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.898957 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.899744 21066 raft_consensus.cc:2376] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2. Re-sending same reply.
I20260709 13:06:27.865996 22547 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:27.900382 22547 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:27.900411 21063 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:27.902020 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.902779 21063 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:27.903539 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.904224 21070 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:27.904299 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.905014 21066 raft_consensus.cc:2368] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:27.906311 22714 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:27.907315 22574 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:27.888548 21064 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:27.892920 22631 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.912243 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.912345 22648 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:27.913013 21064 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:27.913947 22631 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:27.893306 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.905035 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.884539 21214 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:27.884217 21795 rpcz_store.cc:276] 0709 13:06:26.753139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.223842 (+470703us) service_pool.cc:224] Handling call
0709 13:06:27.854265 (+630423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:27.882566 21213 rpcz_store.cc:276] 0709 13:06:26.411588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.181990 (+770402us) service_pool.cc:224] Handling call
0709 13:06:27.862467 (+680477us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b540008fa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:27.862616 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952101248}
I20260709 13:06:27.889014 22676 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:27.898147 21067 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:27.920475 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.921103 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:27.921456 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.921700 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.922406 21218 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:27.879128 22476 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:27.923049 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.923589 22476 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:27.923852 21217 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:27.924607 22476 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:27.927654 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 47073.7us
I20260709 13:06:27.935869 22710 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:06:27.937211 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.938792 21219 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
W20260709 13:06:27.940469 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 120683us
I20260709 13:06:27.936192 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.947800 22602 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:27.950055 22602 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:27.950253 21765 connection.cc:256] RPC call timeout handler was delayed by -1.0267991380000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2022s for a response
I20260709 13:06:27.954326 21035 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:27.964473 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:27.941797 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:27.905794 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:27.991920 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:27.991628 21035 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:28.003363 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 146196us
I20260709 13:06:27.992249 22599 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:28.015208 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:28.016129 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 44198.9us
I20260709 13:06:28.016152 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:27.996016 21068 raft_consensus.cc:1813] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Handling vote request from an unknown peer 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:06:28.018431 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 103233us
I20260709 13:06:27.972858 21065 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:28.004282 22706 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:27.974888 22631 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:06:28.033015 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:27.974120 22657 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:27.972412 22632 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.017939 22599 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.038873 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.042510 21221 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:28.031827 21066 raft_consensus.cc:2368] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:28.052134 21218 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:28.051200 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.055953 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.057600 21220 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:28.045456 22599 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:28.050904 21068 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:06:28.063504 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.065322 21142 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.064218 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.047498 22701 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:28.066922 21141 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:28.067512 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.067055 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.068629 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.067692 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.069307 21221 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:28.057412 22540 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:28.047675 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.056224 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.080237 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.083590 21140 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:28.073179 22720 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:06:28.084385 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:28.084952 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.088328 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.110543 21141 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:28.115811 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.116662 21221 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 2: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:28.107465 22641 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:28.118237 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:28.119379 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:28.120025 22641 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:28.120682 21221 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:28.121032 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.121649 21145 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:28.122146 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.122794 21143 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.094231 22651 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:28.081879 22701 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:28.125908 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.128074 21212 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:28.130587 21141 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.131433 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.132899 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.138269 21798 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:28.134090 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:28.139313 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.140429 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.141215 21793 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:28.141841 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:28.144613 22638 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:28.145527 22657 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:28.128419 22547 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.146988 22632 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:28.147936 22547 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:28.148612 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 86610.5us
I20260709 13:06:28.149571 21767 leader_election.cc:400] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:28.149848 21767 leader_election.cc:403] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:06:28.151510 22428 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:28.155972 21145 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:28.157691 21142 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
W20260709 13:06:28.158913 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 85535.1us
I20260709 13:06:28.165855 21143 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:28.168876 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:28.167747 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 88138.8us
W20260709 13:06:28.171644 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.129673 22651 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.193933 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.176838 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.194918 21143 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.197551 21141 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:28.199679 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.072866 22706 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.128s
I20260709 13:06:28.201121 21796 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:28.202262 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.213500 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:28.214845 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:28.203214 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.203953 22643 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.227715 22643 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:28.165216 22676 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.233196 22676 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:28.199222 22651 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:28.242511 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.243409 21219 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:28.244576 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.208814 22701 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:28.211517 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.202353 21185 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:28.245944 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:28.246539 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50012.5us
I20260709 13:06:28.246793 21217 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:28.218506 21140 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:28.247480 22641 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:28.221005 21767 leader_election.cc:400] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:28.248134 21767 leader_election.cc:403] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:06:28.226387 22599 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:28.251482 22599 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:28.253324 22428 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:28.247604 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.257583 21215 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:28.261148 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.261996 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:28.262465 21798 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:28.248008 22641 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:28.263751 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.265125 22641 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.262585 21796 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:28.247594 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:28.269249 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.269622 22657 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:28.269724 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:28.270303 22657 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:28.271082 22547 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:28.271225 22657 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.271989 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.271770 22547 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:28.274336 22547 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.283111 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:28.290040 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.291039 21216 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:28.294492 21213 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:28.298926 21139 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:28.306349 21143 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:28.355465 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.309769 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.427214 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:28.365659 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:28.428090 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.310122 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:28.314821 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 177831us
I20260709 13:06:28.312089 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:28.219210 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46680.9us
I20260709 13:06:28.432353 21215 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:28.317764 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 124242us
W20260709 13:06:28.314178 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.310884 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:28.429162 21145 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.435425 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:28.404951 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.439502 21221 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:28.437077 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:28.443740 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.466717 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:28.466075 21212 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:28.346835 22476 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.406474 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.469718 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:28.394124 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.473392 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:28.310526 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.309026 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:28.416467 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 167235us
W20260709 13:06:28.494246 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 55977.5us
W20260709 13:06:28.488178 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48465us
I20260709 13:06:28.489753 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.488981 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.493654 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:28.507736 21216 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:28.508813 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:28.391829 22714 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.094s
I20260709 13:06:28.510555 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.516408 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:28.509858 22706 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:06:28.529917 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 58985.9us
W20260709 13:06:28.531976 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 63068.7us
I20260709 13:06:28.533161 21215 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:28.522280 22504 raft_consensus.cc:2764] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:06:28.543082 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.532706 22714 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:28.545819 22735 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:28.563812 22735 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:28.563449 22729 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:28.534791 22706 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:28.564618 22706 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:28.514750 21217 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:28.569082 21145 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:28.564342 22735 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.580719 21220 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
W20260709 13:06:28.549278 22732 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:06:28.600512 22572 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:28.606925 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 63919.8us
I20260709 13:06:28.608053 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.609911 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:06:28.611868 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44435.2us
I20260709 13:06:28.616545 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:28.617602 22714 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:28.622123 22648 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:06:28.623663 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.627931 21142 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:28.628247 21146 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:28.628974 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.629216 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.629772 21142 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:28.630138 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.630587 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.631628 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.630633 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.632045 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.627946 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.634577 21141 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:06:28.636775 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 126379us
I20260709 13:06:28.637204 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.637391 21138 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:06:28.611640 22602 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:28.638890 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:28.639669 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.640262 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.640349 22648 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:28.643553 22735 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:28.643908 22648 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.645833 22735 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:28.646574 21138 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:28.647699 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:28.648607 21139 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:28.650424 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.653525 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.654332 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.657384 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.660559 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.660900 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.660593 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.662642 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.665190 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.665794 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.666286 21071 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:28.666990 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:28.667609 21065 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:28.631166 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.631969 22660 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:28.673295 21071 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:28.663303 21138 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:28.678046 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.649497 22714 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:28.676985 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.641714 22602 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:28.689728 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:28.676278 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54548.5us
I20260709 13:06:28.695955 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.680609 22446 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:28.701814 22446 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:28.702747 22446 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:28.698068 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:28.706179 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.681998 22714 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:28.705718 22572 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:28.685551 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.707911 22714 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.708928 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.709528 21063 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:06:28.680271 22574 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:06:28.711905 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.712610 21070 raft_consensus.cc:2368] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:28.714128 22599 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:28.714748 22599 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:28.715830 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:28.716638 22562 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:28.717013 22562 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:28.717371 22562 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:28.719669 21065 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:28.720516 22735 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:28.720112 22599 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:28.720933 22735 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:28.721374 22735 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.727615 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.728292 21069 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:28.728972 22735 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:28.706863 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 146417us
I20260709 13:06:28.730732 22602 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:28.732141 22602 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.733253 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:28.734165 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.734934 22428 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:28.706962 21067 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:28.735211 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.735491 22428 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:28.735530 22745 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:28.714159 21796 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:28.740043 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.740754 21069 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:28.682317 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:28.719671 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:28.697669 22597 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.799s	user 0.007s	sys 0.015s
I20260709 13:06:28.675836 22485 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:06:28.744357 22649 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:28.741914 21035 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:28.750989 22735 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:28.751158 22710 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:28.744410 21140 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
W20260709 13:06:28.747506 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.752261 22562 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.748879 21066 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:28.754297 22562 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:28.747772 22651 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.756207 21143 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:28.757053 22651 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:28.757541 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:28.758167 21064 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:28.761209 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:28.762501 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 66632.7us
I20260709 13:06:28.766053 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:28.711093 21063 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:28.774510 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.761813 21220 raft_consensus.cc:2376] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4. Re-sending same reply.
W20260709 13:06:28.775352 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55788.9us
I20260709 13:06:28.779449 21218 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:28.786623 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:28.785784 21111 leader_election.cc:336] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.539s (SENT)
I20260709 13:06:28.787948 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.788594 21218 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:28.792179 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:28.764072 22745 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:28.793845 21220 raft_consensus.cc:2376] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4. Re-sending same reply.
I20260709 13:06:28.797648 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.772801 22649 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:28.798836 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.800665 21218 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:28.782267 22597 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:28.795065 21071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:28.804888 21071 raft_consensus.cc:606] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:28.805263 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.805406 21071 raft_consensus.cc:614] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:28.805279 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:28.761848 22706 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:28.806277 21111 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.726s (SENT)
I20260709 13:06:28.809811 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:28.799513 22710 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:28.801550 22574 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:28.827836 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64120.6us
I20260709 13:06:28.846915 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.847957 22562 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:28.848412 22562 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.848763 22562 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:28.800843 22446 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:06:28.852006 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:28.855918 21111 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.765s (SENT)
I20260709 13:06:28.845834 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:28.857064 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:28.863327 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.870208 21140 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:28.870565 21220 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:28.827170 22745 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:28.851613 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:28.871505 21218 raft_consensus.cc:2376] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3. Re-sending same reply.
I20260709 13:06:28.879330 21143 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
W20260709 13:06:28.881464 21111 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.829s (SENT)
I20260709 13:06:28.888645 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:28.904517 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52609.8us
I20260709 13:06:28.883764 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:28.911586 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.908250 21064 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:28.914153 21187 leader_election.cc:400] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:28.914487 21187 leader_election.cc:403] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:06:28.918313 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:28.928730 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 101573us
W20260709 13:06:28.966616 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52576.6us
I20260709 13:06:28.878002 22714 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:28.968370 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:28.968895 21143 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:28.970281 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:28.970275 21111 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.900s (SENT)
I20260709 13:06:28.859186 22649 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:28.875617 22428 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:28.903735 22710 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:28.988332 21111 leader_election.cc:336] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.924s (SENT)
I20260709 13:06:28.988794 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:28.975986 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:29.027387 21212 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:06:29.067698 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79418.8us
W20260709 13:06:28.992693 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.049249 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 73066us
I20260709 13:06:28.994886 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:29.069430 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:28.995362 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.985944 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:28.999843 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:29.073933 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.075063 21218 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
W20260709 13:06:29.081432 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 95549.4us
W20260709 13:06:29.088083 21111 leader_election.cc:336] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.542s (SENT)
I20260709 13:06:29.101894 22701 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:29.089145 22597 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:28.995776 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:29.103143 22597 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.103780 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:29.107172 21213 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:29.107513 22485 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:29.107935 22485 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:29.108301 22485 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.109478 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:29.104306 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.111907 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:28.987932 22750 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 13:06:29.039978 22751 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:06:29.113762 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.017397 22648 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 13:06:29.114209 21218 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:28.970933 22714 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:29.115201 22648 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:28.982303 22632 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
W20260709 13:06:28.990003 22754 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:06:29.115674 22648 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:29.116439 21216 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:29.118005 21144 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.990921 22548 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:29.132889 21140 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:29.105108 21111 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:29.032528 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:29.111919 22446 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.135836 21796 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:29.113016 22757 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:28.988085 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:29.118904 21139 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:29.136951 22757 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:29.137362 22757 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.115489 22714 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:28.985343 22657 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.102497 22701 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.141147 22701 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:29.114957 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:29.115622 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46321.8us
I20260709 13:06:29.119716 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:29.143359 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.144579 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.145393 22632 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:29.145831 22632 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:29.146014 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.146242 22632 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.147172 22641 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:29.147588 22641 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:29.147951 22641 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.149570 22701 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.151672 22701 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:29.151665 22547 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:29.152360 22547 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:29.152865 22547 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.169315 22547 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:28.996210 22749 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:29.212841 21216 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:29.212771 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:29.043749 21793 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:29.176126 21218 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:29.174914 21144 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:29.167236 22632 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.210148 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:28.996254 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:29.035050 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:29.212709 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:29.020658 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 91521.2us
I20260709 13:06:29.131759 22548 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:28.987577 22599 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.224s
I20260709 13:06:29.160346 22714 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:29.259316 22602 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.003s
W20260709 13:06:29.267496 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 501734us
I20260709 13:06:29.256644 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:29.300320 21216 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:29.217296 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:29.271938 21147 raft_consensus.cc:2410] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:06:29.273841 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.116741 22648 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.213018 22749 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:29.307740 22749 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.309185 22749 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.312371 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:29.314497 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.315433 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:29.315990 22744 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:29.317096 21216 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:29.318320 22547 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:29.318481 21216 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:06:29.320279 22547 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:29.217731 22759 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:29.303483 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:29.260003 22668 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:29.210760 22574 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 13:06:29.306545 22648 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:29.339701 21065 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:29.340183 21065 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:29.322363 22745 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:29.323556 22562 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.323863 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.341223 21065 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.342396 21065 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.342903 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:29.343472 22562 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:29.344667 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:29.258531 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 153527us
W20260709 13:06:29.311080 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 196189us
I20260709 13:06:29.325309 21221 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:29.274933 22701 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.334708 22476 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:29.334849 22485 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.335430 21214 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:29.346846 22745 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:29.347250 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:29.358867 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.298947 22706 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:29.360618 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.337884 21063 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:06:29.362069 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.343825 22746 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:29.364233 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:29.332845 22602 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:29.358114 22476 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:29.371160 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 159020us
I20260709 13:06:29.373538 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.374997 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.385835 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.367575 22757 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.361284 22668 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:06:29.366706 22746 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:29.413820 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45905.1us
I20260709 13:06:29.398272 22657 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:29.414158 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55379.1us
I20260709 13:06:29.433454 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:29.422354 22764 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:29.423925 22765 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.400439 22572 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:29.425603 22757 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:29.402531 22574 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:29.399698 22602 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.428004 22476 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.399084 22706 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:29.411779 22676 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:29.440809 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 78820.1us
I20260709 13:06:29.419315 22744 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:29.437088 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.420030 22746 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:29.452232 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.454527 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:29.460479 22749 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:29.461071 22701 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:29.462322 22701 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:29.462669 22701 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:29.463874 22701 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:29.466517 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0039653919999409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
W20260709 13:06:29.467319 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.812s (SENT) [suppressed 57 similar messages]
I20260709 13:06:29.468159 22749 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.468657 22749 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:29.473882 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:29.477921 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:29.481029 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:29.495183 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:29.497046 21110 leader_election.cc:336] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.683s (SENT)
I20260709 13:06:29.444940 22651 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.497571 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.451622 22744 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:29.444335 22641 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.497834 22701 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.531224 22572 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:29.573083 22476 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.474579 22446 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.139s
I20260709 13:06:29.570844 22676 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:29.570376 22706 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:29.583565 22744 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.606326 21212 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
W20260709 13:06:29.605628 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 258470us
I20260709 13:06:29.588225 22714 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:29.580389 22632 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:29.581840 22651 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.660430 21071 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:29.622579 22765 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.616590 22485 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:29.630452 22599 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.666597 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:29.670506 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:29.655918 22668 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:29.667193 22756 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.393s	user 0.007s	sys 0.003s
I20260709 13:06:29.662191 22676 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:29.662890 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:29.682009 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 213717us
W20260709 13:06:29.675889 22762 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
W20260709 13:06:29.679903 22768 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:06:29.683943 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 247662us
I20260709 13:06:29.690894 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.694525 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.673162 22599 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.694169 22780 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:29.692866 21063 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:06:29.695729 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:29.696631 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 201519us
I20260709 13:06:29.721880 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:29.709576 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.709818 22446 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.711540 21219 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:29.671242 22706 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.727094 22562 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:06:29.727334 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 230330us
I20260709 13:06:29.727600 22562 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:06:29.728086 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.728475 22562 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.729676 22562 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.731359 22676 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:29.716361 22778 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:29.687690 22779 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:06:29.692142 22641 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:29.719046 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.685998 22602 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:29.726461 22446 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:29.737501 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.771332 21110 leader_election.cc:336] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.881s (SENT)
I20260709 13:06:29.784528 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.785429 21140 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:29.787128 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 161927us
I20260709 13:06:29.788081 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:29.789819 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:29.786484 21113 leader_election.cc:400] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:29.791839 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:29.794036 21145 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:29.793200 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:29.794406 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:29.793851 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:29.764102 22676 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.755934 22648 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:29.757609 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58124.1us
I20260709 13:06:29.793411 22783 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.799233 21140 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:29.782812 22602 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:29.799177 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 62700.7us
I20260709 13:06:29.804499 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:29.758379 22649 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:29.806625 21138 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:29.807209 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.792039 22668 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.796877 22706 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:29.829262 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.798382 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:29.832835 21801 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:29.781024 22780 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:06:29.835706 21799 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:06:29.845263 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.803356 22710 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:29.785288 22770 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
W20260709 13:06:29.847766 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 138270us
W20260709 13:06:29.849033 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.849537 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:29.809156 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:29.860255 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.827796 22602 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.847101 22785 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:29.853246 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60094.1us
I20260709 13:06:29.846366 22706 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:29.849043 21796 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:29.868382 22485 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:29.869465 22746 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.869891 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.871392 22788 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:06:29.872593 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:29.874112 22773 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:06:29.882822 22795 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:06:29.875247 22485 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:29.874616 22777 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:29.883375 22485 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:29.883854 22485 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.885582 21797 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:29.886157 22791 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.858158 22790 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:29.875917 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:29.859947 22649 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:29.887394 22446 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:29.887580 22649 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:29.887961 22446 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:29.888028 22649 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.888829 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.890537 22602 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:29.891743 21793 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:29.892336 22791 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:29.893206 22706 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.839862 22648 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.895493 22710 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:29.896096 22791 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:29.896956 22648 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:29.849637 22651 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:29.876513 22786 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:06:29.879330 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:29.880303 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:29.880476 21792 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:06:29.880909 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.881702 21065 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.000s
I20260709 13:06:29.882752 22773 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:29.886272 22599 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:29.877655 22759 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:29.854053 22641 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:06:29.907873 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:29.878268 22749 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.910127 21801 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:29.901366 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:29.855093 22764 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:06:29.911895 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:29.907485 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:29.917354 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:29.918735 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 110479us
I20260709 13:06:29.920149 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:29.921593 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:29.923058 21035 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:29.878265 22778 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.912259 22759 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:29.927013 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:29.911580 22485 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.927577 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:29.902717 22701 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:06:29.931020 22651 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:29.939574 22651 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:29.939859 22485 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:29.922777 22785 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:29.940565 22785 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:29.941025 22785 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:29.941183 22701 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:29.941638 22701 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:29.942469 22701 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.907830 22710 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:06:29.928668 22796 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:29.928665 22599 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:29.928387 21065 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:29.926450 22746 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:29.929107 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:29.921963 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:29.944085 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.921612 22779 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:06:29.922852 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:29.970489 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 44750.7us
I20260709 13:06:29.931600 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.925952 22446 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:06:29.940501 22651 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.966836 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:29.943233 22649 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:29.950030 22599 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:29.974524 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 80690us
I20260709 13:06:29.976882 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.977564 21798 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.979856 22548 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:29.963063 22701 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:29.930233 22787 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:06:29.990535 21065 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:06:29.994799 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 63322.7us
W20260709 13:06:29.995544 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51534.8us
I20260709 13:06:29.983134 22599 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.998901 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:29.999243 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:30.001322 22548 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:29.998131 22676 raft_consensus.cc:2764] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:06:30.002028 22802 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:29.987655 22706 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:30.004943 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:30.048050 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 48860.1us
I20260709 13:06:30.064530 21143 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:29.943574 22773 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:30.055940 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51696.5us
I20260709 13:06:30.056800 22807 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:30.050525 22710 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:30.002782 22764 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:06:30.081281 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:30.124104 22779 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:06:30.123848 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58033.4us
I20260709 13:06:30.127435 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:06:30.130992 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49838.4us
I20260709 13:06:29.996523 22759 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.137356 22706 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:30.137962 22759 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:30.138281 22808 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:30.139000 22808 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:30.140192 22808 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:30.141091 22744 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 13:06:30.141574 22744 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.082831 22749 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:30.147269 21799 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:30.126113 22773 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:30.137984 22706 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:30.150182 22706 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.151510 22706 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:30.164855 22810 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:06:30.169157 21798 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:30.172112 21797 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:06:30.124831 22786 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:06:30.134725 22676 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:06:30.191316 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57611.3us
I20260709 13:06:30.205153 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:30.202540 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:30.212507 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:30.219056 22761 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:30.219530 22761 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:30.220353 22761 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.221457 22761 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:30.223644 21110 leader_election.cc:336] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.578s (SENT)
I20260709 13:06:30.224160 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:30.224251 22744 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:30.206321 22787 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:30.225579 22807 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:30.226243 22787 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.227491 22788 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:30.228219 22788 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:30.212563 22676 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:30.228792 22787 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:30.229352 22676 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.212780 22785 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.232506 22785 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:30.229270 22788 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.207765 22770 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.233829 22797 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:30.235325 22785 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:30.257735 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.275468 21110 leader_election.cc:336] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.684s (SENT)
I20260709 13:06:30.276175 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:30.277845 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.248451 22641 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:30.288121 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75693.1us
I20260709 13:06:30.289222 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:30.303167 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:30.308686 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.289517 22602 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.312299 21145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:06:30.297122 22547 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.313627 21145 raft_consensus.cc:606] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:30.315182 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 168979us
I20260709 13:06:30.299842 22785 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:30.319005 22809 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:30.325238 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.320140 21063 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:06:30.251791 22770 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.292017 22745 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:06:30.290171 22641 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:30.343451 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.407233 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 165879us
W20260709 13:06:30.411620 21036 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.514s (SENT)
W20260709 13:06:30.410300 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61113.6us
W20260709 13:06:30.412266 21036 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.514s (SENT)
I20260709 13:06:30.412562 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:30.415552 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 73263us
I20260709 13:06:30.420966 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.307760 22810 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:06:30.421836 21212 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:30.331068 22797 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:30.477392 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:30.434845 21036 leader_election.cc:336] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.530s (SENT)
I20260709 13:06:30.523089 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:30.317414 22602 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.402781 22786 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:30.537777 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 102952us
I20260709 13:06:30.477790 22632 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.559391 22786 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.567932 22786 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:30.572677 22602 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:30.573375 22798 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:06:30.574784 22815 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:06:30.575349 22701 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.003s
I20260709 13:06:30.562826 21065 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
W20260709 13:06:30.578334 22548 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.001s
W20260709 13:06:30.568037 22805 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.434s	user 0.014s	sys 0.009s
I20260709 13:06:30.523928 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:30.332024 22547 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.521461 22797 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:30.592264 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:30.435077 22787 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.158s
I20260709 13:06:30.406457 22809 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.176s
I20260709 13:06:30.634464 21067 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:30.403930 22746 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.193s
I20260709 13:06:30.637648 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:30.585940 22764 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:30.437106 22778 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:30.599661 22797 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:30.659089 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:30.585229 22632 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:06:30.657908 22599 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.000s
I20260709 13:06:30.666666 21069 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:30.597910 22790 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:30.634737 22602 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:30.642935 22787 consensus_queue.cc:1243] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:30.656198 22786 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:30.667982 21065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:30.695741 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:30.694588 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:30.702685 21217 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:30.706000 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:30.706142 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:30.707978 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:30.708364 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:30.707139 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:30.709623 21138 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
W20260709 13:06:30.694054 22760 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.345s	user 0.003s	sys 0.009s
I20260709 13:06:30.711253 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:30.720165 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:30.726698 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.711863 21144 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:06:30.711791 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:30.712076 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:30.714092 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:30.673617 22599 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.720396 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:30.695355 22787 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:30.681949 22809 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:30.713735 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:30.730548 21144 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:30.755156 21147 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:06:30.761304 21147 pending_rounds.cc:85] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 1
I20260709 13:06:30.761751 21138 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:30.712555 21065 raft_consensus.cc:606] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:30.723511 22788 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:30.682085 22602 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.694557 22786 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.691745 22790 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.762413 22787 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:30.761541 21147 pending_rounds.cc:107] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:06:30.783519 21147 raft_consensus.cc:2889] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:30.780071 22824 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:06:30.760933 22809 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:30.759959 22641 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:30.785223 22809 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.789245 22769 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.789834 22769 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.791987 22769 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:30.797448 22790 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:30.799448 22770 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:30.800117 22641 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.800678 22641 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.802886 22641 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:30.803710 22641 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.804198 22641 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.804723 22599 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:30.805263 22770 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.810817 22770 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.812300 22799 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:30.812667 22799 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:30.812837 22710 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:30.813201 22710 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:30.813505 22799 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.814040 22710 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.814788 22799 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:30.816049 22632 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:30.805370 22547 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:30.783351 22788 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.818532 22602 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:30.819541 22788 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:30.806299 22548 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:30.820219 22548 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.822247 22548 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:30.823509 22770 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:30.799569 22786 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:30.807962 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.809298 22572 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:30.824807 21217 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:30.824848 22572 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.815147 22710 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:30.805711 22769 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:30.809820 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.809508 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.816129 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1430) took 1556 ms. Trace:
I20260709 13:06:30.809473 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:30.829061 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:30.829813 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:30.830768 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.833608 21144 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:30.827122 22572 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:30.841424 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:30.841926 21218 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:30.842640 21217 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:30.842948 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.843732 21218 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:30.836853 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:30.843626 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:30.841558 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.847316 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.847335 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.848549 21218 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:06:30.847424 21216 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:06:30.872211 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:30.839879 22757 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.873392 21214 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 2: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:30.848239 21213 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:30.845732 22706 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:30.871176 22809 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:30.917075 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:30.885546 22802 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:30.918401 22802 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.918462 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:30.918885 22802 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:30.871491 22761 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.002s
W20260709 13:06:30.878782 22825 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:06:30.880180 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:30.922739 21215 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:30.840131 22769 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.882673 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:30.884346 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:30.882479 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:30.896905 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:30.936654 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52381.9us
I20260709 13:06:30.908371 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:30.873703 22776 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:30.921456 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.870790 22779 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:06:30.874421 22668 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:06:30.919142 21217 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:30.944077 21220 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:06:30.954212 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71587.3us
W20260709 13:06:30.968227 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 131461us
I20260709 13:06:30.969091 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:30.897624 22757 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.942348 22790 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:30.974634 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:30.954636 22572 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:30.956440 22761 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:30.977031 21217 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 2: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:30.954680 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:30.979460 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:30.980736 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.880125 22787 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.934123 22706 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:30.990584 21215 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:30.991371 21794 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:30.991991 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 151) took 1090 ms. Trace:
I20260709 13:06:30.978401 22446 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.992383 21794 rpcz_store.cc:276] 0709 13:06:29.901907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.902876 (+   969us) service_pool.cc:224] Handling call
0709 13:06:30.936777 (+1033901us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:30.991927 (+ 55150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5297024}
I20260709 13:06:30.992652 22446 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:30.995824 21217 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:30.996023 21215 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:30.997270 21217 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:30.997499 21215 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:30.999939 21215 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:31.001420 21215 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:31.004381 21215 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:30.985343 22676 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:31.008155 22676 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.009080 22676 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.010496 22676 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:31.011827 21146 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:30.955420 22786 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:06:31.021483 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.023411 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.024082 22837 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:06:31.026564 22796 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:31.026979 22796 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:31.028215 22835 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:31.027908 22796 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.028615 22835 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.029186 22796 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:30.975450 22826 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.977407 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:31.040469 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.018430 22759 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:30.995105 22446 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.020426 22838 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:31.051446 22783 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:31.051928 22783 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:31.052850 22783 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.053913 22783 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.056521 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:31.073143 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.075083 22757 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.043468 22749 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:30.839417 21064 rpcz_store.cc:276] 0709 13:06:29.259355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.269096 (+  9741us) service_pool.cc:224] Handling call
0709 13:06:29.333261 (+ 64165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.432312 (+ 99051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:29.435340 (+  3028us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:29.579501 (+144161us) log.cc:844] Serialized 26 byte log entry
0709 13:06:29.725714 (+146213us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:29.778804 (+ 53090us) log.cc:844] Serialized 14 byte log entry
0709 13:06:29.792517 (+ 13713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.815727 (+1023210us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b540011b7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:30.815775 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:30.815827 (+    52us) raft_consensus.cc:1685] finished
0709 13:06:30.815910 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.816047 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20433,"prepare.queue_time_us":80338,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3010,"raft.queue_time_us":41386,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":4839,"spinlock_wait_cycles":51814144,"thread_start_us":166090,"threads_started":2,"wal-append.queue_time_us":190022,"wal-append.run_cpu_time_us":2247,"wal-append.run_wall_time_us":1101451}
I20260709 13:06:31.086092 22811 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:31.086772 22811 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.087980 21138 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:06:31.088346 21138 pending_rounds.cc:85] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 4
I20260709 13:06:31.088554 21138 pending_rounds.cc:107] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:06:31.088593 22706 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.088886 21138 raft_consensus.cc:2889] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:31.090718 22641 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.030551 22761 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:06:31.042654 22826 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.094430 22811 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.095573 22826 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:31.101776 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 61360.9us
I20260709 13:06:31.102805 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:31.113479 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 141819us
I20260709 13:06:31.117841 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.101644 22446 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:31.101985 22811 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.126791 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.129406 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.131778 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:31.029467 22835 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.139482 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.142038 21141 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:31.148607 22765 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.158744 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.160807 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.162542 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.163005 21798 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:31.163619 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.134205 22761 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:31.164582 21801 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:31.165012 22835 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:31.125900 22790 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:31.167645 21141 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:31.168190 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.135171 22710 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.169215 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.127761 22446 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:31.134459 22759 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.171819 22572 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:31.176600 22797 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.003s
I20260709 13:06:31.160523 22842 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.180184 21792 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:31.128345 22428 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:06:31.180920 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:31.184214 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.205475 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.185395 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:31.160248 22757 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:31.164602 22761 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:31.209386 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65159.3us
I20260709 13:06:31.186926 22826 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:31.169301 22562 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:31.198156 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.189971 22840 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:31.201440 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.206548 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.186573 22446 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.193578 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.215587 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.212164 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.216014 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.216177 21143 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:31.216336 21139 raft_consensus.cc:1813] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Handling vote request from an unknown peer 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:31.217020 21139 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 3 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:06:31.214406 21147 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:31.192417 22749 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.221532 21144 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:06:31.223315 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:31.222848 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:31.225080 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.202039 22797 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.191605 22759 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.217236 22777 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.194797 22572 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.228783 22802 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.224893 22597 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:06:31.241925 21064 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:06:31.242326 22802 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.237089 22797 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.226509 22776 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:31.222723 22807 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:06:31.218667 22710 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:31.245528 22710 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.247705 22710 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:31.242690 22844 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:06:31.238372 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.249812 22572 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.228626 22757 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:31.251600 22757 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.251971 22757 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.256775 22648 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:31.257248 22648 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.257603 22648 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:31.258234 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.263664 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.267824 22649 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.268874 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:31.270891 21147 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.269263 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:31.279500 22761 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:31.280272 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55867us
I20260709 13:06:31.282493 21071 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:06:31.283169 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.283614 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:31.284814 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.281772 21063 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:31.289052 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:31.269641 22777 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.296396 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:31.297195 21071 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.268456 22759 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.288280 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:31.319312 22765 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:31.286330 22649 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:06:31.344002 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:31.345041 21792 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:06:31.346395 22846 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:31.334333 22783 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:31.348011 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.347952 22783 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.349236 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.349824 21070 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:31.350144 22783 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:31.349972 22757 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.350651 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.351900 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.352530 21070 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:31.352866 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.353423 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.354135 21070 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:31.352464 21216 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:31.373262 21147 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:31.374352 21140 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:06:31.375530 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:31.325419 22777 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.330642 22808 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:06:31.387705 21799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 150) took 1485 ms. Trace:
I20260709 13:06:31.388177 21799 rpcz_store.cc:276] 0709 13:06:29.901610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.905017 (+  3407us) service_pool.cc:224] Handling call
0709 13:06:29.905353 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:30.159890 (+254537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.386716 (+1226826us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400700154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:31.386900 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555883904}
I20260709 13:06:31.384173 21063 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:06:31.392131 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.393019 21147 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:31.393083 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.393684 21069 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:31.395047 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:31.376468 22817 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:31.414023 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:31.402912 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.418200 21220 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:31.415827 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.405064 21218 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:31.419474 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.421284 21218 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:31.422732 21799 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:31.408129 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:31.423624 21141 raft_consensus.cc:2410] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:31.426714 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.402248 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.427120 21220 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:31.426892 21212 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
W20260709 13:06:31.428009 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.428712 21212 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:06:31.429303 21212 pending_rounds.cc:85] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 1
I20260709 13:06:31.430303 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:31.432394 21212 pending_rounds.cc:107] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:06:31.432721 21212 raft_consensus.cc:2889] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:31.432924 21221 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 2: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:31.431986 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.430934 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:31.431705 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.437687 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:31.390890 22476 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.439853 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:31.421733 22765 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.435213 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:31.451318 21215 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:31.451723 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.453009 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:31.452044 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:31.453902 21217 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:31.429840 22649 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:31.405992 22446 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.405460 22799 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:06:31.469012 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.431036 22817 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:31.463527 22714 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.464885 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:31.472316 21798 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.453850 22765 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.457270 22840 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:31.441077 22807 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:31.476245 22765 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.476583 22757 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.445873 22476 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.466741 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.467311 22806 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:31.451956 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.470418 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.479005 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:31.446318 22788 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:06:31.487828 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.503067 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.504529 21067 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:31.505256 21800 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:31.506080 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.509868 22786 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.511200 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.512821 22757 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.465586 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:31.514856 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 62991.1us
I20260709 13:06:31.478556 22641 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.482312 22788 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:31.515493 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46963.6us
I20260709 13:06:31.515702 21068 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:31.520679 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.504514 22765 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:31.521329 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.522414 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.523600 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.515947 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:31.525159 21798 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:31.546824 22476 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.547369 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.547443 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.548079 21798 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:31.548282 22797 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:31.541498 22765 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.525584 22485 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.548717 22797 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.549108 21795 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:31.528785 21071 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:31.550257 22765 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.550469 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.551483 21070 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:31.534178 22835 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:06:31.551575 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.552835 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:31.553259 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.531580 22788 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.506722 22574 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.503400 21213 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:06:31.482847 22806 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:31.517970 22676 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:31.480511 22714 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.520828 22777 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.536890 22651 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:06:31.543674 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.558230 21767 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:31.562639 22806 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:31.544642 22816 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:31.544446 22791 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:31.545117 22706 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:31.545691 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.545746 22446 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.545876 21797 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:31.553558 21071 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:31.564188 21069 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:31.561664 21035 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:31.540336 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.563022 22842 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:31.571662 22746 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:31.543712 21066 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:31.572669 22769 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.595044 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.595618 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.595752 21063 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:31.596607 21063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:31.597141 21063 raft_consensus.cc:606] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:31.597235 21070 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:31.597523 21063 raft_consensus.cc:614] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:31.598645 21063 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:31.589941 22574 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:31.578258 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:31.579231 22706 consensus_queue.cc:1243] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:31.599647 22574 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.604476 22574 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:31.575212 22817 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.606092 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:31.604477 22769 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.579615 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.579846 22830 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:31.608063 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 433) took 1001 ms. Trace:
I20260709 13:06:31.608485 21066 rpcz_store.cc:276] 0709 13:06:30.606595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.634318 (+ 27723us) service_pool.cc:224] Handling call
0709 13:06:31.536768 (+902450us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:31.543470 (+  6702us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:31.601001 (+ 57531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.608006 (+  7005us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8372608}
I20260709 13:06:31.580446 22838 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:31.609424 22838 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:31.609874 22714 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:31.556911 22796 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:06:31.590260 22777 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:31.611622 22777 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.612107 22777 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.612334 22838 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.613559 22838 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.582693 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.618125 21065 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:31.551033 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.619553 21069 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:31.621574 21069 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:31.622135 21138 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:31.589309 22676 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.593901 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.593919 22648 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:31.544480 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:31.633411 21072 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:31.601832 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.601941 21070 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:31.614637 21145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c20ca56847f64504a85694d10f47558e"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:06:31.637732 21145 raft_consensus.cc:606] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:31.638257 21145 raft_consensus.cc:614] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:31.611825 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:31.590706 22746 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:31.531854 22757 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.592010 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:31.621680 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56659us
W20260709 13:06:31.642503 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49750.9us
I20260709 13:06:31.644867 21144 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:31.642942 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.648312 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:31.615710 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.646586 22714 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:31.645898 22746 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.644184 22791 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:31.633314 22842 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:06:31.656666 22769 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.660765 22476 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:31.667620 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:31.681798 22476 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:31.670271 22830 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:06:31.682925 22476 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.684571 22830 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:31.675369 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:06:31.686635 22853 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:06:31.688005 22852 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:06:31.645366 22706 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:06:31.656809 22649 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
W20260709 13:06:31.689503 22857 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:06:31.665833 22807 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:06:31.696905 22806 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:31.710229 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 284385us
I20260709 13:06:31.712574 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:31.707845 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44866.8us
I20260709 13:06:31.721594 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:31.703699 22714 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:31.737524 22649 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:31.737545 22749 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.737821 22842 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:31.741202 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.674403 22676 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.758920 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.738801 22574 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.761305 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.755846 22749 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.768697 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.750420 22649 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:31.763213 22812 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:06:31.737712 22562 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:06:31.724017 22744 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:06:31.769893 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:31.781468 21187 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.762620 22759 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:31.772804 22870 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:06:31.781626 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60147.9us
I20260709 13:06:31.794215 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.749619 22807 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:06:31.765300 22776 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.765471 22796 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:31.796875 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.779903 22802 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.801625 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:31.803256 22769 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:31.803874 22806 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.795552 22870 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:31.804473 22874 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:06:31.751070 22816 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:06:31.805886 22757 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.807665 22749 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.808261 22845 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:31.780121 22778 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.798099 22790 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:31.795876 22714 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.831460 22602 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:31.768126 22788 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.811512 22485 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:31.828035 22562 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.827844 22807 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:31.821169 22790 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.828899 22649 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.824249 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.804481 22870 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.809994 22648 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:31.848109 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.833031 22806 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:31.812872 22786 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:31.903406 22786 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:31.903857 22786 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:31.888926 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:31.904874 21218 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:31.906205 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:31.907400 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:31.908457 21142 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:31.883942 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59830.4us
I20260709 13:06:31.885730 21219 raft_consensus.cc:2368] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:31.892165 22574 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:31.884706 22562 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.894869 22788 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:31.899955 22865 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:31.901871 21213 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:06:31.884289 22790 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:31.887936 22641 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:31.918936 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:31.919109 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:31.925549 22562 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.913770 22648 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:31.930243 22641 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:31.932300 22651 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:31.934033 22651 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.952109 22809 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:31.952607 22809 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.953485 22809 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.953869 20968 catalog_manager.cc:5697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:06:31.955358 20968 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 3, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 3 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:31.933292 22446 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:31.957111 22783 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:31.957489 22446 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:31.957664 22783 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.958510 22446 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.959507 22446 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:31.961226 22783 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:31.964694 22808 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.935071 22651 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.965212 22808 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.973973 22797 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.975986 22791 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:31.976576 22791 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.977083 22706 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:31.977622 22791 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:31.964694 22783 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:31.979521 22783 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.977640 22706 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.983170 22791 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.984282 22811 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:31.984728 22811 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:31.985788 22811 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:31.990571 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:31.961836 20968 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 3 to 4, term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to <none>, VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) evicted. New cstate: current_term: 2 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:31.991631 22651 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.993813 22783 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:31.995085 22808 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:31.992437 20968 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.999704 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:32.000914 21035 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:32.004042 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:31.925737 22602 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:31.924049 22485 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.926661 22787 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.924065 22714 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
W20260709 13:06:32.013233 22871 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:06:32.007128 22572 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:06:32.018077 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:32.005826 22749 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:32.019749 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:32.030280 21143 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.997720 20968 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:32.019340 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:32.031133 21218 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:32.039120 20968 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:32.040812 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 380977us
I20260709 13:06:32.041199 20968 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:32.034546 22749 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:32.045364 22749 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.043646 20968 catalog_manager.cc:5697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:06:32.051349 21067 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:06:32.051906 21067 pending_rounds.cc:85] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 1
I20260709 13:06:32.052155 21067 pending_rounds.cc:107] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:06:32.052382 21067 raft_consensus.cc:2889] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:32.053287 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:32.048632 20968 catalog_manager.cc:5697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 3, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 3 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:32.057796 21069 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:32.032941 22809 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:32.009478 22602 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:32.056393 20968 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:32.067262 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.085517 21143 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
W20260709 13:06:32.088408 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:32.085963 22878 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:06:32.096818 21792 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:32.099174 22761 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:32.116060 22706 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:32.116557 22761 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:32.120837 22761 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:32.125595 22811 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:32.126700 22777 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.002s
I20260709 13:06:32.116540 22602 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:32.127239 22777 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.132665 22797 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:32.134130 22485 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:32.136085 22870 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.138372 22870 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:32.140378 22706 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:32.129993 22777 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:32.177707 22786 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:32.191067 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 409280us
I20260709 13:06:32.199460 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:32.192852 22790 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.203191 22790 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:32.203737 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:32.204851 22786 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:32.205291 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:32.208482 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:32.209677 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:32.215557 22811 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:32.216759 22791 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:32.217207 22791 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:32.181546 22835 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.218032 22791 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:32.228245 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 444517us
I20260709 13:06:32.225451 22791 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:32.249512 21070 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:32.222505 22835 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.278704 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:32.279809 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:32.283322 22745 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:32.279870 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50326.8us
I20260709 13:06:32.284849 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:32.285633 21792 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:32.287376 22835 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:32.291903 21141 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:32.294787 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:32.295619 22676 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.296346 22676 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:32.297626 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:32.298633 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:32.299218 21142 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:32.300642 22765 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:32.301045 22765 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:32.302927 21799 raft_consensus.cc:2368] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:32.226783 22787 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:06:32.251890 22649 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:32.310372 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.314913 21142 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:32.318104 21069 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:32.242970 22840 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:06:32.305245 22882 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:32.337966 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 407431us
I20260709 13:06:32.322528 22485 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:32.333698 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:32.337339 21793 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:32.351389 22788 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:32.337765 22882 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.344314 22597 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.310376 21110 leader_election.cc:400] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:32.360225 21110 leader_election.cc:403] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:06:32.362088 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:32.362713 21071 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:32.363742 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:32.361532 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51371us
I20260709 13:06:32.366519 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:32.367331 21068 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:32.357396 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:06:32.370183 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.371274 21140 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:32.371876 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:32.327566 22786 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:32.318575 22877 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:32.329926 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:32.280827 22668 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:32.329973 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:32.330577 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:32.365469 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:32.385308 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.390273 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:32.381946 21140 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:32.389384 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:32.375917 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 456873us
I20260709 13:06:32.343559 22676 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:32.357198 22882 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:32.392885 21069 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:32.394732 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:32.395612 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:32.398322 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:32.398684 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:32.408247 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:32.359782 22744 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.411109 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:32.407102 21220 raft_consensus.cc:2376] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4. Re-sending same reply.
I20260709 13:06:32.413749 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:32.415889 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:32.416618 21218 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:32.415096 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:32.404093 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 74228.5us
I20260709 13:06:32.419459 21217 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:06:32.373768 22706 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:32.435341 22890 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:06:32.402673 22887 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:06:32.437227 22765 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.438449 22765 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:32.441412 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:32.442257 21214 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:32.433457 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:32.443871 22788 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:32.444332 22788 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.420254 22770 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:06:32.444680 22788 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:32.446900 22770 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:32.429373 22744 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.448784 22885 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.449358 22885 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.424981 22562 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:32.388190 22597 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.401782 22574 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:32.499886 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:32.379812 22884 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.359071 21792 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:32.394354 22572 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.508786 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:32.402587 22870 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.432116 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:32.392293 22877 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:32.394490 22886 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:32.425705 21216 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:06:32.459762 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:32.418306 21220 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:32.404390 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:32.474277 21218 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:32.407320 22786 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:32.401376 21064 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:32.401470 22668 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.511269 22889 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:06:32.532336 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 128065us
I20260709 13:06:32.533325 22885 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:32.534385 22891 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:32.498368 22890 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.536399 22485 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:32.465546 22770 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.537240 22770 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:32.537343 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:32.537971 21792 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:32.452178 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:32.539599 21219 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:32.528944 22572 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:32.541564 22572 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:32.535002 22714 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:32.451232 22744 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:32.402719 22769 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.543670 21143 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:32.515848 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:32.633288 21147 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:32.510046 21796 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:32.530468 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:32.634402 21218 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:32.634670 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.635196 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:32.547041 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:32.536115 22890 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:32.604692 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55582.4us
I20260709 13:06:32.534600 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:32.457734 22597 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:32.406152 22710 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.225s
I20260709 13:06:32.630510 21216 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:32.513581 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:32.651190 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 104255us
I20260709 13:06:32.659792 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:32.651190 21218 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
W20260709 13:06:32.679478 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 144945us
I20260709 13:06:32.630678 22889 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:32.689465 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.692798 21111 leader_election.cc:400] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:32.693600 21111 leader_election.cc:403] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:06:32.509235 22574 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.530031 22870 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:32.636072 22886 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:32.694821 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:32.697995 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 47272.3us
I20260709 13:06:32.698714 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:32.700490 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:32.543668 22786 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:32.706959 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.677402 22651 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:32.678067 22887 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:32.712286 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.530189 22884 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:32.714298 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.679550 22894 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:32.724987 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:32.725611 21140 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:32.638227 22769 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:32.709184 22714 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:32.713049 22485 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:32.708869 22874 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:32.711846 22836 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:32.688258 21214 raft_consensus.cc:2376] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4. Re-sending same reply.
I20260709 13:06:32.717746 22651 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.742898 22651 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:32.697219 22706 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:06:32.726876 22894 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:32.737504 22746 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.065s	user 0.008s	sys 0.006s
I20260709 13:06:32.728317 22835 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:32.753019 22893 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:06:32.727896 22887 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.761374 22668 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:32.764482 22870 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:32.764736 22884 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:32.764979 22889 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.766215 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 451) took 1188 ms. Trace:
I20260709 13:06:32.739619 22485 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:32.749534 22836 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:32.798768 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:32.790509 22788 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.764472 22890 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:32.802536 22791 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.551s	user 0.000s	sys 0.009s
I20260709 13:06:32.770334 22641 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:32.805600 21219 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:32.786500 21066 rpcz_store.cc:276] 0709 13:06:31.578018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.112987 (+534969us) service_pool.cc:224] Handling call
0709 13:06:32.113444 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.113889 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.113946 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:32.114505 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:06:32.532367 (+417862us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:32.635705 (+103338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.647682 (+ 11977us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.765962 (+118280us) raft_consensus.cc:1685] finished
0709 13:06:32.765991 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.766142 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6531,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":140,"thread_start_us":394883,"threads_started":1,"wal-append.queue_time_us":597711}
I20260709 13:06:32.930121 22574 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:32.810427 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 402260us
W20260709 13:06:32.907274 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.767323 22894 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:32.815980 22485 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:32.916841 21070 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.006s	sys 0.015s
I20260709 13:06:32.923256 21795 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:32.923669 22888 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:32.923770 22649 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.003s
I20260709 13:06:32.923735 22476 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:32.927868 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:32.933173 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:32.773047 22746 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.949995 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:32.906875 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 481496us
I20260709 13:06:32.951166 21799 raft_consensus.cc:2410] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:06:32.939822 21221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.011s	sys 0.008s
W20260709 13:06:32.951967 21221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.011s	sys 0.008s
W20260709 13:06:32.946842 21071 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.004s	sys 0.016s
I20260709 13:06:32.775504 22835 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:06:32.952857 21071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.005s	sys 0.016s
I20260709 13:06:32.951217 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:32.751062 22886 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.944571 22890 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:32.943636 22668 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:32.965438 21800 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:32.945387 21792 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
W20260709 13:06:32.965914 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 255660us
I20260709 13:06:32.919461 22882 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.955000 22904 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:32.977757 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:32.979010 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:32.963425 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:32.950567 22476 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:32.961848 22602 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:32.961601 22892 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:32.959673 22877 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:32.964834 22791 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.007766 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:33.009095 22913 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:33.009384 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:33.011094 22745 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:33.028646 22770 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.030649 21212 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:32.950361 22888 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:32.985886 22892 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.985153 22786 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:32.971259 22890 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:32.972470 22668 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.019768 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:33.046270 21801 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:33.023434 22783 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:32.968750 22836 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.037397 22892 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:33.039572 22648 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.025425 22791 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.052635 22783 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:33.048871 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.054098 22915 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:33.079312 21214 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:33.079573 22882 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.081007 22602 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.081306 22890 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.081610 22602 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.076449 22783 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.078877 22887 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:33.079491 22915 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.054047 21795 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:33.108996 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:33.109789 21147 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:33.115366 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:33.049139 22476 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.091950 22788 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.092913 22745 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:33.124949 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.103554 22887 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:33.068794 22648 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.097550 22913 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:33.126677 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 47971.8us
I20260709 13:06:33.110880 22890 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:33.124636 22651 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.129595 22887 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.131778 22890 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.129408 22745 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:33.135708 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.127203 22770 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.144080 22888 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.146289 22915 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:33.147517 21142 raft_consensus.cc:2368] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:33.146853 22651 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.149174 22745 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:33.151631 22791 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.152208 22648 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.156893 21800 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
W20260709 13:06:33.156602 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50723.5us
I20260709 13:06:33.161301 22894 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.163621 21147 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:33.159518 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:33.147529 22835 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:33.164780 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:33.171859 22744 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:06:33.165638 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.172287 22744 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.166500 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:33.172653 22744 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.173327 22651 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:33.173751 22651 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.174082 22651 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:33.175709 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:33.182657 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:33.186028 21141 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:33.191260 21221 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:33.195443 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:33.198405 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.200273 21218 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:33.206619 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:33.211474 22894 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.213297 21220 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:33.200273 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.216523 21216 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:33.217617 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.218341 21216 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:33.214418 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.221608 21220 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:33.225504 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:33.232635 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:33.233228 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.234071 21221 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:33.235623 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.156409 22913 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.235148 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:33.268124 21221 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:33.274081 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:33.276233 21220 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:06:33.220284 22446 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.189441 22791 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:33.280234 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:33.286810 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 47891.7us
I20260709 13:06:33.283878 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.271346 22778 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:33.275262 22886 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.271162 22777 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.220844 22485 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.277547 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:33.283165 22562 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:33.296128 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.328922 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:33.295687 22597 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:33.293334 22797 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:06:33.334090 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:33.291349 22446 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.307827 22886 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.352264 22886 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.303731 22926 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.352881 22835 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:33.353391 22835 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.353044 22926 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.353788 22835 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:33.356938 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:33.357591 21794 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:33.345105 21066 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:33.358466 22890 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.360679 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:33.361501 21800 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:33.310348 22777 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.366107 22777 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.366729 21070 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:33.334734 22706 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:33.373350 22706 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.373201 22783 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.373885 22706 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.374182 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.375434 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.376015 21219 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:33.376245 22887 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.327894 22757 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:33.377035 22757 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.377046 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.343956 22790 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:33.379717 22917 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.283s	user 0.004s	sys 0.000s
I20260709 13:06:33.380295 21219 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:33.343675 22874 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:33.382972 22485 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:33.383083 22874 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:33.384147 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:33.335800 22906 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:06:33.341817 22597 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.385243 22906 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:33.348575 22870 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:33.385133 22651 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.385622 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:33.385835 22870 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:33.386426 22597 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.386790 22870 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.348953 21798 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:06:33.336822 22892 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.350283 22446 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.399530 21220 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:33.379843 22790 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.292891 22791 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.388307 22744 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.401633 22790 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.401670 22791 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.381162 21796 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:06:33.401964 22892 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:33.402155 22744 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.403129 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:33.403473 22890 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.403806 21796 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:33.404042 22890 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.385658 22906 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:33.403409 21138 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:33.405637 22906 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.406131 22890 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.406854 22906 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.391335 22757 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.403424 21799 raft_consensus.cc:2368] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:33.403138 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:33.408892 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:33.343160 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:33.342520 21216 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:33.388012 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 547) took 1008 ms. Trace:
I20260709 13:06:33.388092 22870 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.391525 21798 pending_rounds.cc:85] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 4
I20260709 13:06:33.384045 22874 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:33.411420 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 68424.3us
I20260709 13:06:33.411993 21800 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:33.411232 21801 rpcz_store.cc:276] 0709 13:06:32.379503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.381929 (+  2426us) service_pool.cc:224] Handling call
0709 13:06:32.927721 (+545792us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:33.387805 (+460084us) spinlock_profiling.cc:243] Waited 97.1 ms on lock 0x7b54002eaed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:33.387943 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227503744}
I20260709 13:06:33.415174 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:33.334640 21795 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:06:33.431059 21221 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:33.386873 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:33.329524 22562 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.439033 21220 raft_consensus.cc:2368] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:33.433184 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.476512 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.438853 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:33.478513 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:33.481981 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:33.486795 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:33.387272 22651 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.490440 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:33.488780 22761 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:33.489532 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:33.490368 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.488406 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:33.440040 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:33.492161 21800 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:33.492744 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.437553 21798 pending_rounds.cc:107] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:06:33.433980 22714 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.433568 22757 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:33.494948 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:33.496718 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.496718 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:33.436640 22783 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.439365 22791 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:33.519675 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.433717 22744 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:33.502912 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:33.490882 22890 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:33.523258 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:33.523617 21221 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:33.524410 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:33.524907 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.486603 22874 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.518105 21220 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.494134 22884 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:06:33.516573 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.498716 22907 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:33.489547 22597 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:33.534317 22597 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.536916 22597 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.538532 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.522248 22877 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:33.539716 22877 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:33.540679 22877 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.522457 22777 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:33.543215 22668 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.544073 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.546439 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.548520 21216 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:33.548668 22746 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.526582 21221 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:33.517917 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:33.522210 22921 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.514798 21798 raft_consensus.cc:2889] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:33.553846 21111 leader_election.cc:400] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:33.567070 21221 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:06:33.558116 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.556149 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.568503 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.563014 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:33.565704 21220 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:33.561985 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:33.569036 21069 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:33.522866 22757 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.527025 22890 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.513904 22714 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.522624 22744 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.580329 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.582144 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:33.579741 21221 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:06:33.585311 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.585037 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:33.584443 21216 raft_consensus.cc:2368] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:33.604630 21765 leader_election.cc:400] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
I20260709 13:06:33.605126 21765 leader_election.cc:403] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:06:33.552038 22884 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.605923 22649 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3.
W20260709 13:06:33.606621 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.606813 21219 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:33.607887 22714 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.600385 22574 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:33.608780 22574 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:33.609598 22574 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.610146 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:33.610417 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:33.610841 21792 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:33.610924 22769 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:33.611233 22904 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:33.613131 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:33.590161 22668 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:33.614082 22668 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:33.614642 22668 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:33.615584 22668 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.617231 22668 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:33.565841 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 76970.7us
I20260709 13:06:33.593201 22676 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.619822 22907 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:33.619990 22676 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.620926 22907 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.622745 22676 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.592928 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.570108 22921 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.495985 22761 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.625533 22921 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:33.625777 22761 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.597751 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1474) took 1224 ms. Trace:
I20260709 13:06:33.626611 21218 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:33.566125 22933 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:06:33.613392 22891 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.627593 22891 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.628007 21068 rpcz_store.cc:276] 0709 13:06:32.372923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.403228 (+ 30305us) service_pool.cc:224] Handling call
0709 13:06:32.927122 (+523894us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:32.929134 (+  2012us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.221312 (+292178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.221341 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.221781 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.431664 (+209883us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:33.487640 (+ 55976us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.487906 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.487975 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.551364 (+ 63389us) raft_consensus.cc:1685] finished
0709 13:06:33.597527 (+ 46163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.597684 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":296061,"prepare.queue_time_us":57506,"raft.queue_time_us":342833,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":20157,"spinlock_wait_cycles":47747456,"thread_start_us":144918,"threads_started":1,"wal-append.queue_time_us":223617}
I20260709 13:06:33.629946 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.631058 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.631963 21068 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:06:33.592887 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:33.635775 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.566709 22809 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:33.565678 22796 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:33.626737 22706 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.641347 21216 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:33.528084 22791 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.639266 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.646631 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:33.647202 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 121655us
I20260709 13:06:33.597289 22757 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:33.597649 22890 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.604418 22744 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:33.555034 22943 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:33.744863 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:33.753763 21068 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:33.636446 22838 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:33.570143 22892 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 13:06:33.760167 21144 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:33.766875 21138 raft_consensus.cc:2410] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:33.767397 21221 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:33.652070 22651 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.648315 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 550) took 1260 ms. Trace:
I20260709 13:06:33.705237 22446 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:33.735512 22710 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:33.735845 22777 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:33.650491 22921 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:33.754350 22943 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:33.810330 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67646.5us
I20260709 13:06:33.815502 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:33.630159 22835 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.754680 22572 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.650146 22941 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:33.753623 22830 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:06:33.775172 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:33.839314 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 545) took 1462 ms. Trace:
W20260709 13:06:33.840167 22940 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:06:33.844817 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:33.849857 21796 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:33.815806 22446 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.869762 22446 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.871182 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:33.872156 22889 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:33.872706 22889 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:33.872767 21795 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:33.873701 22889 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.815070 22710 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.870991 22446 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:33.820288 22943 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:33.811185 22651 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.885365 22749 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.885898 22791 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.885921 22749 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.879525 22676 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:06:33.888098 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1471) took 1523 ms. Trace:
I20260709 13:06:33.888180 22891 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:33.863863 22597 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:33.778436 22797 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:33.813110 22921 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.811019 22838 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:33.813212 22814 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.863863 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2200) took 1464 ms. Trace:
I20260709 13:06:33.866608 22769 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.866768 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.867076 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.758445 22884 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.860782 22790 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.888550 21072 rpcz_store.cc:276] 0709 13:06:32.364193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.389299 (+ 25106us) service_pool.cc:224] Handling call
0709 13:06:33.162631 (+773332us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:33.162875 (+   244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.163323 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.163363 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.163889 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.473070 (+309181us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:33.516834 (+ 43764us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.517106 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.517249 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.887819 (+370570us) raft_consensus.cc:1685] finished
0709 13:06:33.887906 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.888038 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135285,"prepare.queue_time_us":43510,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":4435,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":7038,"spinlock_wait_cycles":4549888,"thread_start_us":308770,"threads_started":1,"wal-append.queue_time_us":433333}
I20260709 13:06:33.898633 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.876775 22446 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:33.862413 22562 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:33.885878 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 204203us
I20260709 13:06:33.894586 21212 rpcz_store.cc:276] 0709 13:06:32.399801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.429209 (+ 29408us) service_pool.cc:224] Handling call
0709 13:06:32.634980 (+205771us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:33.863659 (+1228679us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b540015d454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:33.863804 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":920380032}
I20260709 13:06:33.895179 21217 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:33.836376 22572 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.875345 21792 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:33.815312 22777 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.888793 22749 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.916031 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:33.916750 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2202) took 1516 ms. Trace:
I20260709 13:06:33.924218 21217 rpcz_store.cc:276] 0709 13:06:32.400286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.464928 (+ 64642us) service_pool.cc:224] Handling call
0709 13:06:33.866937 (+1402009us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:33.916686 (+ 49749us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3433344}
I20260709 13:06:33.774930 21798 rpcz_store.cc:276] 0709 13:06:32.387885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.389928 (+  2043us) service_pool.cc:224] Handling call
0709 13:06:33.348392 (+958464us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:33.348605 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.598018 (+249413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.645549 (+ 47531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23475968}
W20260709 13:06:33.925396 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.900132 22814 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.919211 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:33.926262 22791 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:33.858528 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:33.893947 22921 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.863567 22941 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.927325 22882 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:33.927706 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:33.928068 21072 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:33.926824 22791 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.928754 22941 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:33.928820 22791 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:33.895689 22597 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:33.896346 22809 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:33.921178 22649 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:33.915216 22927 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:33.971304 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:33.896737 22797 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.977906 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:33.930919 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 72486.7us
I20260709 13:06:33.933807 22749 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:33.915385 22790 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:33.990020 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.991511 22648 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:33.991685 22597 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:33.996853 22649 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.001259 21795 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:06:33.990986 21064 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:34.008772 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.009580 21064 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:34.009887 22651 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:34.010309 21794 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.652s	user 0.012s	sys 0.000s
I20260709 13:06:34.010676 21794 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:34.012477 22777 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:34.016219 22906 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.000s
I20260709 13:06:34.010877 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.018044 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.028463 21072 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:34.029794 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:33.899760 22769 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.025640 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.025827 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.018918 22602 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.026674 22887 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.033540 22887 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.034166 22769 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.035606 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.036366 21795 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:34.037667 22710 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.861222 21797 rpcz_store.cc:276] 0709 13:06:32.376953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.394092 (+ 17139us) service_pool.cc:224] Handling call
0709 13:06:33.349477 (+955385us) spinlock_profiling.cc:243] Waited 66.3 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:33.349698 (+   221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.349962 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:33.532853 (+182891us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.777908 (+245055us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:33.809662 (+ 31754us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:33.818601 (+  8939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.830055 (+ 11454us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.839184 (+  9129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":253701,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":7511,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":9620,"spinlock_wait_cycles":152527232,"thread_start_us":360704,"threads_started":2,"wal-append.queue_time_us":344488}
I20260709 13:06:34.042363 22744 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.001319 22913 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.043902 22957 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.044595 22957 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.045464 22958 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.045853 22874 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:34.046000 22958 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.046381 22874 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.047768 21799 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:34.048563 22790 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.049119 22476 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.050310 22957 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:34.050451 22870 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.613s	user 0.005s	sys 0.004s
I20260709 13:06:34.050915 22649 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.027107 22838 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.053376 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:33.984890 22706 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.029219 22777 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:34.055480 22885 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:34.056288 22777 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.056365 22885 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.058411 22777 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.058570 22885 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:33.985654 22749 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.059247 22749 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.060155 22925 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:34.060467 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2193) took 1663 ms. Trace:
I20260709 13:06:34.062307 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.059160 22885 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:34.063223 21063 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:34.060705 22925 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:34.063840 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 114471us
W20260709 13:06:34.065654 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 326054us
I20260709 13:06:34.067781 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.072962 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.071715 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.074196 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.074092 21143 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:34.080825 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.080673 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.084756 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.085803 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:34.084794 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:34.086827 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.088312 21140 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:34.065841 22925 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.066023 21064 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:34.090822 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.063405 22706 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:34.092450 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:34.092864 22574 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.093032 22957 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.021241 22927 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.093412 22830 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:34.093297 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.095237 22797 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:33.979511 22446 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:34.098330 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:34.099313 21068 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:34.068734 22757 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.049120 22874 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:34.147204 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 48955.2us
I20260709 13:06:34.130726 21139 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:34.148418 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.148496 21035 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:34.149322 22778 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:34.149566 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.068326 21185 leader_election.cc:400] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:34.150279 21185 leader_election.cc:403] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:34.107497 21144 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:34.150359 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.150486 22778 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:34.151475 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.097419 22870 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.133507 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.153024 21068 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:34.153908 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.134573 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:34.157509 22884 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:34.158304 22926 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.159354 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.159695 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:34.160099 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 131) took 1126 ms. Trace:
I20260709 13:06:34.160487 21215 rpcz_store.cc:276] 0709 13:06:33.033802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.033933 (+   131us) service_pool.cc:224] Handling call
0709 13:06:34.159180 (+1125247us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:34.160043 (+   863us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14341376}
I20260709 13:06:34.161684 22962 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:34.162094 22962 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:34.162451 22962 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.100253 22777 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:34.165117 22835 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.155051 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.166043 21068 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:34.100176 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:34.167397 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.100584 22796 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:34.167783 21142 raft_consensus.cc:2410] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:06:34.168605 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.168406 22796 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.169157 21142 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
W20260709 13:06:34.170265 22944 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:06:34.103157 22562 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:34.103673 22955 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:34.171480 22562 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.171602 22955 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:34.103534 22676 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.145478 22814 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.108659 21146 raft_consensus.cc:2368] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:34.125926 21184 leader_election.cc:400] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:34.179894 21184 leader_election.cc:403] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:34.128145 22574 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.151904 22948 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:34.158092 22884 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.167558 21067 raft_consensus.cc:2368] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:34.061170 21213 rpcz_store.cc:276] 0709 13:06:32.396648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.401937 (+  5289us) service_pool.cc:224] Handling call
0709 13:06:32.627957 (+226020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.713128 (+ 85171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.717905 (+  4777us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:32.736909 (+ 19004us) log.cc:844] Serialized 26 byte log entry
0709 13:06:32.972271 (+235362us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:33.070737 (+ 98466us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.226114 (+155377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.252252 (+ 26138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.379058 (+126806us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:33.391613 (+ 12555us) raft_consensus.cc:1685] finished
0709 13:06:33.399547 (+  7934us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.060079 (+660532us) spinlock_profiling.cc:243] Waited 89.1 ms on lock 0x7b54000bc7d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:34.060395 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53000,"prepare.queue_time_us":56304,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":110260,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":30300,"spinlock_wait_cycles":221123840,"thread_start_us":387757,"threads_started":3,"wal-append.queue_time_us":316834}
I20260709 13:06:34.172564 21144 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:34.108355 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:34.173017 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 142293us
I20260709 13:06:34.179117 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.137636 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.138249 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.195219 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.182091 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.152393 22930 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 13:06:34.137405 22950 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:06:34.178580 22955 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.187460 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.144279 22887 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.145006 22749 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.184028 22710 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:34.187234 22948 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:34.108565 22907 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:34.182101 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 71052.8us
I20260709 13:06:34.135711 22958 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.143505 22927 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.229363 22921 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.177290 22769 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:34.199887 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1466) took 1837 ms. Trace:
I20260709 13:06:34.190122 22874 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:34.197788 22964 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:34.171974 22562 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.216776 22948 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.141904 22890 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.132031 22957 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.152393 22870 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.189486 22923 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:34.219434 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.220445 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:34.278946 21215 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.221212 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.221848 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.281095 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.222357 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:34.222436 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.282013 21216 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:34.282642 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.218321 22959 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:34.283972 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.285243 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.224112 22941 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:34.226392 22749 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:34.226876 22910 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:06:34.234683 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.141911 22943 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.298164 21145 raft_consensus.cc:2368] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:34.236277 21068 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:34.198184 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.239770 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:34.101217 22809 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.256408 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:34.275547 22783 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:34.182585 22676 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.205955 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.135898 22830 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.192065 22777 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:34.217911 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:34.223487 22926 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:34.286438 21212 raft_consensus.cc:2368] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:34.289671 21221 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:34.337762 21144 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:06:34.338858 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.290580 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.258538 22927 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:34.296350 21219 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:34.343366 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.297369 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.238437 22791 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.300439 21139 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:34.308166 22964 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:34.280942 22874 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:34.362780 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.345659 22884 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.363442 21067 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:34.363890 22874 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:34.302089 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45792.7us
I20260709 13:06:34.257033 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.369004 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.305207 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:34.369900 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 112981us
I20260709 13:06:34.323678 21069 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:34.370602 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.371735 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.302559 21143 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:34.372416 21069 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:34.315407 22783 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.373216 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.374306 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.374804 22783 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.344349 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.263368 21064 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:34.304787 22925 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.258491 21065 rpcz_store.cc:276] 0709 13:06:32.362799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.372273 (+  9474us) service_pool.cc:224] Handling call
0709 13:06:32.424963 (+ 52690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.529230 (+104267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.531085 (+  1855us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:32.535933 (+  4848us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.160934 (+625001us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b1800154060. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f463f058f 00007f7f3f42f571 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd
0709 13:06:33.367944 (+207010us) spinlock_profiling.cc:243] Waited 70 ms on lock 0x7b1800154060. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:06:33.569598 (+201654us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:33.583995 (+ 14397us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.651745 (+ 67750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.735513 (+ 83768us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.012712 (+277199us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b1800154060. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:06:34.169681 (+156969us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:34.182494 (+ 12813us) raft_consensus.cc:1685] finished
0709 13:06:34.184960 (+  2466us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.199807 (+ 14847us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184181,"prepare.queue_time_us":14463,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":119249,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":73627,"spinlock_wait_cycles":270354176,"thread_start_us":184502,"threads_started":1,"wal-append.queue_time_us":1169279}
I20260709 13:06:34.355660 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1470) took 1991 ms. Trace:
I20260709 13:06:34.358580 22890 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.394193 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.359172 21217 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:34.360850 22870 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.337273 22777 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.393374 21071 rpcz_store.cc:276] 0709 13:06:32.363960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.364519 (+   559us) service_pool.cc:224] Handling call
0709 13:06:32.364789 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.365214 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.365252 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:32.387780 (+ 22528us) log.cc:844] Serialized 26 byte log entry
0709 13:06:32.955837 (+568057us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:32.970178 (+ 14341us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.271628 (+301450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.271712 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.007963 (+736251us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b18002465e0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:06:34.226035 (+218072us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:34.355394 (+129359us) raft_consensus.cc:1685] finished
0709 13:06:34.355428 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.355594 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":313274,"prepare.queue_time_us":14400,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":315178,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":30724,"spinlock_wait_cycles":63534208,"thread_start_us":526758,"threads_started":1,"wal-append.queue_time_us":600587}
I20260709 13:06:34.350314 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.302443 22744 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.375373 22904 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:34.375823 22641 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:34.371748 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:34.376724 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:34.378182 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.409066 21068 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:06:34.292996 22959 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.380314 22830 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.381848 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:34.382205 22955 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.383492 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.350330 22927 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.386132 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.448500 21799 raft_consensus.cc:2368] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:34.341325 21216 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:34.390596 21212 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:34.392163 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:34.405014 22574 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.397275 21215 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:34.417481 22648 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:34.382640 22967 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:34.417714 21063 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:34.345357 22921 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.454875 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.455607 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.456282 21063 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.456416 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.400593 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.252831 22710 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.410785 22904 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:34.408902 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.377758 22962 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.418581 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:34.411931 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:34.412544 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 124889us
I20260709 13:06:34.423484 21185 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:06:34.483222 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 59796.8us
I20260709 13:06:34.484743 21185 leader_election.cc:400] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:34.485121 21185 leader_election.cc:403] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:06:34.400527 22877 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:34.350835 22964 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.486382 22877 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:34.486529 22964 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.486958 22877 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.393118 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.488816 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.489506 21066 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.425109 21069 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:34.491552 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.435499 22922 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:34.436787 22891 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:34.416129 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.437717 22974 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:34.495869 22974 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.441388 22955 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.442107 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.442889 22572 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.444046 21797 raft_consensus.cc:2368] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:34.444633 22777 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.403757 22933 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:34.452528 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 60439.8us
I20260709 13:06:34.395401 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.454198 22765 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:34.454810 22967 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:34.512650 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 117341us
I20260709 13:06:34.515028 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.458397 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.399550 21067 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
W20260709 13:06:34.465885 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 57066us
I20260709 13:06:34.413496 22796 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:34.457181 21065 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:34.454474 22648 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.485212 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.499569 21218 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
W20260709 13:06:34.500643 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 82172.1us
I20260709 13:06:34.501067 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.494781 22891 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:34.529381 21110 leader_election.cc:400] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:34.498756 22602 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:34.455890 22791 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.508571 21036 leader_election.cc:400] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:34.454841 22714 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:34.518342 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.520216 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:34.483408 22904 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.490803 22921 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:34.522852 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.524603 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.537683 21036 leader_election.cc:403] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:34.537582 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.538058 21145 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:34.525446 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.538440 21140 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.526274 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.528352 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.540580 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.542296 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.529999 21110 leader_election.cc:403] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:34.530131 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45020.7us
I20260709 13:06:34.545910 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:34.546516 21185 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:34.498332 22974 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:34.551198 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.551841 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.552369 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:34.509573 22641 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:34.531313 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.512180 22765 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.532320 21141 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:34.528228 22676 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.553541 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:34.543061 21067 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:34.530004 22648 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.532063 22602 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.529156 22796 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.531647 22891 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:34.521061 22933 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:34.586217 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52323.2us
I20260709 13:06:34.587693 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:34.554625 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.571511 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:34.548976 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.571889 22978 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:34.537441 22714 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:34.573045 22956 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:34.573168 22959 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:34.574182 22976 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:06:34.574553 22957 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.575654 22962 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.549530 22887 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:34.534096 22809 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:34.577719 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.578013 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:34.578356 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:34.611143 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64713.8us
I20260709 13:06:34.565075 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:34.578745 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.548383 22921 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.582517 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.601894 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.614394 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:34.615437 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50422.3us
I20260709 13:06:34.617570 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:34.618865 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.604666 22956 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.526909 22830 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:34.584254 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:34.584885 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.552572 22476 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:34.588764 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.610196 22927 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.566244 22765 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.563468 22836 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:34.557358 22943 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.591919 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.576813 22791 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:34.575157 22967 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.581379 22602 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:34.577672 22572 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:34.641178 22562 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.584489 22923 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:34.642488 22923 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.642910 22923 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:34.625249 22830 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:34.643558 22830 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.643675 21069 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:34.601833 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.636650 22887 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:34.643733 22984 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
W20260709 13:06:34.605589 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 113510us
I20260709 13:06:34.592823 22744 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:34.594929 22925 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.611799 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.654445 21070 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.624666 22956 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:34.583029 21140 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:34.623430 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:34.625538 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:34.625706 21217 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:34.590673 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:34.653925 22562 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.619467 22955 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:34.614827 22921 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.614847 22574 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:34.620314 22957 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:34.633654 21767 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.620908 22890 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:34.625173 22710 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.638084 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.639730 22933 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.640493 21066 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:34.611982 21067 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:34.656116 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.630652 22676 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.662286 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.676707 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1490) took 1046 ms. Trace:
I20260709 13:06:34.662981 21219 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.664233 21216 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:34.625402 22714 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.632972 22836 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:34.631227 22476 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.667421 22955 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.632951 22791 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.666077 22870 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:34.677132 21067 rpcz_store.cc:276] 0709 13:06:33.630044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.578529 (+948485us) service_pool.cc:224] Handling call
0709 13:06:34.676642 (+ 98113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.683905 22676 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.729223 22676 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.685158 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.730441 21221 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:34.686044 22809 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:34.683352 21216 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:06:34.688179 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.672752 22744 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:34.686595 22836 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:34.715487 22955 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:34.680877 22890 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:34.733526 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.748894 21216 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:34.734293 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1481) took 1175 ms. Trace:
I20260709 13:06:34.738791 22877 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.740674 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1480) took 1182 ms. Trace:
I20260709 13:06:34.738833 22964 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.751324 21067 rpcz_store.cc:276] 0709 13:06:33.558464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.739638 (+1181174us) service_pool.cc:224] Handling call
0709 13:06:34.739971 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.740198 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.740241 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.740327 (+    86us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:34.740374 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.740456 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.740599 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.753257 21147 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.753748 22964 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.753150 22877 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.721777 22796 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.679571 22749 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
W20260709 13:06:34.760653 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 72619.4us
I20260709 13:06:34.767091 21036 leader_election.cc:400] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:34.767623 21219 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:34.768062 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.761693 21067 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:06:34.769482 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 464) took 1208 ms. Trace:
I20260709 13:06:34.691677 22933 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:34.770011 21067 rpcz_store.cc:276] 0709 13:06:33.561292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.752480 (+1191188us) service_pool.cc:224] Handling call
0709 13:06:34.752874 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.769196 (+ 16322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.769417 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.790568 22830 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.764858 21147 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:34.689141 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:34.750515 21069 rpcz_store.cc:276] 0709 13:06:33.558944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.577468 (+1018524us) service_pool.cc:224] Handling call
0709 13:06:34.734097 (+156629us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b54003a2fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:34.734235 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74754944}
I20260709 13:06:34.731222 22948 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.772851 21221 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:34.723023 22791 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.732429 22446 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.780336 22765 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.731484 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1483) took 1172 ms. Trace:
I20260709 13:06:34.789276 22891 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.734889 21065 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:34.800451 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:34.800218 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:34.884608 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 84483us
I20260709 13:06:34.801996 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 465) took 1240 ms. Trace:
I20260709 13:06:34.885689 21067 rpcz_store.cc:276] 0709 13:06:33.561514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.801037 (+1239523us) service_pool.cc:224] Handling call
0709 13:06:34.801383 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.801583 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.801617 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.801683 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:34.801722 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.801788 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.801931 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.778259 22641 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:34.765518 22943 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:34.820169 21795 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:34.889140 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.799381 22714 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:34.890467 21068 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:34.799263 22476 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.800681 22933 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:34.732262 22809 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.822465 21063 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:34.892463 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.814389 22830 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.899773 21070 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:34.891773 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 467) took 1329 ms. Trace:
I20260709 13:06:34.823870 21213 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:34.889652 22710 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:34.952865 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.978596 21070 raft_consensus.cc:2368] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
W20260709 13:06:34.939167 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46773.3us
I20260709 13:06:34.797629 22602 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.795269 22836 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.896566 22906 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:06:34.883116 22923 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.958629 22964 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.984411 22964 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:34.985774 22923 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:34.893503 22562 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:34.986433 22964 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.958786 22777 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:34.952859 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1485) took 1392 ms. Trace:
I20260709 13:06:34.954144 21067 rpcz_store.cc:276] 0709 13:06:33.561952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.890949 (+1328997us) service_pool.cc:224] Handling call
0709 13:06:34.891297 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.891463 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.891486 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.891528 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:34.891560 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.891609 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.891715 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:34.963912 22986 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:34.972124 22948 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:34.992619 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.972759 21796 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:34.998128 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:34.880621 21066 rpcz_store.cc:276] 0709 13:06:33.559520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.730578 (+1171058us) service_pool.cc:224] Handling call
0709 13:06:34.730897 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.731072 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.731102 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.731158 (+    56us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:34.731201 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.731264 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.731419 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.976559 22765 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:34.803324 22890 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.979681 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:34.819180 22648 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:34.981168 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:34.991701 21212 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:35.000658 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:34.991845 21065 rpcz_store.cc:276] 0709 13:06:33.560124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.734166 (+1174042us) service_pool.cc:224] Handling call
0709 13:06:34.734479 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.887787 (+153308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.936328 (+ 48541us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:34.991992 22830 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:34.994957 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.996061 21793 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:34.995975 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:34.996742 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:34.997365 22602 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:35.002103 21067 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:35.010993 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:35.004312 22676 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:34.986490 22836 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:35.004410 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:35.017724 22891 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:35.018211 21792 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:35.015853 21799 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:35.005497 22830 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:35.019274 22809 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:35.017289 21795 raft_consensus.cc:2368] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:35.019770 22921 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.020941 21215 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:35.022089 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:35.022730 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:35.023056 21213 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:35.023173 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:35.023622 21220 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:35.024252 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:35.024623 22648 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:35.008431 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:35.025817 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:35.026168 21063 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:35.026566 21799 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:35.027761 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 468) took 1465 ms. Trace:
I20260709 13:06:35.028141 21063 rpcz_store.cc:276] 0709 13:06:33.562172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.026917 (+1464745us) service_pool.cc:224] Handling call
0709 13:06:35.027208 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.027406 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.027445 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:35.027496 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:35.027534 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.027594 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.027698 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:35.029331 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:35.032387 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:35.036578 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:35.036789 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:35.030207 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:35.005939 22777 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:35.045083 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:35.045455 22744 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:35.045761 21065 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:35.046366 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:35.047451 22676 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:35.047854 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:35.047937 22676 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:35.045765 22838 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:06:35.048954 22676 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.049690 22925 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:35.050206 22925 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:35.051066 22925 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:35.051623 22830 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:35.052282 22836 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:35.052932 22757 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:35.054606 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1482) took 1495 ms. Trace:
I20260709 13:06:35.054857 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:35.057019 22765 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:35.057703 22765 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:35.058543 22765 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:35.056983 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:34.983206 22956 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.062090 22956 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:35.062202 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:35.063410 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:35.064172 21219 raft_consensus.cc:2368] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:35.043664 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:35.065634 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:35.055048 21064 rpcz_store.cc:276] 0709 13:06:33.559250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.135706 (+576456us) service_pool.cc:224] Handling call
0709 13:06:35.054426 (+918720us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b540011c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:35.054547 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113399168}
I20260709 13:06:35.047078 22744 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:35.063582 22964 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:35.069420 22956 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:35.087743 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:35.072631 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:35.071228 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:35.078859 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:35.081707 21799 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:35.071616 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:35.083890 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:35.084741 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:35.133466 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:35.133845 21219 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:35.134393 21796 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:35.071010 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:35.026177 22921 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:35.124018 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:35.135577 21146 raft_consensus.cc:2410] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:06:35.135747 21221 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:35.123591 21215 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:35.098724 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:35.141940 21142 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:35.062891 21213 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:35.149941 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:35.006834 22955 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.089454 22602 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.161707 22956 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.162423 22602 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.162529 22956 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.151098 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:35.164985 22602 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:35.165818 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:35.166019 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:35.168903 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:35.170619 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:35.150669 21142 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:35.082479 22814 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:35.166471 21139 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:35.080674 22744 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:35.153580 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 86696.5us
I20260709 13:06:35.030076 22904 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.171850 21147 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:35.180850 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:35.175539 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:35.175696 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:35.161867 21793 raft_consensus.cc:2368] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:35.151183 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:35.070067 22791 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.173662 22814 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.181995 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:35.219442 21138 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:06:35.199728 22602 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.184880 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:35.185418 22921 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:35.185467 22809 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:35.222767 22809 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.178843 22957 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:06:35.223043 22830 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:35.192930 21140 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:06:35.220927 21141 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:35.188441 21072 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:35.188181 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:35.215762 22814 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:35.183208 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:35.237792 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55895.2us
I20260709 13:06:35.182080 21142 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:35.188428 21036 leader_election.cc:400] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:35.240872 21036 leader_election.cc:403] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:06:35.238252 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53232.2us
I20260709 13:06:35.241487 22978 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:35.242479 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54194.9us
W20260709 13:06:35.245565 21187 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 2.156s (SENT) [suppressed 8 similar messages]
I20260709 13:06:35.246361 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 479) took 1638 ms. Trace:
I20260709 13:06:35.246731 21067 rpcz_store.cc:276] 0709 13:06:33.608009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.041068 (+1433059us) service_pool.cc:224] Handling call
0709 13:06:35.041446 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.041728 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:35.089305 (+ 47577us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.245965 (+156660us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:35.246027 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:35.246094 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.246188 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.246306 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21548,"prepare.queue_time_us":46374,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":2095,"raft.queue_time_us":886,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":156218,"threads_started":1}
I20260709 13:06:35.252146 22446 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:35.252638 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:35.253499 22446 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:35.253720 22778 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:35.253942 22446 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:35.254309 22778 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:35.254715 22778 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:35.254948 22446 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.257385 22446 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:35.254793 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:35.259893 22983 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:35.251159 21067 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:35.223145 22809 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:35.267727 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 84577.4us
I20260709 13:06:35.274503 21184 leader_election.cc:400] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:35.277230 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:35.277843 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:35.232252 22904 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:35.226167 22956 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:35.184391 22744 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:35.196004 22955 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:35.288431 21221 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:35.185708 22964 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:35.289039 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:35.271056 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:35.290195 21141 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
W20260709 13:06:35.292549 21072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 474) took 1687 ms (client timeout 2095 ms). Trace:
I20260709 13:06:35.278946 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:35.267896 21147 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
W20260709 13:06:35.292982 21072 rpcz_store.cc:269] 0709 13:06:33.605297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.029433 (+424136us) service_pool.cc:224] Handling call
0709 13:06:35.292378 (+1262945us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540019d2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:35.292482 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2662719872}
I20260709 13:06:35.285415 21184 leader_election.cc:400] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:35.293949 21184 leader_election.cc:403] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:06:35.295687 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 169) took 1142 ms. Trace:
I20260709 13:06:35.296062 21068 rpcz_store.cc:276] 0709 13:06:34.153301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.411595 (+258294us) service_pool.cc:224] Handling call
0709 13:06:35.295533 (+883938us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b5400215554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:35.295636 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937742080}
I20260709 13:06:35.297850 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:35.300159 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:35.303553 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:35.305312 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:35.307135 21072 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:35.309630 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:35.309871 21799 raft_consensus.cc:2368] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:35.311363 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:35.311537 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:35.312041 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 121) took 1094 ms. Trace:
I20260709 13:06:35.311681 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:35.315928 21144 rpcz_store.cc:276] 0709 13:06:34.195771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.195933 (+   162us) service_pool.cc:224] Handling call
0709 13:06:34.196230 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.338227 (+141997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.289656 (+951429us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54002b3ed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:35.289824 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324745088}
I20260709 13:06:35.280854 22923 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:35.312542 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:35.313446 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:35.328455 21142 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:35.312417 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:35.328737 21146 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:35.328975 21067 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:35.324034 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 77651.2us
I20260709 13:06:35.324765 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:35.331717 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46440.2us
I20260709 13:06:35.332362 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:35.283502 22830 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:35.334194 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56452.7us
I20260709 13:06:35.338074 21145 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:35.223174 22602 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } attrs { replace: true } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:35.335462 21801 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:06:35.338537 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49617.8us
I20260709 13:06:35.288108 22714 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:35.309048 22757 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:35.318236 22887 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:35.337253 22597 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:35.340008 21036 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:06:35.359398 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:35.359934 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:35.360529 21067 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:35.360929 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:35.348439 21212 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:35.362356 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:35.363111 21212 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:35.364125 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:35.351456 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:35.353475 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:35.330593 22923 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.365216 21215 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:35.365296 22923 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:35.366981 21215 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:35.325796 22927 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:35.342797 21146 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:35.368500 21215 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:35.371076 21212 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:35.294749 22648 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:06:35.376384 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:35.377859 22906 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:35.378327 21035 leader_election.cc:400] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:35.310864 22890 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:06:35.339570 22913 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:35.342232 22877 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:06:35.439931 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 63649.5us
I20260709 13:06:35.460793 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:35.346114 22476 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:35.339023 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:35.449787 21035 leader_election.cc:403] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:06:35.457746 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 145259us
I20260709 13:06:35.456004 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 168) took 1304 ms. Trace:
I20260709 13:06:35.360157 22948 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:35.464847 22836 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:35.465893 22791 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:35.387168 22562 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:35.337003 22830 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.470963 22602 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:35.472105 22969 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:06:35.479203 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:35.478860 21071 rpcz_store.cc:276] 0709 13:06:34.151200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.044442 (+893242us) service_pool.cc:224] Handling call
0709 13:06:35.044731 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.224630 (+179899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.224677 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:35.224723 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:35.224753 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.455621 (+230868us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54002ad5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:35.455792 (+   171us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.455947 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413127040}
W20260709 13:06:35.488848 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 116351us
I20260709 13:06:35.388376 22921 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:35.496268 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 161548us
W20260709 13:06:35.491372 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 113181us
I20260709 13:06:35.534586 21113 leader_election.cc:400] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:35.534924 21113 leader_election.cc:403] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:35.499854 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:35.483076 21072 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:35.575681 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:35.576110 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 76374.4us
I20260709 13:06:35.576740 21220 raft_consensus.cc:2368] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
W20260709 13:06:35.580653 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 43658.6us
I20260709 13:06:35.351249 22714 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.455399 22761 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:35.360035 22964 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 13:06:35.446538 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:35.440356 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 478) took 1832 ms. Trace:
W20260709 13:06:35.524818 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:35.669019 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58081.7us
W20260709 13:06:35.527319 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 140398us
I20260709 13:06:35.471730 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:35.375665 22757 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.670069 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:35.489248 22988 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 13:06:35.671391 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 317949us
I20260709 13:06:35.542902 22906 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:35.586573 22993 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:06:35.497530 22597 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:35.537762 22744 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:35.566499 21139 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:35.539479 22562 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:35.535455 22921 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.608654 21140 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:06:35.580415 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45953.8us
I20260709 13:06:35.538494 21793 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:35.508479 22877 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:35.640861 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 63770.9us
I20260709 13:06:35.719347 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:35.584467 22952 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.646s	user 0.013s	sys 0.016s
I20260709 13:06:35.455797 21064 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:35.573426 22983 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:35.541051 21035 leader_election.cc:400] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:35.580901 21796 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:35.577940 22930 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:35.691751 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:35.580868 22791 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:35.571072 21212 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:35.694001 21143 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:35.708896 21144 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:35.720963 21035 leader_election.cc:403] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:06:35.721282 21214 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:06:35.802363 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 261483us
I20260709 13:06:35.717343 22921 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:35.740898 22887 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.833549 22562 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:35.840744 22714 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:35.841070 22887 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:35.834436 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:35.843811 21795 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:35.849143 23004 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 13:06:35.833158 21794 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:35.839102 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 466) took 2277 ms. Trace:
I20260709 13:06:35.868358 21069 rpcz_store.cc:276] 0709 13:06:33.561736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.821935 (+1260199us) service_pool.cc:224] Handling call
0709 13:06:35.055376 (+233441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:35.055727 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.055756 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:35.056181 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 13:06:35.257999 (+201818us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:35.258064 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.258137 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.477217 (+219080us) raft_consensus.cc:1685] finished
0709 13:06:35.477246 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.838849 (+361603us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b540011c1d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:35.839035 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":920,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"spinlock_wait_cycles":536032128,"thread_start_us":201489,"threads_started":1,"wal-append.queue_time_us":415333}
I20260709 13:06:35.689435 21063 rpcz_store.cc:276] 0709 13:06:33.607640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.029122 (+1421482us) service_pool.cc:224] Handling call
0709 13:06:35.029382 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.029585 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:35.060916 (+ 31331us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.313822 (+252906us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.336045 (+ 22223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:35.340573 (+  4528us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:35.344192 (+  3619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.357994 (+ 13802us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.388095 (+ 30101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15560,"prepare.queue_time_us":42608,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":22592,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":12058,"thread_start_us":177441,"threads_started":1,"wal-append.queue_time_us":322678}
W20260709 13:06:35.807082 22993 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:06:35.828972 22906 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
W20260709 13:06:35.856194 23001 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
W20260709 13:06:35.878113 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 162711us
W20260709 13:06:35.892532 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 173366us
W20260709 13:06:35.877492 22923 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.511s	user 0.005s	sys 0.006s
I20260709 13:06:35.874867 22955 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:06:35.889955 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 214440us
I20260709 13:06:35.902355 22446 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:35.902499 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:35.902863 22446 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:35.903967 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:35.905862 22446 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:35.907457 22446 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:35.870417 22913 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3.
I20260709 13:06:35.870939 22765 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:06:35.922411 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:35.878089 22887 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:35.923095 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:35.880038 23011 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:35.956568 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 264920us
I20260709 13:06:35.925233 22791 raft_consensus.cc:2764] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:06:35.985005 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:35.986326 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:35.975558 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71673us
I20260709 13:06:35.982219 21147 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:35.957813 22887 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.912106 22923 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:35.882516 22877 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:35.993597 22830 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:35.971822 23020 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:35.951719 23025 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:06:35.996034 22887 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:36.004230 23006 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:06:36.001168 22830 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.001859 23024 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:36.004967 22838 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:36.022373 22921 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.022957 21793 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:36.028457 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:36.034657 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61940.5us
I20260709 13:06:36.008389 22836 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:06:36.018100 22830 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:36.041767 22838 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:36.046059 22838 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:36.047741 22809 consensus_meta.cc:287] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.761s	user 0.011s	sys 0.000s
I20260709 13:06:36.048266 22809 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:36.048856 23000 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.452s	user 0.006s	sys 0.000s
I20260709 13:06:36.049448 21141 raft_consensus.cc:2368] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:06:36.052711 22809 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.023995 23024 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:36.054315 22985 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.026254 23023 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 13:06:36.060904 22814 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.791s	user 0.005s	sys 0.006s
I20260709 13:06:36.061409 22814 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:36.062223 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 391980us
I20260709 13:06:36.063485 22648 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.065407 23014 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:36.065913 23014 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.057539 22476 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:36.066543 22602 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:36.068085 23014 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:36.062299 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 389989us
I20260709 13:06:36.067705 22476 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.029965 21795 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:35.706820 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:36.030632 22870 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:06:36.058063 22769 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:36.102885 22923 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:36.066586 22978 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.825s	user 0.007s	sys 0.004s
W20260709 13:06:36.109954 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46624.1us
I20260709 13:06:36.110070 22978 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:36.109048 21143 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:06:36.111258 21185 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:36.057068 23024 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:36.110951 23038 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
W20260709 13:06:36.109642 22778 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.854s	user 0.012s	sys 0.000s
I20260709 13:06:36.117341 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:36.118868 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:36.057858 22904 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:06:36.122871 22923 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:36.124087 22476 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:06:36.121055 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 115597us
I20260709 13:06:36.128553 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:06:36.125140 23026 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:06:36.122184 22778 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:36.135954 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 94899.2us
I20260709 13:06:36.114746 23013 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:06:36.135810 22676 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:36.145797 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:36.147183 21218 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:36.148698 22562 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.152554 22887 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.153680 21214 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:36.154263 22830 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.154433 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:36.155345 21792 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:36.143642 22921 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:36.166628 23021 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:36.149286 22562 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.167270 23021 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.168972 22562 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.171703 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:36.172480 21214 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:36.144501 22676 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:36.175729 22676 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.181478 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.182125 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.182957 21069 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:36.188042 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.189683 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.182327 21071 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:36.195708 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.196470 21068 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:36.196811 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.197590 21067 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:36.198621 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.198954 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.199303 21067 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:36.196484 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.214432 21214 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:36.221988 21072 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:36.225332 21069 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:36.229053 22921 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:36.218590 22906 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:06:36.234449 22906 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:36.221138 22985 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.193328 22562 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:36.235476 22985 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:36.235572 22778 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:36.172227 23034 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:06:36.238727 23021 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.218844 22476 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:36.229424 21068 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:36.239272 21792 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:36.243630 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.255005 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.240136 23005 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.234792 22906 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:36.250732 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:36.275434 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:36.277189 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:36.277537 21187 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:36.246886 21067 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:36.256553 21069 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:36.257695 21795 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:36.257992 22476 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.258906 23023 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:36.260694 21794 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:36.261802 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.243440 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.267254 23040 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:36.268468 23046 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:36.270079 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:36.270265 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:36.271190 23010 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.271435 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:36.272722 21220 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:36.289109 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:36.241863 22602 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.317385 21069 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:36.249646 22985 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:36.320351 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.230496 22913 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:06:36.323230 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:36.330644 21067 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:06:36.331051 23014 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:36.331552 23014 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:36.324469 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 145173us
I20260709 13:06:36.241156 23036 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 13:06:36.297979 22809 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:36.297451 22955 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:36.325994 21212 raft_consensus.cc:2410] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:06:36.297809 22891 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:36.286695 22778 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:36.299018 23023 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:36.335570 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:36.337167 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:36.337922 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.338908 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.322290 22904 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:36.341192 21067 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:36.343407 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:36.344077 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.319603 22870 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:06:36.345616 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:36.363523 21139 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:36.345912 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.365191 21142 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:06:36.351274 21147 raft_consensus.cc:1813] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Handling vote request from an unknown peer 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:36.365705 21147 raft_consensus.cc:2368] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:36.351480 21069 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:06:36.352684 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:36.353696 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:36.345249 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.368269 21212 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:06:36.356551 21036 leader_election.cc:400] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:36.358613 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.369237 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.359469 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.370296 21069 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:36.370682 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.367561 21767 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:36.372912 21069 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:36.361464 21141 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:36.332664 23035 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:36.360116 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:36.369903 21071 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:36.388950 21071 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:06:36.394613 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.395429 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:36.358731 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:36.396740 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.402522 21071 raft_consensus.cc:2376] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4. Re-sending same reply.
W20260709 13:06:36.406308 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47661.6us
I20260709 13:06:36.325104 23040 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:36.455443 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47929.8us
I20260709 13:06:36.371335 22927 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:06:36.454872 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59566.4us
I20260709 13:06:36.456215 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:36.368098 22955 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:36.457454 21139 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:36.368249 22809 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:36.342016 22602 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.332538 23010 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.369509 23027 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:36.350464 22778 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.398694 23021 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:36.365919 22930 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:06:36.368975 22562 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 13:06:36.461500 22956 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:36.497539 22714 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:36.498163 22602 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:36.500018 23010 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.502661 22830 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.458420 23035 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:36.408541 23009 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 13:06:36.341372 23046 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.407593 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.509929 22809 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:36.507761 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.466545 23038 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:36.512465 21185 leader_election.cc:400] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4.
I20260709 13:06:36.459725 22891 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:36.460139 22927 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:36.516448 21147 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:36.523626 23021 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:36.398958 22948 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 13:06:36.406586 22838 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 13:06:36.407970 22676 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 13:06:36.523103 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.527619 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:36.366564 22906 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.520279 22714 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:36.505874 22955 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:36.507788 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.521837 22956 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:36.519624 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.536409 21185 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:36.515129 23050 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:06:36.539716 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.541463 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.460701 23023 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.573573 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:36.544027 22927 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.544996 23046 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:36.546957 23040 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:36.543150 22891 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:36.586158 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:36.587057 23021 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.549242 22778 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.565343 22809 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.567797 22906 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:36.581411 23038 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:36.568958 22955 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.571487 23035 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.557003 23024 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.583683 21063 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:36.573312 22956 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:36.573426 22749 consensus_queue.cc:1175] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "46c1676121b64b7d9bcf2227cc639dd3" responder_term: 3 status { last_received { term: 3 index: 5 } last_committed_idx: 4 last_received_current_leader { term: 3 index: 5 } }
I20260709 13:06:36.585125 23023 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:36.585436 22921 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:36.585510 22978 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:36.604799 22948 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:36.515970 23014 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.525710 21065 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:36.610172 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:36.662643 21220 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:36.614332 21212 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:36.615943 21218 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:36.621750 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:36.629431 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 240341us
I20260709 13:06:36.584085 22814 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:36.665310 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:36.643954 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.591924 22597 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:36.592165 22926 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:36.587205 22891 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.659178 21145 raft_consensus.cc:2410] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:06:36.666960 21036 leader_election.cc:400] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:06:36.667423 22446 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:36.669971 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52524.7us
I20260709 13:06:36.667061 21144 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:36.675837 21036 leader_election.cc:403] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:36.673015 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.688099 22446 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:06:36.673323 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:36.690887 21221 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:36.699577 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:36.700392 21798 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:36.701064 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:36.701881 22978 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:36.670292 23013 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:36.704694 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:36.665135 23005 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:36.711745 22446 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:06:36.712337 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.713927 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.719197 21072 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:36.721959 21215 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:36.732666 21221 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:36.738409 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:36.739490 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:36.740252 23060 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:36.741518 23057 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:06:36.677263 22597 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.744493 21066 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:36.745515 22597 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.747254 21064 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:36.722378 23031 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:36.755568 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.757978 21072 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:36.759392 23027 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:36.671613 22778 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:06:36.666658 22948 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.782963 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:36.750590 21064 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:36.745846 22710 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:36.723670 23056 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:36.779024 23013 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:36.773705 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:36.802443 23024 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:36.803485 22926 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:36.794399 22710 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:36.738034 23005 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.818121 22913 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:36.821341 23046 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:36.824862 22926 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:36.836378 22913 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.837687 23013 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:36.829838 22948 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.839774 22926 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:36.840718 22710 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.817750 23056 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.845276 23005 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:36.850152 22714 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:36.862874 22714 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:36.868180 22757 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:36.869035 22710 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:36.868170 22913 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:36.863042 22956 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:36.873723 22927 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:36.887840 22956 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
W20260709 13:06:36.794633 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 105953us
I20260709 13:06:36.932345 21065 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:36.934624 21066 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:36.940917 21143 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:36.770913 21070 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:36.944219 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:36.945741 21146 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:36.956284 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.968966 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.971076 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:36.971271 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:36.971652 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:37.016821 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:06:37.022223 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51438us
I20260709 13:06:37.023983 22562 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.025254 22809 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.025097 22562 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.026103 22809 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:37.027803 23021 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:37.030311 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:36.974751 20969 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: leader changed from <none> to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:37.035632 22956 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:37.036448 22956 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:37.038602 23046 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:37.044766 22778 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:37.045363 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:37.046312 21063 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:37.049618 23005 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:37.050450 23005 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.052536 22562 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:37.053043 23005 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:37.053663 23005 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:37.057629 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:37.058254 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:37.058526 21070 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:37.059055 21145 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:37.039084 22956 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.088881 21068 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:37.103885 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:37.111554 22964 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:37.112248 22964 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.045406 22778 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:37.112613 22757 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:37.112730 22964 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:37.079903 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:37.132356 22446 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.141438 22778 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.145758 22446 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:37.146692 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:37.147909 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:37.148377 22955 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:37.149026 22955 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:37.149441 22955 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:37.150435 22836 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:37.151139 22836 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:37.152132 22757 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.152612 22836 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:06:37.153263 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 137068us
I20260709 13:06:37.154801 22757 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:37.157866 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.148444 22446 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.172001 21795 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:06:37.172495 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.172937 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.171988 22913 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.174185 22913 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.184514 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:37.196038 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 128299us
I20260709 13:06:37.184540 21217 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:37.199397 21036 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:06:37.207729 21036 leader_election.cc:403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:06:37.211210 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56909.4us
I20260709 13:06:37.222960 23007 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:37.225340 23007 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.226181 23027 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:37.226877 23027 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.227500 22796 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:37.242304 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:37.245251 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:37.247272 22796 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:37.249425 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 466542us
I20260709 13:06:37.249585 22930 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:37.250125 22930 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:37.250448 22796 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.252511 23022 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 13:06:37.257604 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 76970.6us
I20260709 13:06:37.252934 22809 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.248046 21795 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:37.262383 21068 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
W20260709 13:06:37.265281 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.265126 22926 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:37.266616 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:37.267700 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:37.268558 21145 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:37.270160 21219 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:37.271714 22913 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:37.272889 22887 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:37.276651 22926 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:37.280431 23007 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.281368 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:37.281023 22744 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.300155 21072 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:06:37.300642 21072 pending_rounds.cc:85] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 2
I20260709 13:06:37.300902 21072 pending_rounds.cc:107] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:06:37.301210 21072 raft_consensus.cc:2889] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:37.302832 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 103786us
I20260709 13:06:37.263849 23021 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:06:37.295780 22446 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:06:37.304008 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:37.313380 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 153762us
I20260709 13:06:37.327497 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:37.311877 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:37.308776 22913 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:37.376878 22913 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:37.377517 22913 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:37.318127 22769 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.378259 22769 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:37.380506 23054 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.843s	user 0.002s	sys 0.004s
I20260709 13:06:37.293853 22930 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.385847 22930 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:37.360064 21212 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:37.361089 21018 placement_policy_util.cc:248] tablet cc53d52fdeaa441b9e9d329e69d94440: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 13:06:37.309163 23010 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:06:37.364482 22641 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:37.365422 21072 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:06:37.370393 22710 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.500s	user 0.011s	sys 0.000s
I20260709 13:06:37.370622 23023 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:06:37.376240 21065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 504) took 1188 ms (client timeout 1554 ms). Trace:
I20260709 13:06:37.376943 21214 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:37.389590 22641 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:37.391490 21146 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:37.359813 22744 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:37.393085 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:37.311363 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:37.428603 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 101256us
I20260709 13:06:37.391929 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 516) took 1171 ms. Trace:
I20260709 13:06:37.324548 23027 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.328312 22485 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:37.450726 21070 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:37.453830 23032 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.467384 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:37.432956 22964 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.473232 22602 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:37.474303 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:37.474764 22964 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.476323 22769 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:37.495828 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:37.552359 23032 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.637109 21072 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:37.519431 22597 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:37.639123 22597 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:37.589895 21203 tablet_service.cc:1558] Processing DeleteTablet for tablet cc53d52fdeaa441b9e9d329e69d94440 with delete_type TABLET_DATA_TOMBSTONED (TS 46c1676121b64b7d9bcf2227cc639dd3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56796
I20260709 13:06:37.522179 21140 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:37.586452 21070 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:06:37.494807 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:37.564818 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:37.495564 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:37.461520 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:37.440186 21069 rpcz_store.cc:276] 0709 13:06:36.219988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.374986 (+154998us) service_pool.cc:224] Handling call
0709 13:06:36.376202 (+  1216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.376517 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.376554 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.377099 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.479201 (+102102us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:36.479254 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.479369 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.379896 (+900527us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b240013a558. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f2b59c4 00007f7f3f2b3acb 00007f7f3f46edc9 00007f7f3f46ea55 00007f7f3f46e09d 00007f7f420a010e 00007f7f420b788c 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221
0709 13:06:37.386442 (+  6546us) raft_consensus.cc:1685] finished
0709 13:06:37.386470 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.391720 (+  5250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10546432,"thread_start_us":101132,"threads_started":1,"wal-append.queue_time_us":140828}
I20260709 13:06:37.638736 21072 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:37.595338 20968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 239) took 8493 ms. Trace:
I20260709 13:06:37.578511 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:37.575548 20969 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:37.665946 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:37.554145 23035 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.008s	sys 0.054s
W20260709 13:06:37.680159 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 227173us
W20260709 13:06:37.682144 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 103838us
I20260709 13:06:37.639000 23032 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:37.492630 22749 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:37.686962 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.517311 23056 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:37.704766 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 140020us
W20260709 13:06:37.655072 21066 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.035s	sys 0.007s
W20260709 13:06:37.686281 23035 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.008s	sys 0.062s
W20260709 13:06:37.435710 21065 rpcz_store.cc:269] 0709 13:06:36.187649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.218456 (+ 30807us) service_pool.cc:224] Handling call
0709 13:06:37.375920 (+1157464us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b540026eb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:37.376135 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464263424}
W20260709 13:06:37.683909 22836 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.024s	sys 0.047s
W20260709 13:06:37.711743 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 438285us
I20260709 13:06:37.713970 21070 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:06:37.722227 22836 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.024s	sys 0.048s
I20260709 13:06:37.715442 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:37.519568 22955 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.703629 22930 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:06:37.711310 21066 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.036s	sys 0.007s
I20260709 13:06:37.710994 23032 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:37.733705 21145 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:37.640786 22597 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.712668 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:37.493131 22641 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:37.744474 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.713074 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:37.754393 21071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.000s	sys 0.049s
I20260709 13:06:37.728446 22744 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:37.729017 22602 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.758690 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:37.743455 20969 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 2 to 4, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 4 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:37.759595 21795 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:37.654541 20968 rpcz_store.cc:276] 0709 13:06:29.102314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.106662 (+  4348us) service_pool.cc:224] Handling call
0709 13:06:29.280480 (+173818us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b5400002490. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f8640 00007f7f482cfb76 00007f7f483d9c3e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc 00007f7f46899392 00007f7f4085748c 00007f7f408568c6
0709 13:06:37.595027 (+8314547us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:32.785804 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:32.931868 (+146064us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:32.931904 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:06:32.931922 (+    18us) tablet.cc:662] Decoding operations
0709 13:06:33.049315 (+117393us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:33.051615 (+  2300us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:33.053937 (+  2322us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:06:33.163649 (+109712us) tablet.cc:701] Row locks acquired
0709 13:06:33.163676 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:06:33.163832 (+   156us) write_op.cc:270] Start()
0709 13:06:33.163948 (+   116us) write_op.cc:276] Timestamp: P: 1783602393163787 usec, L: 0
0709 13:06:33.163982 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:06:33.212647 (+ 48665us) log.cc:844] Serialized 4015 byte log entry
0709 13:06:33.938126 (+725479us) op_driver.cc:464] REPLICATION: finished
0709 13:06:34.289990 (+351864us) write_op.cc:301] APPLY: starting
0709 13:06:34.292794 (+  2804us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:34.389205 (+ 96411us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:34.400968 (+ 11763us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:34.432947 (+ 31979us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:34.433292 (+   345us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:06:34.433362 (+    70us) write_op.cc:312] APPLY: finished
0709 13:06:34.433795 (+   433us) log.cc:844] Serialized 77 byte log entry
0709 13:06:34.434019 (+   224us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:34.434205 (+   186us) write_op.cc:454] Released schema lock
0709 13:06:34.434270 (+    65us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":51022,"reported_tablets":22,"spinlock_wait_cycles":4890496,"child_traces":[["op",{"apply.queue_time_us":318884,"apply.run_cpu_time_us":4889,"apply.run_wall_time_us":144755,"num_ops":10,"prepare.queue_time_us":581359,"prepare.run_cpu_time_us":19287,"prepare.run_wall_time_us":716202,"raft.queue_time_us":353014,"raft.run_cpu_time_us":18627,"raft.run_wall_time_us":287969,"replication_time_us":756134,"thread_start_us":1205136,"threads_started":4,"wal-append.queue_time_us":316210,"wal-append.run_cpu_time_us":22003,"wal-append.run_wall_time_us":1904308}]]}
W20260709 13:06:37.762077 21799 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.020s	sys 0.001s
W20260709 13:06:37.771014 21799 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.020s	sys 0.001s
I20260709 13:06:37.762194 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:37.731603 22930 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:37.740571 23032 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.721613 22913 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.777011 22602 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:37.791724 22744 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:37.793061 20969 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:37.734123 23046 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 13:06:37.788281 22476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.014s	sys 0.017s
I20260709 13:06:37.740398 22836 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:06:37.845563 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:37.769438 22927 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:37.769860 22597 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:37.798398 23032 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:37.812232 21068 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:06:37.780562 22930 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:37.740142 22749 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:37.850898 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.854472 21067 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.026s	sys 0.039s
W20260709 13:06:37.776471 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.864835 21067 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.027s	sys 0.039s
W20260709 13:06:37.858217 21035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.012s	sys 0.061s
W20260709 13:06:37.865433 21035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.012s	sys 0.061s
W20260709 13:06:37.845719 22476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.014s	sys 0.017s
I20260709 13:06:37.852228 20969 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 3, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 3 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:06:37.851585 22887 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.001s	sys 0.031s
W20260709 13:06:37.848038 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66095.5us
I20260709 13:06:37.862394 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:37.880096 21216 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:06:37.788714 23083 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:06:37.850641 22927 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.850844 21214 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:37.891217 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:37.880942 22887 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.001s	sys 0.031s
W20260709 13:06:37.873864 21213 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.007s	sys 0.007s
I20260709 13:06:37.906213 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:37.906512 21212 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:37.906880 23028 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:37.907354 21221 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:06:37.907642 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 94295.4us
I20260709 13:06:37.907930 21220 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:37.909286 21793 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:06:37.909583 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.911155 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:37.899804 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:37.911746 22870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.016s	sys 0.046s
I20260709 13:06:37.911759 22904 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:37.912482 21218 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:37.912919 23072 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.893121 22927 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:37.914307 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:37.941375 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 90644.2us
I20260709 13:06:37.915964 22957 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:37.917853 21213 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.008s	sys 0.008s
W20260709 13:06:37.938787 22870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.016s	sys 0.046s
I20260709 13:06:37.983621 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 517) took 1729 ms. Trace:
I20260709 13:06:37.900790 23023 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
W20260709 13:06:37.949810 22874 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.018s	sys 0.039s
I20260709 13:06:38.005630 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:37.993772 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:37.975594 22913 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:38.009991 22874 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.020s	sys 0.039s
I20260709 13:06:37.994920 22955 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:38.028350 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:38.037742 21070 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:37.986675 21067 rpcz_store.cc:276] 0709 13:06:36.245223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.504846 (+259623us) service_pool.cc:224] Handling call
0709 13:06:36.505162 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.505361 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:36.936586 (+431225us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.873458 (+936872us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:37.887557 (+ 14099us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:37.894464 (+  6907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.943299 (+ 48835us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.974120 (+ 30821us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14442,"prepare.queue_time_us":376029,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":29749,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"thread_start_us":1330461,"threads_started":3,"wal-append.queue_time_us":301020}
W20260709 13:06:38.043123 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 59075.2us
I20260709 13:06:38.055935 22676 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.117s
W20260709 13:06:38.063607 23066 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.202s	user 0.000s	sys 0.002s
I20260709 13:06:38.065678 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:38.071645 23036 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:38.079741 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:38.080168 21063 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:38.085299 23007 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:37.977021 22957 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.062479 23014 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.062078 23093 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.054080 23040 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:38.061136 23057 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.110s
I20260709 13:06:38.095708 21221 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:06:38.028488 22791 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.270s
I20260709 13:06:38.019744 23072 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.086999 22749 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:37.988200 22904 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.133296 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:38.147730 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.155357 22830 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.157172 23093 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.158206 23035 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:38.141754 23036 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:38.018808 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:38.101467 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49220.3us
I20260709 13:06:38.181125 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:38.181921 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:38.129648 23007 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:06:38.203186 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.221272 23009 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:06:38.222625 23007 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.223977 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:38.224248 23007 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:38.225203 22925 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:38.225962 22925 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.228399 22925 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:38.206815 23035 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:38.230400 22830 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.119865 22476 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:06:38.232761 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2388) took 1873 ms. Trace:
I20260709 13:06:38.230454 21067 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:38.234498 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:38.236109 21065 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:38.243549 21138 rpcz_store.cc:276] 0709 13:06:36.358925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.359155 (+   230us) service_pool.cc:224] Handling call
0709 13:06:38.232415 (+1873260us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54004236d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:38.232564 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3338326400}
I20260709 13:06:38.228711 23072 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:38.188177 22602 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:38.195557 22744 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:38.265807 21068 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:38.236339 23093 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:38.175489 23014 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:38.282732 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.253368 23028 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:38.317095 23028 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:38.281898 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 267117us
W20260709 13:06:38.312645 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46873.7us
I20260709 13:06:38.258940 22955 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:38.323791 21144 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:06:38.271149 23009 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.238759 23040 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.270591 23089 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:38.261968 23005 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:06:38.341650 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:38.341894 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:38.343954 21069 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:38.313711 22930 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.341665 23106 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.106s
I20260709 13:06:38.329614 22925 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:38.352253 21070 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:38.361030 21212 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:06:38.364040 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:38.365427 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.339363 22964 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:38.338156 22744 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:38.367717 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 45738.8us
I20260709 13:06:38.334340 23020 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:06:38.352629 21064 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:38.382851 21069 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:06:38.361820 22955 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:38.389937 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:38.390041 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:38.433449 21146 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:38.449150 23104 tablet_replica.cc:333] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:06:38.449875 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:38.451421 23112 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:38.451687 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:38.452105 23112 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.453127 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:38.374282 23009 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.455209 22830 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:38.456074 22904 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:38.456732 23009 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:38.457371 23108 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:06:38.458813 21797 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:06:38.459685 23040 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:38.460162 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:38.460583 23085 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:06:38.461231 23114 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:38.461309 22984 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:38.461747 23046 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.462001 23114 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.462172 22984 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.462381 23046 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.463752 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1520) took 2241 ms. Trace:
I20260709 13:06:38.464396 22984 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:38.464668 23046 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:38.468755 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:38.467644 23111 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.343628 23035 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.465837 22476 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:38.464141 21071 rpcz_store.cc:276] 0709 13:06:36.222732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.403532 (+180800us) service_pool.cc:224] Handling call
0709 13:06:36.940042 (+536510us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:36.940572 (+   530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.941264 (+   692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.941300 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.942151 (+   851us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.755011 (+812860us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:37.755066 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.755152 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:38.463449 (+708297us) raft_consensus.cc:1685] finished
0709 13:06:38.463468 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.463688 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106325,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":264,"spinlock_wait_cycles":67683584,"thread_start_us":811617,"threads_started":1,"wal-append.queue_time_us":1379188}
I20260709 13:06:38.475337 21147 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:38.428683 23028 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:38.475353 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.384801 22744 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.454449 23112 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:38.469244 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:38.468585 21217 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:38.470280 22930 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:38.514292 23087 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.653s	user 0.004s	sys 0.000s
I20260709 13:06:38.448629 21068 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:38.491655 21070 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:06:38.516331 21070 pending_rounds.cc:85] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 4
I20260709 13:06:38.516701 21070 pending_rounds.cc:107] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:06:38.517073 21070 raft_consensus.cc:2889] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:38.493765 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:38.497494 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:38.497857 21146 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:38.498292 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1697) took 1754 ms. Trace:
I20260709 13:06:38.502704 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:38.504192 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:06:38.519784 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 134136us
I20260709 13:06:38.509369 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:38.429741 22925 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.510905 23111 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.372225 23036 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.513494 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:38.520397 21212 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:38.520907 21144 rpcz_store.cc:276] 0709 13:06:36.743927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.763431 (+ 19504us) service_pool.cc:224] Handling call
0709 13:06:38.246431 (+1483000us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:38.340208 (+ 93777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.498039 (+157831us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54004236d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:38.498191 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3357264640}
I20260709 13:06:38.445773 22749 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:38.533111 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:38.521967 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2394) took 1465 ms. Trace:
I20260709 13:06:38.524677 22476 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.538683 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:38.431529 22964 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.525542 22641 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:38.525653 22814 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:38.512080 23046 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.531708 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:38.518784 23112 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:38.543385 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.548687 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:38.523523 23040 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:38.488021 22955 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.432337 23089 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.530480 22930 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:38.561707 21145 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:38.558552 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:38.572538 21218 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:38.512511 22602 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:06:38.539201 23083 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:38.569590 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:38.577584 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:38.578219 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50324.2us
I20260709 13:06:38.541941 22830 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.539597 22887 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:38.522006 22874 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 13:06:38.570513 23097 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:06:38.575646 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:38.536008 21142 rpcz_store.cc:276] 0709 13:06:37.056673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.059816 (+  3143us) service_pool.cc:224] Handling call
0709 13:06:37.060301 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.060893 (+   592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.060943 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:37.061547 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.441728 (+380181us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:37.441777 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.441848 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:38.513553 (+1071705us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:38.516283 (+  2730us) raft_consensus.cc:1685] finished
0709 13:06:38.516306 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.521788 (+  5482us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3833,"raft.queue_time_us":6912,"raft.run_cpu_time_us":470,"raft.run_wall_time_us":41403,"spinlock_wait_cycles":8851712,"thread_start_us":379579,"threads_started":1,"wal-append.queue_time_us":703269}
I20260709 13:06:38.601394 21145 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:38.692737 22964 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:38.556633 22749 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.612680 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:38.590843 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 85516us
I20260709 13:06:38.688808 21141 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:38.706827 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:38.647383 23089 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:38.688446 23083 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.611891 23096 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:38.609659 23014 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:38.596812 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:38.696722 23056 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.726231 21146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c918605cdf4c39b832a695968308a3"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:06:38.729070 23022 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:06:38.739929 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:38.576383 22814 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.743578 21218 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:38.726895 23114 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:38.750545 21212 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
W20260709 13:06:38.778698 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.573966 23112 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.819640 21215 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:38.592325 23046 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:38.752218 23032 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.865s	user 0.012s	sys 0.000s
I20260709 13:06:38.747264 23083 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:38.711534 22830 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:38.819792 22927 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.014s
W20260709 13:06:38.825924 23007 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.016s
I20260709 13:06:38.820039 22913 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:38.711314 22887 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.816506 23089 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:38.816807 23014 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:38.854825 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 127714us
I20260709 13:06:38.822822 22476 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:38.829516 22925 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:38.753239 23056 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.861366 23130 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:38.887115 21067 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:38.822263 23096 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.833478 23032 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.863397 22927 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.894793 23130 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:38.897584 23100 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:38.917424 21143 raft_consensus.cc:2368] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:38.896437 22744 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:38.915630 21793 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:38.916039 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:38.916364 23078 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.978s	user 0.000s	sys 0.005s
I20260709 13:06:38.924599 23111 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:38.932121 23086 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.086s	user 0.000s	sys 0.009s
W20260709 13:06:38.932736 23123 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:06:38.934418 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 522) took 2358 ms. Trace:
I20260709 13:06:38.935009 21066 rpcz_store.cc:276] 0709 13:06:36.575962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.938239 (+362277us) service_pool.cc:224] Handling call
0709 13:06:36.953282 (+ 15043us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.954131 (+   849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.954201 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.955085 (+   884us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.743457 (+788372us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:37.752450 (+  8993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.765179 (+ 12729us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:38.931495 (+1166316us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:38.933983 (+  2488us) raft_consensus.cc:1685] finished
0709 13:06:38.934047 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:38.934328 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95790,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":153,"spinlock_wait_cycles":53477248,"thread_start_us":771550,"threads_started":1,"wal-append.queue_time_us":809776}
I20260709 13:06:38.895992 23014 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:38.907620 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 207527us
I20260709 13:06:38.897748 23089 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:38.957315 23111 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:38.958295 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 251546us
I20260709 13:06:38.958089 23111 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.959839 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:38.960810 22476 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:38.960866 23111 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:38.963017 21146 raft_consensus.cc:606] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:38.964509 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:38.919314 23100 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:38.961464 22476 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:06:38.966436 22749 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:38.955617 23040 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:06:39.018625 22814 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:39.021250 23141 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:39.021382 23096 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:39.021828 23141 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.022125 23089 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:39.022230 23141 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:39.023442 23040 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:39.023638 23144 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:39.005508 23112 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:39.042645 23091 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.193s	user 0.000s	sys 0.005s
I20260709 13:06:39.034114 23056 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:39.025487 22887 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.024658 23100 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:39.078356 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:39.079769 23139 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.061s	user 0.001s	sys 0.001s
W20260709 13:06:39.080657 23090 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.749s	user 0.006s	sys 0.000s
W20260709 13:06:39.084259 23126 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.146s	user 0.000s	sys 0.003s
I20260709 13:06:39.084499 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.085266 23104 raft_consensus.cc:2243] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:39.085439 22927 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:39.019550 23093 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.086269 21767 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:39.086565 23143 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.003s
W20260709 13:06:39.087751 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61987.5us
I20260709 13:06:39.088397 23014 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:39.088985 23032 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:39.090118 23141 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.090535 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:39.000191 23130 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:39.141692 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 55529us
I20260709 13:06:39.142704 23130 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:39.144400 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:39.099645 22874 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:39.099897 22955 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:39.140703 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50261.5us
I20260709 13:06:39.146541 22814 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:39.146684 23022 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:06:39.147895 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 69681.9us
I20260709 13:06:39.152470 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:39.156698 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:39.147586 23022 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:39.162019 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.164776 21072 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:39.164996 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:39.166533 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:39.168534 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.161554 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:39.147212 22814 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:39.188766 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.189172 21799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 188) took 2220 ms. Trace:
I20260709 13:06:39.189777 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.189627 21799 rpcz_store.cc:276] 0709 13:06:36.968587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.009782 (+ 41195us) service_pool.cc:224] Handling call
0709 13:06:37.010695 (+   913us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.077942 (+ 67247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.077986 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:37.142593 (+ 64607us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.771428 (+628835us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:37.771465 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.771528 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:39.036443 (+1264915us) spinlock_profiling.cc:243] Waited 6.03 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:39.188874 (+152431us) raft_consensus.cc:1685] finished
0709 13:06:39.188891 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.189095 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8644,"raft.queue_time_us":99279,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":171,"spinlock_wait_cycles":13863680,"thread_start_us":628268,"threads_started":1,"wal-append.queue_time_us":1105013}
I20260709 13:06:39.191536 23141 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.156816 23007 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:39.193282 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.194370 21067 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:39.158073 23136 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:39.189268 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:39.206244 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:39.209511 23007 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:39.168466 22887 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:39.168237 23014 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:39.148346 23093 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:39.221403 21185 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:39.225546 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:39.227092 23014 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.230064 23022 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:39.230693 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:39.233036 23151 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:06:39.172030 22925 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 13:06:39.234236 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67799.5us
I20260709 13:06:39.212914 23111 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:39.215633 22476 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:39.238317 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.238346 22476 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.239986 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.245679 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:39.247488 22476 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.248232 22476 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:39.248812 23039 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:39.249253 23039 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.249610 23039 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:39.247434 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.238255 23111 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:39.260803 23147 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:39.260968 23083 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.267553 22641 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:39.267889 23141 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:39.268153 22641 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.268491 23141 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.268981 23083 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.240870 21063 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:39.270226 22641 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.270695 23141 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.251292 23104 raft_consensus.cc:2272] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:39.271720 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:39.272950 22984 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:39.244238 23122 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:39.277468 23104 ts_tablet_manager.cc:1916] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:39.277647 23093 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:39.279376 23101 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.079s	user 0.002s	sys 0.002s
I20260709 13:06:39.280922 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 480) took 2933 ms. Trace:
I20260709 13:06:39.281066 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 609) took 1509 ms. Trace:
I20260709 13:06:39.281486 21216 rpcz_store.cc:276] 0709 13:06:37.771479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.021617 (+250138us) service_pool.cc:224] Handling call
0709 13:06:38.087794 (+ 66177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:38.465351 (+377557us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:38.465382 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:38.465443 (+    61us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:38.465482 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.280815 (+815333us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400442ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:39.280862 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.280985 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131728,"raft.queue_time_us":132871,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":564387200}
W20260709 13:06:39.303812 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.305706 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.306603 21801 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:39.235617 22814 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:39.305634 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:39.321666 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:39.282601 23007 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:39.311781 21214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2247) took 1762 ms (client timeout 1500 ms). Trace:
I20260709 13:06:39.302929 23122 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.330454 22904 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:39.341527 21797 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:39.343956 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.354305 21794 raft_consensus.cc:2410] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:39.373569 22476 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.374521 23111 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:39.372038 22904 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:39.445200 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.426796 22641 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:39.445892 21067 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:39.452690 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.447096 22904 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:39.524828 21801 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
W20260709 13:06:39.345266 21214 rpcz_store.cc:269] 0709 13:06:37.646841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.663311 (+ 16470us) service_pool.cc:224] Handling call
0709 13:06:39.283135 (+1619824us) spinlock_profiling.cc:243] Waited 91.6 ms on lock 0x7b540013c754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:39.302438 (+ 19303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210627328}
I20260709 13:06:39.431018 22476 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.537179 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.537827 21067 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:39.449337 22641 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:39.453207 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.540385 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:39.281364 21796 rpcz_store.cc:276] 0709 13:06:36.347502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.347935 (+   433us) service_pool.cc:224] Handling call
0709 13:06:37.051592 (+703657us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:37.052034 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.131941 (+ 79907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.131996 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:37.385365 (+253369us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b2c00012928. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:06:37.385961 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.844988 (+459027us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:37.845041 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.845164 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:39.278665 (+1433501us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:39.280558 (+  1893us) raft_consensus.cc:1685] finished
0709 13:06:39.280585 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.280739 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114094,"raft.queue_time_us":107118,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":172,"spinlock_wait_cycles":115728256,"thread_start_us":451735,"threads_started":1,"wal-append.queue_time_us":667161}
I20260709 13:06:39.522563 23122 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:39.544888 23144 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:39.563624 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.548043 23111 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:39.572126 21067 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
W20260709 13:06:39.574265 21018 auto_rebalancer.cc:629] Failed to schedule move for tablet 40d1baf8579949fbb0d5d45a18fefe35: Illegal state: Replica 9384358ce43f4e3fac2ae44bdf11c389 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:06:39.574765 23111 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:39.581017 21018 auto_rebalancer.cc:629] Failed to schedule move for tablet de53f43e2a0e4a91a1767df521fc2b8c: Illegal state: Replica 46c1676121b64b7d9bcf2227cc639dd3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:06:39.571070 21066 raft_consensus.cc:2410] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:39.556569 23144 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:06:39.594067 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.595211 23153 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:39.596135 23153 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.596844 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:39.597683 21067 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:06:39.606621 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.571158 23136 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:06:39.606912 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.619462 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.629642 22641 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:39.630146 21214 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:39.591480 23144 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:39.636410 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.599196 23093 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.636796 21066 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:39.655711 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.646350 23153 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:39.659122 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.662186 23039 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.662871 23035 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:39.647688 23093 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.670118 23093 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:39.671502 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.671774 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.668107 23104 ts_tablet_manager.cc:1929] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:39.676620 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:39.676923 21067 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:39.679803 21212 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:39.653024 22957 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:39.682576 23104 log.cc:1199] T cc53d52fdeaa441b9e9d329e69d94440 P 46c1676121b64b7d9bcf2227cc639dd3: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/wals/cc53d52fdeaa441b9e9d329e69d94440
I20260709 13:06:39.685083 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.686888 23104 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56796 (request call id 79) took 2097 ms. Trace:
I20260709 13:06:39.687321 23104 rpcz_store.cc:276] 0709 13:06:37.589195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.589476 (+   281us) service_pool.cc:224] Handling call
0709 13:06:38.367955 (+778479us) ts_tablet_manager.cc:1151] Deleting tablet cc53d52fdeaa441b9e9d329e69d94440
0709 13:06:38.390822 (+ 22867us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:06:39.598646 (+1207824us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:39.667956 (+ 69310us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:39.686809 (+ 18853us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":652585,"tablet-delete.queue_time_us":712405,"thread_start_us":229782,"threads_started":1}
I20260709 13:06:39.687007 21213 raft_consensus.cc:2410] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:39.693534 21213 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:39.696144 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.697522 22476 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.737448 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.741304 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32968 (request call id 2) took 1093 ms. Trace:
I20260709 13:06:39.741807 21146 rpcz_store.cc:276] 0709 13:06:38.648050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.674454 (+ 26404us) service_pool.cc:224] Handling call
0709 13:06:39.741101 (+1066647us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540030af54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42516853 00007f7f474e460a 00007f7f40a6a7d4 00007f7f40a6a762 00007f7f40a6a691 00007f7f40a6a60c 00007f7f40a698c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:39.741202 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507955456}
I20260709 13:06:39.743706 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.744745 21216 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:39.746539 21216 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:06:39.751726 21797 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:39.759193 23144 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:39.764653 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 590) took 1547 ms. Trace:
I20260709 13:06:39.766659 23144 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:39.769508 23166 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:06:39.769106 23107 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:39.767077 21798 rpcz_store.cc:276] 0709 13:06:38.217240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.225500 (+  8260us) service_pool.cc:224] Handling call
0709 13:06:39.763666 (+1538166us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:39.763894 (+   228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:39.764128 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:39.764159 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:39.764210 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:39.764248 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.764315 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.764445 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28756224}
I20260709 13:06:39.792009 23107 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:39.793260 23107 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:39.794250 23107 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:39.796572 23171 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:06:39.802275 22830 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:39.804370 23103 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.574s	user 0.003s	sys 0.022s
W20260709 13:06:39.826511 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:39.826800 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.267s	user 0.021s	sys 0.010s
W20260709 13:06:39.827189 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.267s	user 0.021s	sys 0.010s
W20260709 13:06:39.827467 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 1.26769e+06us
I20260709 13:06:39.817658 22930 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.834915 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:39.836522 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:39.838907 23153 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:39.839543 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:39.840088 22904 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:39.839679 23153 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:39.840970 23072 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:39.841495 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:39.842859 21219 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:39.843922 23046 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:39.844188 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:39.844517 23093 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:39.847962 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 603) took 2101 ms. Trace:
I20260709 13:06:39.848403 21215 rpcz_store.cc:276] 0709 13:06:37.746455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.747081 (+   626us) service_pool.cc:224] Handling call
0709 13:06:39.847819 (+2100738us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b54002eead4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:39.847895 (+    76us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2072850176}
I20260709 13:06:39.850020 23132 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:39.830441 22930 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:39.856609 22930 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:39.858670 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 192) took 1496 ms. Trace:
I20260709 13:06:39.858562 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.857102 22930 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:39.859382 21214 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:39.859511 22930 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:39.860409 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.860481 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:39.864130 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.864668 21221 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:39.869310 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.871057 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 543) took 1938 ms. Trace:
I20260709 13:06:39.871480 23046 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:39.871538 21065 rpcz_store.cc:276] 0709 13:06:37.932894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.951698 (+ 18804us) service_pool.cc:224] Handling call
0709 13:06:39.869294 (+1917596us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b54002ad0d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:39.870991 (+  1697us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44947328}
I20260709 13:06:39.872272 23046 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:39.874019 21184 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b6dd6862dd14baba4a82c5a837bc802. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:39.874570 23046 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.872509 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.883178 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:39.883333 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:39.883343 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.859141 21800 rpcz_store.cc:276] 0709 13:06:38.362285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.390866 (+ 28581us) service_pool.cc:224] Handling call
0709 13:06:38.460492 (+ 69626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:38.522785 (+ 62293us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:38.572284 (+ 49499us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.893255 (+320971us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:38.893303 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:38.893348 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.858344 (+964996us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b54005e01d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:39.858385 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.858494 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14865,"prepare.queue_time_us":24110,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":19459,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5289,"spinlock_wait_cycles":142708608,"thread_start_us":264727,"threads_started":1,"wal-append.queue_time_us":387974}
I20260709 13:06:39.885946 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.886152 23111 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.885771 22597 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:39.888336 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:39.868818 23144 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:39.889945 23144 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:39.890480 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.891084 23144 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:39.892364 21143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2403) took 2423 ms (client timeout 1644 ms). Trace:
W20260709 13:06:39.892850 21143 rpcz_store.cc:269] 0709 13:06:37.468912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.572089 (+103177us) service_pool.cc:224] Handling call
0709 13:06:37.710438 (+138349us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:39.892090 (+2181652us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54004245d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:39.892255 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2746693632}
I20260709 13:06:39.885684 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.895437 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.846990 22904 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:39.895439 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:39.895500 21217 raft_consensus.cc:2368] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:39.900364 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 492) took 1381 ms. Trace:
I20260709 13:06:40.083004 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.091818 21793 rpcz_store.cc:276] 0709 13:06:38.519208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.591228 (+ 72020us) service_pool.cc:224] Handling call
0709 13:06:39.900119 (+1308891us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54006c1e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:39.900282 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383885056}
I20260709 13:06:40.055429 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:39.966986 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:39.969481 21767 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:40.095532 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 212404us
W20260709 13:06:40.019954 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.901067 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 167) took 1425 ms. Trace:
I20260709 13:06:40.096768 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:39.927127 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.018890 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:40.093478 21111 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:39.906425 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:40.020632 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.098945 21218 rpcz_store.cc:276] 0709 13:06:38.475707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.523199 (+ 47492us) service_pool.cc:224] Handling call
0709 13:06:39.900805 (+1377606us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b54000bc7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:39.900979 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29517056}
I20260709 13:06:40.018440 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:40.106645 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.995631 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:40.099581 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 130195us
I20260709 13:06:40.114526 21140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:06:40.114883 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.115182 21140 raft_consensus.cc:606] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:40.080986 23035 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.123899 21036 leader_election.cc:400] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:40.080296 23009 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:40.081172 22791 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:40.103113 23158 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:40.054601 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.090345 22964 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.126421 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.019100 23114 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:40.019378 22809 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:39.965778 22930 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:40.128459 23096 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:39.968307 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 591) took 1705 ms. Trace:
I20260709 13:06:40.146584 22791 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.162258 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:40.163188 21800 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
W20260709 13:06:39.968817 21794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 589) took 1808 ms (client timeout 1745 ms). Trace:
I20260709 13:06:40.132740 21068 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:40.078497 22874 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:39.885721 23039 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:40.093357 22830 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:40.134691 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.055521 23014 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:39.886147 23153 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:39.995177 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:40.105922 23083 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:40.135512 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.136020 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.182842 21144 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:40.183813 21800 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:40.145270 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.136359 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.146899 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.147617 21069 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:40.129736 21036 leader_election.cc:403] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:40.123899 22814 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:40.187297 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1540) took 1877 ms. Trace:
I20260709 13:06:40.128827 22957 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:39.993186 23144 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:40.110110 23023 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:06:40.176412 22476 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.176699 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:40.177591 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:40.179607 21217 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:40.150259 23009 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.159138 23106 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:40.185918 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 69685.5us
I20260709 13:06:40.165961 22809 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.187784 21070 rpcz_store.cc:276] 0709 13:06:38.310445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.525950 (+215505us) service_pool.cc:224] Handling call
0709 13:06:39.567582 (+1041632us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:39.586213 (+ 18631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:39.601828 (+ 15615us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:39.601880 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:39.601945 (+    65us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:39.602010 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.602101 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:40.187109 (+585008us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b54000dc350. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:40.187223 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6637,"raft.queue_time_us":7094,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5835,"spinlock_wait_cycles":44483200}
I20260709 13:06:40.147725 23093 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:40.174511 23057 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.194919 23144 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.156018 22964 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.157836 23114 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.177611 22791 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:40.152668 23035 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.167861 23096 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.210711 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:40.213630 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.214763 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:40.278685 21143 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:39.708267 20955 catalog_manager.cc:5028] TS 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): tablet cc53d52fdeaa441b9e9d329e69d94440 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
W20260709 13:06:40.166182 21794 rpcz_store.cc:269] 0709 13:06:38.160582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.190857 (+ 30275us) service_pool.cc:224] Handling call
0709 13:06:39.041426 (+850569us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:39.968587 (+927161us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400571054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:39.968744 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424851968}
I20260709 13:06:40.217549 23032 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:40.162313 21795 rpcz_store.cc:276] 0709 13:06:38.266417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.278959 (+ 12542us) service_pool.cc:224] Handling call
0709 13:06:38.362961 (+ 84002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:38.609916 (+246955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:38.621816 (+ 11900us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:38.648908 (+ 27092us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:38.694381 (+ 45473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.968072 (+1273691us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b540060efd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:39.968127 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.968248 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492975872}
I20260709 13:06:40.218214 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.219611 23014 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:40.220090 23009 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:40.192324 22814 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.220805 23106 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.204448 22476 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.193648 22930 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:40.185966 23083 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:40.234103 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:40.187533 21065 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:40.247121 23114 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:40.251902 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:40.252727 22809 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:40.279495 20955 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 571325us
I20260709 13:06:40.187000 23153 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.183353 22874 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.302899 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:40.303297 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:40.304495 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 86386.2us
W20260709 13:06:40.306137 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 161415us
I20260709 13:06:40.274808 23007 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:40.314059 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.315450 21146 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:40.319445 23007 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.297448 23046 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:40.323521 23119 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:40.324079 23014 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:40.325331 21147 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:40.244019 23057 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.328154 22964 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:40.332620 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.334940 22814 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:40.357331 21142 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
W20260709 13:06:40.340708 23185 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:06:40.307529 21796 raft_consensus.cc:2410] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:40.286093 23032 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.318116 23136 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:40.302843 23083 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:40.351960 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.352018 21143 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:40.352218 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:40.351154 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 219801us
I20260709 13:06:40.355156 22476 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:40.338982 23186 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.237s	user 0.002s	sys 0.000s
I20260709 13:06:40.339540 23023 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:40.309525 23113 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:40.361488 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.365478 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.369577 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.373570 21143 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:40.374616 20969 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 162) took 9222 ms. Trace:
I20260709 13:06:40.372740 21141 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:40.378098 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.378736 21147 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:40.371587 23023 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:40.379703 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.380841 23096 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:40.380935 23007 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:40.380646 23023 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.381958 23023 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:40.382469 23106 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:40.384356 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:40.385107 21794 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:40.387526 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.389197 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.362361 22809 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:40.376876 22597 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:40.375876 21111 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:40.414206 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:40.368335 23032 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:40.380966 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.424798 21035 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.402601 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.411078 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.397639 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:40.404165 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.816s (SENT) [suppressed 5 similar messages]
W20260709 13:06:40.472023 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67880.4us
I20260709 13:06:40.476284 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:40.404758 23187 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:06:40.410391 23144 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:40.391255 23153 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:40.429816 22913 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:40.476110 23122 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:40.481519 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.482203 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.492823 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.502707 21221 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:40.406711 22874 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:40.404946 23057 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:40.409058 22836 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:40.393155 23035 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:40.430053 23093 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:40.392103 22597 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.511940 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.513458 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.476485 23072 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:40.391729 22809 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.392257 23113 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.514132 21217 pending_rounds.cc:85] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 2
I20260709 13:06:40.416865 23009 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 13:06:40.375077 20969 rpcz_store.cc:276] 0709 13:06:31.146169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.146383 (+   214us) service_pool.cc:224] Handling call
0709 13:06:40.368994 (+9222611us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:38.458075 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:38.458821 (+   746us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:38.458884 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:06:38.458899 (+    15us) tablet.cc:662] Decoding operations
0709 13:06:38.459703 (+   804us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:38.459790 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:38.459825 (+    35us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:06:38.460611 (+   786us) tablet.cc:701] Row locks acquired
0709 13:06:38.460673 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:06:38.460837 (+   164us) write_op.cc:270] Start()
0709 13:06:38.460979 (+   142us) write_op.cc:276] Timestamp: P: 1783602398460774 usec, L: 0
0709 13:06:38.461020 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:06:38.461800 (+   780us) log.cc:844] Serialized 2148 byte log entry
0709 13:06:39.186794 (+724994us) op_driver.cc:464] REPLICATION: finished
0709 13:06:39.374968 (+188174us) write_op.cc:301] APPLY: starting
0709 13:06:39.375081 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:39.375254 (+   173us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:39.426327 (+ 51073us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:39.508463 (+ 82136us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:39.552899 (+ 44436us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:06:39.552970 (+    71us) write_op.cc:312] APPLY: finished
0709 13:06:39.568235 (+ 15265us) log.cc:844] Serialized 47 byte log entry
0709 13:06:39.577027 (+  8792us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:39.577154 (+   127us) write_op.cc:454] Released schema lock
0709 13:06:39.577225 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:39.577638 (+   413us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:39.577667 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":804448,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":156523,"apply.run_cpu_time_us":4418,"apply.run_wall_time_us":202671,"num_ops":5,"prepare.queue_time_us":515214,"prepare.run_cpu_time_us":12298,"prepare.run_wall_time_us":157179,"raft.queue_time_us":55948,"raft.run_cpu_time_us":29853,"raft.run_wall_time_us":115919,"replication_time_us":725615,"thread_start_us":374313,"threads_started":4,"wal-append.queue_time_us":437040}]]}
I20260709 13:06:40.486316 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:40.547724 21069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:40.549810 21215 raft_consensus.cc:2410] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
W20260709 13:06:40.549880 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 233252us
I20260709 13:06:40.554570 21217 pending_rounds.cc:107] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:06:40.562666 21069 raft_consensus.cc:606] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.562853 21217 raft_consensus.cc:2889] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:40.501134 22913 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.550352 21220 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:40.524171 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:40.505242 23122 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:40.528211 22836 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:40.576078 22791 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:40.576212 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:40.576560 22791 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:40.577026 22791 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:40.577085 23009 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:40.577514 23009 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:40.577920 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:40.547863 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:40.572028 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 89903.3us
I20260709 13:06:40.580942 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:06:40.582409 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 265982us
I20260709 13:06:40.583155 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:40.584964 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:40.585624 21067 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:40.578341 23009 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.567890 22913 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:40.565582 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:40.494369 22830 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:40.598482 21142 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:40.513147 23114 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.529385 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:40.553112 23057 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:06:40.571370 21798 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:40.579698 22836 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:40.557137 23093 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.600172 21147 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:40.574725 23122 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:40.603986 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:40.606755 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:40.704326 23193 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:06:40.704183 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.640281 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 196) took 2557 ms. Trace:
W20260709 13:06:40.650305 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.587265 22676 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:40.743141 23009 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:40.759553 23196 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
W20260709 13:06:40.705680 23188 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
W20260709 13:06:40.743672 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 160641us
W20260709 13:06:40.672907 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 90230.2us
I20260709 13:06:40.632766 21140 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:40.747356 22964 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:06:40.747484 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 143631us
I20260709 13:06:40.608629 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.704183 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:40.604633 23114 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:40.750164 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:40.608880 23199 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.139s
I20260709 13:06:40.757088 21147 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:40.739108 23050 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:40.832322 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:40.833464 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:40.834086 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52341.3us
I20260709 13:06:40.716056 23121 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:40.796887 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:40.824246 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 74139.8us
I20260709 13:06:40.848330 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.824877 21146 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:40.858085 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:40.858340 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 61542.6us
I20260709 13:06:40.847796 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.862782 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.876516 21221 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:40.848621 21185 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:40.883333 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.886197 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:40.869952 23203 ts_tablet_manager.cc:933] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:40.832453 22904 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:40.806561 22964 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:40.904917 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56369.4us
I20260709 13:06:40.920217 21146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:06:40.780500 22676 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:40.854900 22874 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:40.950420 21146 raft_consensus.cc:606] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.952260 21146 raft_consensus.cc:614] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:40.878686 23153 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:40.961535 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 101818us
W20260709 13:06:40.965775 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 130754us
I20260709 13:06:40.962427 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:40.977078 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 147943us
I20260709 13:06:40.981881 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.925354 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:40.953037 23201 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:06:40.986109 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:40.871418 23121 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:40.942268 22809 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:40.962025 23153 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:40.999799 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:41.002421 21036 leader_election.cc:400] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:41.004220 21036 leader_election.cc:403] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:06:41.004287 21070 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:40.982264 22964 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:40.952091 22874 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:40.988443 22930 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:40.983582 22814 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:41.013921 23212 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:06:41.013355 20967 catalog_manager.cc:5697] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:41.006453 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62902.5us
I20260709 13:06:40.941985 23113 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:41.026993 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45171us
I20260709 13:06:41.016418 20967 catalog_manager.cc:5697] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:41.032083 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.035002 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:41.036830 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:41.007134 23153 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:41.006798 23132 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:41.035847 21144 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:06:41.009898 22930 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.742388 21064 rpcz_store.cc:276] 0709 13:06:38.082670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.443684 (+361014us) service_pool.cc:224] Handling call
0709 13:06:39.870753 (+1427069us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:39.871219 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:39.871252 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:39.871903 (+   651us) log.cc:844] Serialized 52 byte log entry
0709 13:06:40.130085 (+258182us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:40.135197 (+  5112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.148780 (+ 13583us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:40.480302 (+331522us) raft_consensus.cc:1685] finished
0709 13:06:40.481002 (+   700us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:40.639810 (+158808us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002ad0d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:40.640148 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2729,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":3088237824,"thread_start_us":233558,"threads_started":1,"wal-append.queue_time_us":329219}
I20260709 13:06:40.993022 23122 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.062566 20967 catalog_manager.cc:5697] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:41.117650 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.137856 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:41.017743 23046 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:41.006349 22791 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.147714 21070 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:41.148923 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.149811 21070 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:41.105962 22874 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.113595 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.107885 22930 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:41.114929 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.031179 22814 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.172817 20967 catalog_manager.cc:5697] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 2 to 3, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:41.116279 23121 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:41.117089 23114 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:41.120569 23083 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.734s	user 0.007s	sys 0.003s
I20260709 13:06:41.117595 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:41.063370 22676 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:41.130808 23032 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.655s	user 0.005s	sys 0.002s
W20260709 13:06:41.210538 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 93015.6us
I20260709 13:06:41.137161 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:41.212109 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 75091.5us
I20260709 13:06:41.156299 21067 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:41.165620 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.175442 21141 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:41.140774 23106 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:41.131526 22904 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:41.169590 23192 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.590s	user 0.003s	sys 0.003s
I20260709 13:06:41.144779 23046 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:41.154726 23113 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:41.148145 23222 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:41.216455 23106 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:41.200232 23083 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.226548 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.154341 23007 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:06:41.188241 23121 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.231714 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:41.229781 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 81899.1us
I20260709 13:06:41.189395 20967 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:41.187175 23189 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:41.226311 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 88563.5us
I20260709 13:06:41.217767 23114 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:41.208425 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.178470 23220 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:41.219790 23072 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:41.232498 23046 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:41.268946 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.246248 23106 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:41.272259 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 243728us
I20260709 13:06:41.220892 23032 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.235412 22874 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:41.294936 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c20ca56847f64504a85694d10f47558e"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:06:41.300669 21144 raft_consensus.cc:606] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:41.278244 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:41.300668 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56416.2us
I20260709 13:06:41.302532 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:41.269786 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.318432 21067 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:41.275315 23234 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:41.324771 23234 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.329249 23234 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:41.326912 23230 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.330119 23230 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.275758 23121 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.331334 23083 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:41.334618 23083 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.286572 23220 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.336524 23032 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:41.337275 23220 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:41.301287 23229 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:41.338264 23229 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.338785 23229 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.340049 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:41.271027 23189 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.324810 23072 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.286077 23022 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:06:41.349795 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:06:41.361737 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 76890.9us
I20260709 13:06:41.334872 23153 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.341204 23229 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:41.335121 23083 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.407466 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:41.411783 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 109340us
I20260709 13:06:41.413014 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:41.414449 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:41.416397 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:41.371434 22814 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:41.423328 23039 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:41.424038 23039 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:41.424945 23039 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.423182 23225 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.427737 23235 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:41.428063 23241 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:41.428329 23235 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.428637 23241 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.428848 23235 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:41.429150 23241 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:41.427762 23225 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.379791 23238 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.380191 23122 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:41.357671 23233 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:41.324693 22904 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.437304 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:41.404175 22791 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:41.427984 23219 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:06:41.433604 23225 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:41.439816 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:41.439999 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:41.447595 21216 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:06:41.445791 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 404008us
I20260709 13:06:41.454339 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:41.458148 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.361086 23072 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.440514 23240 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:41.453758 23022 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:41.451611 22964 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.468416 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:41.473493 21145 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:41.474501 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:41.476537 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:41.477821 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.480652 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:41.481344 21146 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:41.481876 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:41.460300 23096 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:41.519480 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45040.2us
W20260709 13:06:41.534700 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 97465.6us
W20260709 13:06:41.548189 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66393.5us
I20260709 13:06:41.459794 21213 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:41.471923 22791 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:41.470824 23022 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:41.480572 23225 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:41.442780 23238 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.550350 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.550392 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:41.519204 23239 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:41.600140 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49910.4us
I20260709 13:06:41.522940 23227 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:41.606248 23240 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:41.608580 23250 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:06:41.606841 21765 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.609309 23238 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:41.609907 23245 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:41.610214 23007 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:41.610630 23007 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:41.610980 23007 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:41.614203 23246 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:41.616451 23225 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:41.619500 22964 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:41.564343 22791 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.620316 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.555295 23022 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:41.549921 23229 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:06:41.639781 22836 consensus_meta.cc:287] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.895s	user 0.006s	sys 0.007s
I20260709 13:06:41.647457 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:41.603401 22930 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.644416 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1766) took 1038 ms. Trace:
I20260709 13:06:41.603662 23230 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:41.659523 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 183084us
W20260709 13:06:41.635284 23191 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.007s	sys 0.019s
I20260709 13:06:41.658134 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.551532 23096 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.632668 22964 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:41.699923 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 115854us
I20260709 13:06:41.625231 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.704411 21072 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:06:41.643172 23007 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.681283 21796 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:41.681740 23249 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:41.681862 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:41.683338 23241 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.684726 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:41.670178 21141 rpcz_store.cc:276] 0709 13:06:40.606427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.608959 (+  2532us) service_pool.cc:224] Handling call
0709 13:06:40.609245 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.215343 (+606098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.644112 (+428769us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b540042f754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:41.644343 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1301545728}
I20260709 13:06:41.702095 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:41.658210 22791 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:41.706215 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:41.706588 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 156311us
I20260709 13:06:41.688999 23153 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:41.678638 23191 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.007s	sys 0.019s
W20260709 13:06:41.705770 22913 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.104s	user 0.009s	sys 0.005s
I20260709 13:06:41.702904 22964 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:41.707978 23083 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:41.735801 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:41.736860 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:41.673874 22836 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.720369 23230 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.720064 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:41.730530 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.732256 23236 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.737064 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:41.753907 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.736724 23207 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:41.738996 23249 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.744722 22904 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:41.759467 21145 raft_consensus.cc:2368] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5.
W20260709 13:06:41.830740 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 76994.3us
I20260709 13:06:41.760015 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:41.750813 22964 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:41.816278 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:41.844995 21185 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:41.846014 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.846645 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:41.753306 23158 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:41.849020 21765 leader_election.cc:400] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5.
I20260709 13:06:41.846549 22930 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:41.854018 21765 leader_election.cc:403] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:06:41.770668 23241 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:41.815846 21793 raft_consensus.cc:2410] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:06:41.751441 22913 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.772864 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:41.772606 23230 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:41.761795 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:06:41.851208 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 91293.1us
I20260709 13:06:41.759845 23072 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:41.832183 23245 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.848217 23231 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:41.872538 22913 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:41.854882 23132 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:41.903901 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:41.900942 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:41.875092 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:41.933806 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:41.838171 23249 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:41.861400 21792 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:41.861500 23158 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.875401 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:41.765978 23046 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.766553 23114 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:41.888931 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:41.944211 21071 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:41.884021 23262 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:41.944830 21793 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:41.945322 23262 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.946022 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:41.871183 23007 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:41.856108 23207 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.902233 22597 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:41.947512 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:41.949035 21797 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:41.891618 21138 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:06:41.949628 21138 pending_rounds.cc:85] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 6
I20260709 13:06:41.949883 21138 pending_rounds.cc:107] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:06:41.949806 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:41.950273 21138 raft_consensus.cc:2889] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:41.896849 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:41.951185 22597 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:06:41.951725 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54974.8us
I20260709 13:06:41.952234 23114 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:41.952612 23114 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:41.952642 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:41.953485 23114 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.954715 23114 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:41.901669 23230 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:41.899762 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:41.956158 22836 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:41.956826 22836 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:41.957167 22836 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:41.957516 22836 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:41.957590 22676 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:41.958171 22676 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.958588 22676 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:41.960393 23235 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.001s
I20260709 13:06:41.960839 22874 raft_consensus.cc:993] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: : Instructing follower 46c1676121b64b7d9bcf2227cc639dd3 to start an election
I20260709 13:06:41.960804 23235 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.961498 22874 raft_consensus.cc:1081] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Signalling peer 46c1676121b64b7d9bcf2227cc639dd3 to start an election
I20260709 13:06:41.900405 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:41.889595 23217 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:41.900650 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:41.823792 23236 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:41.963573 23217 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:41.966313 23236 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:41.872907 21218 raft_consensus.cc:2368] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7.
I20260709 13:06:41.906278 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:41.970526 21143 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:41.900070 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:41.901736 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:41.900219 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1557) took 1116 ms. Trace:
I20260709 13:06:41.973666 21070 rpcz_store.cc:276] 0709 13:06:40.783921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.274276 (+490355us) service_pool.cc:224] Handling call
0709 13:06:41.294600 (+ 20324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.299988 (+  5388us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:41.379343 (+ 79355us) log.cc:844] Serialized 14 byte log entry
0709 13:06:41.445009 (+ 65666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:41.445081 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:41.445137 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.899979 (+454842us) spinlock_profiling.cc:243] Waited 5.1 ms on lock 0x7b54003a2fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:41.900021 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:41.900149 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":131799,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":93,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":9529,"spinlock_wait_cycles":11725696,"thread_start_us":127967,"threads_started":2,"wal-append.queue_time_us":170651}
I20260709 13:06:41.978523 23096 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:41.980373 23251 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
I20260709 13:06:41.868320 23256 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 13:06:41.881978 22904 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:41.893548 23189 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:41.914347 23106 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.610s	user 0.005s	sys 0.002s
W20260709 13:06:41.912081 23202 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.787s	user 0.000s	sys 0.013s
I20260709 13:06:41.903509 23072 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.904533 23144 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.903126 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:41.905880 23136 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:41.883570 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.941790 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:41.878813 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:41.943199 23254 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:41.907599 23093 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:41.904067 23039 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.904626 23083 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:41.940714 22930 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:41.882543 23239 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 13:06:41.908017 23245 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:42.173322 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 324467us
I20260709 13:06:42.084242 21071 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
W20260709 13:06:42.215915 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 278892us
I20260709 13:06:42.027303 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.216735 21070 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
W20260709 13:06:42.216039 21071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1559) took 1430 ms (client timeout 1616 ms). Trace:
W20260709 13:06:42.219455 21071 rpcz_store.cc:269] 0709 13:06:40.785597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.925352 (+139755us) service_pool.cc:224] Handling call
0709 13:06:41.888709 (+963357us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:42.215977 (+327268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42788224}
I20260709 13:06:42.085449 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 179) took 1236 ms. Trace:
I20260709 13:06:42.220628 21218 rpcz_store.cc:276] 0709 13:06:40.848956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.857730 (+  8774us) service_pool.cc:224] Handling call
0709 13:06:42.085269 (+1227539us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001257d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:42.085387 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2331181952}
I20260709 13:06:42.015241 21143 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:06:42.216763 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:42.022424 23106 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.235464 22930 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:42.043920 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.236706 23106 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:42.031898 23144 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.041894 23039 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.238996 23144 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.242511 23039 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.076511 23114 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:42.041469 23254 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:42.244181 23254 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:42.228924 22904 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:42.254050 22904 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.256814 22904 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.261921 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:42.263057 23132 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:42.199684 23236 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:42.264024 23236 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.264741 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:42.265434 21065 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:42.266261 23093 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:42.266431 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:42.268270 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:42.268778 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 554) took 2877 ms. Trace:
I20260709 13:06:42.269176 21063 rpcz_store.cc:276] 0709 13:06:39.391784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.431329 (+ 39545us) service_pool.cc:224] Handling call
0709 13:06:40.243874 (+812545us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:40.288384 (+ 44510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:40.352217 (+ 63833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.357457 (+  5240us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:40.393561 (+ 36104us) log.cc:844] Serialized 26 byte log entry
0709 13:06:40.715698 (+322137us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:40.727025 (+ 11327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.727128 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:41.911459 (+1184331us) spinlock_profiling.cc:243] Waited 910 us on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:42.268525 (+357066us) raft_consensus.cc:1685] finished
0709 13:06:42.268553 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.268717 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1008,"raft.queue_time_us":8447,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":114,"spinlock_wait_cycles":9953920,"thread_start_us":240240,"threads_started":1,"wal-append.queue_time_us":629546}
W20260709 13:06:42.275871 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 373742us
I20260709 13:06:42.281638 21765 leader_election.cc:400] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7.
I20260709 13:06:42.047416 22874 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:42.085820 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.214620 23235 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.048905 23158 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.267068 23236 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:42.289734 23203 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 1.165s	user 0.006s	sys 0.023s
I20260709 13:06:42.048905 21064 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:42.294293 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:42.274526 23136 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5.
I20260709 13:06:42.297911 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.318783 21220 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:42.298671 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.299229 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.302223 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.320732 21070 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:42.280889 23233 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:42.023819 23072 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.042126 23083 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.208810 23262 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.143673 23022 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:42.306703 23203 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.006s	sys 0.024s
I20260709 13:06:42.302151 23093 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:42.296095 22904 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:42.317405 23236 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:42.342960 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.333520 23233 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.353801 23233 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.327020 23222 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.355113 23222 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.355841 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:42.356235 23210 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.109s	user 0.000s	sys 0.008s
I20260709 13:06:42.357561 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.358233 21066 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:42.361846 23072 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.362573 22904 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.364120 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.364799 21064 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:42.364979 22904 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:42.339042 23234 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.009s	user 0.012s	sys 0.000s
I20260709 13:06:42.365852 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.365700 23234 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.366446 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 172) took 2220 ms. Trace:
I20260709 13:06:42.304301 22874 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.368461 21800 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:42.368939 21799 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:42.370847 23203 tablet_copy_client.cc:323] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
W20260709 13:06:42.365878 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 457775us
I20260709 13:06:42.366933 21139 rpcz_store.cc:276] 0709 13:06:40.146044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.155897 (+  9853us) service_pool.cc:224] Handling call
0709 13:06:40.166403 (+ 10506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:40.185089 (+ 18686us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:40.412028 (+226939us) log.cc:844] Serialized 14 byte log entry
0709 13:06:40.835340 (+423312us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:40.835396 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:40.835443 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.366119 (+1530676us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b540036db54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:42.366168 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.366333 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":206798,"prepare.queue_time_us":401547,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":251062,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":27924,"spinlock_wait_cycles":58221056,"thread_start_us":564043,"threads_started":3,"wal-append.queue_time_us":816712}
I20260709 13:06:42.351006 22964 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.366515 21064 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:42.396452 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.403904 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.404618 23050 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:42.404666 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 8b6dd6862dd14baba4a82c5a837bc802 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:06:42.405493 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 8b6dd6862dd14baba4a82c5a837bc802 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-8b6dd6862dd14baba4a82c5a837bc802
I20260709 13:06:42.405229 23050 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.408190 23050 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.409489 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:42.410645 23106 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:42.412024 21070 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:42.419533 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.431306 21231 tablet_copy_source_session.cc:215] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:42.433408 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:42.439945 23083 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.441531 21064 pending_rounds.cc:85] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 2
I20260709 13:06:42.441982 21064 pending_rounds.cc:107] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:06:42.442535 21064 raft_consensus.cc:2889] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:42.418259 22676 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.445007 23222 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.446529 22676 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.446916 23234 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.447659 21792 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:42.451617 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 131587us
I20260709 13:06:42.454255 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:42.455426 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:42.448393 23245 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:42.430986 23022 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.454566 23190 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:42.438576 23249 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.469612 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:42.458153 23236 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:42.472806 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 134331us
I20260709 13:06:42.472918 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 579) took 1004 ms. Trace:
I20260709 13:06:42.475600 21070 rpcz_store.cc:276] 0709 13:06:41.468347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.471086 (+1002739us) service_pool.cc:224] Handling call
0709 13:06:42.472438 (+  1352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.472605 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.472635 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:42.472673 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:42.472699 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.472755 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.472858 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:42.460557 23253 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:42.463967 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.460299 23205 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:42.465271 23245 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:42.454684 23254 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.470170 23189 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.492015 21070 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:42.492898 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 580) took 1024 ms. Trace:
I20260709 13:06:42.493305 21070 rpcz_store.cc:276] 0709 13:06:41.468577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.476962 (+1008385us) service_pool.cc:224] Handling call
0709 13:06:42.491585 (+ 14623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.492553 (+   968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.492817 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:42.471796 22676 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.479991 23114 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:42.465982 23190 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.502130 23114 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:42.502704 23114 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:42.507797 23121 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.481722 23230 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:42.513183 23230 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.515303 22874 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:42.472255 23273 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:42.482144 22836 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:06:42.479619 23236 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:42.499236 21214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 641) took 2007 ms (client timeout 1559 ms). Trace:
W20260709 13:06:42.500214 23204 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.259s	user 0.007s	sys 0.009s
W20260709 13:06:42.523878 21214 rpcz_store.cc:269] 0709 13:06:40.492060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.495202 (+  3142us) service_pool.cc:224] Handling call
0709 13:06:42.460158 (+1964956us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:42.499155 (+ 38997us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42333056}
I20260709 13:06:42.500522 23249 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.481312 23253 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:42.528671 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 79446.8us
I20260709 13:06:42.533779 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:42.542454 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 584) took 1073 ms. Trace:
I20260709 13:06:42.542877 21070 rpcz_store.cc:276] 0709 13:06:41.469501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.534089 (+1064588us) service_pool.cc:224] Handling call
0709 13:06:42.541918 (+  7829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.542111 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.542134 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:42.542175 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:42.542212 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.542269 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.542393 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:42.546542 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 585) took 1066 ms. Trace:
I20260709 13:06:42.547690 21070 rpcz_store.cc:276] 0709 13:06:41.479741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.543859 (+1064118us) service_pool.cc:224] Handling call
0709 13:06:42.545981 (+  2122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.546169 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.546196 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:42.546244 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:42.546278 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.546338 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.546449 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:42.525384 23236 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:42.545169 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1574) took 1514 ms. Trace:
I20260709 13:06:42.522744 22676 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.551198 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.532249 23253 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:42.551910 21212 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:42.520355 23217 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:42.496819 23245 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.550082 21067 rpcz_store.cc:276] 0709 13:06:41.030974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.324094 (+293120us) service_pool.cc:224] Handling call
0709 13:06:41.465483 (+141389us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:41.476003 (+ 10520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:41.476033 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:41.549653 (+ 73620us) log.cc:844] Serialized 52 byte log entry
0709 13:06:42.362779 (+813126us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b180004bf40. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:06:42.470918 (+108139us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:42.487807 (+ 16889us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.496437 (+  8630us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.530108 (+ 33671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.530223 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.544996 (+ 14773us) raft_consensus.cc:1685] finished
0709 13:06:42.545015 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.545125 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45646,"prepare.queue_time_us":43270,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":94,"raft.queue_time_us":84964,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":37139,"spinlock_wait_cycles":15864960,"thread_start_us":107951,"threads_started":1,"wal-append.queue_time_us":946576}
I20260709 13:06:42.557673 22676 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.527014 22836 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.561343 22836 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.564620 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.565181 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.566061 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.566648 21143 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:42.546491 23230 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.568321 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.574290 23254 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.498394 23189 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.578910 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 586) took 1098 ms. Trace:
I20260709 13:06:42.579931 21070 rpcz_store.cc:276] 0709 13:06:41.480749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.548475 (+1067726us) service_pool.cc:224] Handling call
0709 13:06:42.578403 (+ 29928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.578602 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.578626 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:42.578665 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:42.578696 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.578748 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.578848 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:42.525236 23273 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.590022 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:42.591471 22964 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.592291 22904 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:42.592867 22904 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:42.594411 23198 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:42.490222 23205 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.595232 23198 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:42.596485 23198 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.597909 23205 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:42.655318 21215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2275) took 1808 ms (client timeout 1500 ms). Trace:
I20260709 13:06:42.662544 21068 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:42.609122 21036 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.500s (SENT)
I20260709 13:06:42.691565 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:42.689023 21215 rpcz_store.cc:269] 0709 13:06:40.846624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.846736 (+   112us) service_pool.cc:224] Handling call
0709 13:06:42.640865 (+1794129us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b540013c754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:42.655183 (+ 14318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1949604736}
I20260709 13:06:42.598451 23189 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:42.690508 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:42.716105 23228 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:42.717196 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 108119us
I20260709 13:06:42.718044 23228 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:42.718552 23228 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.695399 21219 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 2. Current term is 3. Ops: []
W20260709 13:06:42.673991 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 71261.5us
I20260709 13:06:42.696480 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.719677 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.697150 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.720777 21144 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:42.720959 23228 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.721678 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:42.722484 21144 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:42.722559 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:42.722803 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.723356 23230 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:42.723706 23228 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:42.723939 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.724257 23228 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:42.724560 21070 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:42.724714 23228 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:42.724956 23207 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:42.725463 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.727953 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.728701 21221 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:42.690583 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.730005 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:42.730399 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.731576 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.732159 21221 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:42.729312 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:42.736214 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:42.736433 23245 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.697273 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.738515 21139 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:42.698469 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.739557 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.739748 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.739916 23265 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:42.697675 23046 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.740787 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.697068 21143 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:42.700929 23244 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.147s	user 0.005s	sys 0.008s
I20260709 13:06:42.741400 21140 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:42.675417 22904 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:42.701736 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1576) took 1260 ms. Trace:
I20260709 13:06:42.742238 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.608733 23254 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:42.709201 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:42.742794 23224 raft_consensus.cc:993] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: : Instructing follower b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:42.697675 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.743438 23224 raft_consensus.cc:1081] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Signalling peer b70fc04de37e4239a8336534f0c9c0fb to start an election
I20260709 13:06:42.710536 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:42.709748 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:42.711422 23217 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.699784 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:42.745558 23217 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.712358 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:42.747674 23217 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.711874 23039 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:42.713138 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:42.748626 21801 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:42.712287 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:42.708731 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:42.719789 21146 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:42.743088 23254 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.754456 23039 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:42.709568 23205 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.742636 21069 rpcz_store.cc:276] 0709 13:06:41.441097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.441893 (+   796us) service_pool.cc:224] Handling call
0709 13:06:41.442182 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.442389 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:41.452122 (+  9733us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.701167 (+1249045us) spinlock_profiling.cc:243] Waited 996 us on lock 0x7b1800298840. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a
0709 13:06:42.701384 (+   217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:42.701446 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:42.701497 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.701565 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.701678 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11385,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":6499,"raft.queue_time_us":2075,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":78,"spinlock_wait_cycles":2291456,"thread_start_us":6700,"threads_started":1,"wal-append.queue_time_us":17415,"wal-append.run_cpu_time_us":1758,"wal-append.run_wall_time_us":1136587}
I20260709 13:06:42.744246 21799 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:42.742152 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.751964 22964 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:42.756325 21143 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:42.756549 22964 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:42.756685 23242 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:42.757138 23242 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:42.755854 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.752959 23249 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:42.753254 23190 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.739612 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:42.755041 23039 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:42.689512 23273 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.751317 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:42.785982 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.757503 21143 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:42.799546 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.733151 23114 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.801265 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.607972 21066 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:42.721683 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.807979 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.738437 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.810416 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.805682 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:42.805743 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:42.806365 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 55211.1us
I20260709 13:06:42.801494 21144 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:42.808501 21141 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:42.809088 23241 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:06:42.759601 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 215) took 1649 ms. Trace:
I20260709 13:06:42.811326 21068 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:42.825160 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.815995 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:42.827018 21138 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:42.823335 21072 rpcz_store.cc:276] 0709 13:06:41.110581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.324817 (+214236us) service_pool.cc:224] Handling call
0709 13:06:41.337457 (+ 12640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.704966 (+367509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.759234 (+1054268us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b540001b3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:42.759512 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843073792}
I20260709 13:06:42.830111 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:42.831162 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.832239 23237 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:42.832855 23046 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:42.833320 23046 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:42.833516 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 573) took 1366 ms. Trace:
I20260709 13:06:42.815225 23072 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:42.833700 23237 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:42.834334 23237 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:42.834179 23072 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.811890 23136 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:42.836310 23136 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:42.836405 21065 rpcz_store.cc:276] 0709 13:06:41.466656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.043531 (+576875us) service_pool.cc:224] Handling call
0709 13:06:42.833319 (+789788us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b540019dcd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:42.833456 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65646848}
I20260709 13:06:42.837306 23136 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.838567 23136 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:42.838742 23237 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.843918 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.839996 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.844615 21220 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:42.851881 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.856082 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:42.842468 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:42.857965 21138 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:42.824887 21799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:54648
I20260709 13:06:42.859081 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:42.859463 21799 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:42.859675 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.859926 21799 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:42.861483 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:42.837697 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.869782 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.859705 21801 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:42.870939 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.833724 23046 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:42.873546 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:42.874512 23211 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:42.875398 23245 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:42.875813 23245 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:42.876160 23245 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:42.876106 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:42.841761 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:42.879776 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1577) took 1436 ms. Trace:
I20260709 13:06:42.880549 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:42.880226 21064 rpcz_store.cc:276] 0709 13:06:41.443263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.440725 (+997462us) service_pool.cc:224] Handling call
0709 13:06:42.441281 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:42.451863 (+ 10582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.453543 (+  1680us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:42.482234 (+ 28691us) log.cc:844] Serialized 26 byte log entry
0709 13:06:42.694639 (+212405us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:42.823385 (+128746us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.879453 (+ 56068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.879537 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.879586 (+    49us) raft_consensus.cc:1685] finished
0709 13:06:42.879604 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.879721 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10746,"prepare.queue_time_us":76623,"thread_start_us":133020,"threads_started":1,"wal-append.queue_time_us":194075}
I20260709 13:06:42.881515 23009 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:42.882220 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.780704 23205 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.872056 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.885076 23205 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.886319 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.886574 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.887727 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:42.887995 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:42.653469 23253 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.723848 23230 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.891652 23237 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:42.892268 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.898178 21219 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:42.900002 23254 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.801265 23039 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:42.900686 21219 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:42.811965 22930 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:06:42.903643 21069 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:42.904184 22930 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:42.904594 22930 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:42.905040 21145 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:42.905352 22930 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:42.906040 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 61719.3us
I20260709 13:06:42.906605 22930 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:42.907552 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:42.908254 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.908778 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.909488 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:42.910112 23276 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:42.910367 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.915808 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1582) took 1017 ms. Trace:
I20260709 13:06:42.910683 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:42.927000 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44088.3us
W20260709 13:06:43.046047 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.923705 21068 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:42.949431 21219 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:42.882969 21064 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:42.916343 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.875195 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:42.913911 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:43.070351 21143 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:42.929594 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:42.815599 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 589) took 1248 ms. Trace:
I20260709 13:06:43.072768 21144 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:43.068964 21214 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:42.892031 23236 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.076670 23205 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:43.077589 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 202461us
I20260709 13:06:43.069691 21799 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.064925 21067 rpcz_store.cc:276] 0709 13:06:41.898150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.556049 (+657899us) service_pool.cc:224] Handling call
0709 13:06:42.556594 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:42.557068 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.557098 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:42.558146 (+  1048us) log.cc:844] Serialized 513 byte log entry
0709 13:06:42.584084 (+ 25938us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:42.584139 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.584240 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.915491 (+331251us) raft_consensus.cc:1685] finished
0709 13:06:42.915521 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.915729 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":25411,"threads_started":1,"wal-append.queue_time_us":285704}
I20260709 13:06:43.087129 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:43.090818 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.081924 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:42.757648 22964 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.008280 23225 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.815797 23190 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:42.889082 21063 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:42.979040 23132 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:42.948282 22597 consensus_queue.cc:1175] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" responder_term: 4 status { last_received { term: 4 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 4 index: 5 } }
I20260709 13:06:42.822769 22904 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:43.092796 23114 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.080446 21066 rpcz_store.cc:276] 0709 13:06:41.565358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.357206 (+791848us) service_pool.cc:224] Handling call
0709 13:06:42.813240 (+456034us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:43.104883 23225 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:42.882296 23228 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.093325 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.140700 21214 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:43.141875 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.097654 21799 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:42.979099 23254 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:43.008437 23093 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:43.098870 21070 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:43.078966 23211 raft_consensus.cc:2764] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:06:42.757982 23242 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.015267 23083 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.104130 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:42.994506 21796 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:43.093829 23121 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:06:43.110855 23230 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.111857 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:42.979270 23106 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 13:06:43.114763 22791 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:43.081115 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.133163 23225 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:43.116355 23132 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:43.153195 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.165953 23093 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.206225 23093 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.183707 22836 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:43.164741 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:43.171778 21216 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:43.171767 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.202673 21221 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:43.176178 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 72139.9us
I20260709 13:06:43.207890 23093 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:43.195441 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:43.198119 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 117068us
I20260709 13:06:43.220492 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.088460 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:43.114738 23190 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.203876 23242 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:43.221589 21067 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:43.164188 23245 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.239248 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.166126 23254 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.240039 21801 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:06:43.240461 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.214524 21110 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.191664 21065 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:43.207082 23132 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:43.206779 22814 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:43.241431 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 165526us
I20260709 13:06:43.245868 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:43.235802 21213 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:43.212150 21066 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:43.251016 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.189416 23083 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.254176 21064 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:43.263020 21212 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:43.253196 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.258467 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:43.212536 22836 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:43.273068 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.248411 22814 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:43.277160 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.280000 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:43.280791 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:43.259130 23273 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:43.285107 23273 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.285543 23273 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:43.254968 23107 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:43.254753 23205 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.235489 22791 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:06:43.283751 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:43.295817 21145 raft_consensus.cc:2368] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:43.284358 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:43.274621 22814 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:43.299974 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:43.294893 23290 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:06:43.301023 21217 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
W20260709 13:06:43.301441 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 224311us
I20260709 13:06:43.239457 23093 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.307606 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.281378 23228 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.274921 22836 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:43.306814 21036 leader_election.cc:400] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:43.303117 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:43.312415 21036 leader_election.cc:403] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:06:43.311463 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:43.318265 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.322623 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:43.324700 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:43.322381 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.302150 23236 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.333596 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:43.335721 23217 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:43.344125 23249 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:43.347414 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:43.395223 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.357360 23245 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.358845 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:43.363907 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:43.366595 23291 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.082s	user 0.002s	sys 0.000s
I20260709 13:06:43.371310 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:43.379745 21221 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:43.402303 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.321252 21213 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c20ca56847f64504a85694d10f47558e"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:56816
I20260709 13:06:43.404031 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:43.322767 23288 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:06:43.320866 23234 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:43.397603 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.392611 23009 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:43.373739 23046 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.501s	user 0.001s	sys 0.009s
I20260709 13:06:43.316550 23136 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:06:43.422251 21801 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:43.409261 21066 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:43.385839 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.320381 23295 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:43.428619 21069 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.002s
I20260709 13:06:43.416116 23046 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.367211 23205 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:43.432164 21069 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:43.407362 22904 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:43.450557 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:43.396803 23107 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:06:43.413436 23009 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.470935 23306 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.472151 23217 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:43.472754 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 75387.2us
W20260709 13:06:43.476531 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 158052us
I20260709 13:06:43.468889 21068 raft_consensus.cc:2376] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4. Re-sending same reply.
W20260709 13:06:43.480432 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 94712.6us
I20260709 13:06:43.490252 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:43.417846 21213 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:43.491797 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:43.492492 21215 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:43.493656 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 122492us
W20260709 13:06:43.496393 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 132544us
W20260709 13:06:43.480108 23289 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
W20260709 13:06:43.480511 23291 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:06:43.497768 21801 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:43.499118 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:43.476409 22904 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.478317 22791 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:43.491200 23144 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:43.468116 23205 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:43.514535 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:43.488369 23009 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:43.505017 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 107530us
I20260709 13:06:43.507828 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.488747 23114 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.492201 21213 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.476042 23276 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:43.515445 21140 raft_consensus.cc:2368] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7.
I20260709 13:06:43.477085 23245 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:06:43.519412 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:43.522358 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.527825 21068 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:43.528707 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:43.507918 22904 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:43.533157 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:43.505865 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:43.535702 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:43.520543 23205 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:43.514909 23144 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.511448 23253 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.545436 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:43.534195 23225 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.530771 23297 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:43.524549 23303 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:43.509061 23234 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:43.528919 23083 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:43.534029 23050 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:43.514355 23039 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:06:43.537467 23254 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.567222 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:43.586648 21146 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:43.587083 21065 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:43.574796 23234 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.592797 23046 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:43.593385 23292 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:06:43.593528 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:43.557065 21796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 206) took 1868 ms (client timeout 2378 ms). Trace:
I20260709 13:06:43.576251 23217 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:43.594507 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 344915us
W20260709 13:06:43.594962 21796 rpcz_store.cc:269] 0709 13:06:41.686255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.902844 (+216589us) service_pool.cc:224] Handling call
0709 13:06:43.544307 (+1641463us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54003bb854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:43.554160 (+  9853us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1775834880}
I20260709 13:06:43.597046 23190 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.576814 23132 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:43.544564 22930 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:06:43.579062 23273 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.580286 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:43.580746 21140 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:43.559999 23303 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.584455 23203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6dd6862dd14baba4a82c5a837bc802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:43.585431 23072 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.598701 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:43.558408 23297 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.597656 23190 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:06:43.602449 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 68875.5us
I20260709 13:06:43.552706 23144 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.604883 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:43.605296 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.606317 21138 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:43.602305 21216 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:43.560808 23237 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.668s	user 0.013s	sys 0.004s
W20260709 13:06:43.585482 23294 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:06:43.629179 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:43.637291 21214 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:43.624089 23205 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.598011 23050 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.632345 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:43.610796 23225 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:43.639393 21143 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:43.640501 21146 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:06:43.644371 21147 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:43.618777 23297 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.615844 23303 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.638561 23305 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.640925 23050 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:43.643357 23225 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:43.658985 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 123221us
I20260709 13:06:43.663178 21071 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:43.665974 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.666880 21071 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:43.669059 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:43.680671 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 75849.4us
I20260709 13:06:43.687402 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:43.655421 23306 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:06:43.701647 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 168557us
I20260709 13:06:43.702147 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.660751 23225 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:43.653189 23035 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:06:43.710531 21071 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:06:43.711839 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54594.6us
I20260709 13:06:43.713333 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.656030 23312 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:06:43.704018 23306 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.717413 23237 raft_consensus.cc:2955] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:43.708248 22814 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.721041 23318 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:43.722512 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.723750 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.725021 23218 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:43.710618 22836 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.754025 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:43.766108 23093 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.756691 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.760756 22814 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.762904 23050 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.705120 23234 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:06:43.728607 23306 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:43.728430 23304 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:43.780654 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.780983 21220 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:43.781847 21215 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:43.785382 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.744490 23218 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:43.787271 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:43.762974 21065 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:06:43.790496 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 88435.5us
I20260709 13:06:43.790584 21111 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:43.791700 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:43.793406 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:43.793334 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:43.788657 23304 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:43.795300 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.797168 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:43.798063 23050 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:43.798484 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:43.799191 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:43.792546 22814 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:43.802840 22814 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.803238 22814 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:43.803994 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:43.804611 23302 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:43.805059 23302 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.805094 21065 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:43.805510 23302 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:43.806419 21065 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:43.793969 23035 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:43.794107 23273 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.808655 23234 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.811354 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:43.812299 21065 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:43.786907 23132 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.826408 23205 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.826964 22836 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.827999 23144 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.830333 23009 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.833541 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 598) took 1099 ms. Trace:
I20260709 13:06:43.834913 21111 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:43.835708 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 169791us
I20260709 13:06:43.836450 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.836530 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:43.836881 23305 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.837577 23305 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:43.837625 23313 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.838085 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.838306 23305 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:43.838496 23313 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:43.838955 23313 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.839349 23093 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:43.839442 23313 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.839818 23093 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.840281 23093 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:43.840399 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:43.833983 21067 rpcz_store.cc:276] 0709 13:06:42.733847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.573306 (+839459us) service_pool.cc:224] Handling call
0709 13:06:43.573634 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.573828 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:43.574180 (+   352us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.833196 (+259016us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.833250 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:43.833291 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.833364 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.833482 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":176482,"prepare.queue_time_us":9506,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":37,"raft.queue_time_us":177356,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":82179,"threads_started":1,"wal-append.queue_time_us":180647}
I20260709 13:06:43.849407 23303 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.844769 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.839003 23305 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:43.851801 23132 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:43.852049 23305 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:43.852216 21218 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:43.852962 23305 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.854139 22814 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.851352 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.856449 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.857244 23132 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:43.857916 23132 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:06:43.859376 23315 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:06:43.860510 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.861887 23203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:54634 (request call id 515) took 3459 ms. Trace:
I20260709 13:06:43.859315 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.888096 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.888693 23121 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:43.889289 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.889457 23121 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:43.889991 23121 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.887188 23203 rpcz_store.cc:276] 0709 13:06:40.402354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.402593 (+   239us) service_pool.cc:224] Handling call
0709 13:06:40.929010 (+526417us) ts_tablet_manager.cc:934] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
0709 13:06:41.909812 (+980802us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f3f4120ac 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42bcaa 00007f7f3f42bbd2 00007f7f47490131
0709 13:06:43.527198 (+1617386us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:06:43.834034 (+306836us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:43.834155 (+   121us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:06:43.861071 (+ 26916us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:06:43.861360 (+   289us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:43.861680 (+   320us) ts_tablet_manager.cc:989] Replica registered.
0709 13:06:43.861803 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":28848,"spinlock_wait_cycles":2444032,"tablet-copy.queue_time_us":251021,"thread_start_us":1450254,"threads_started":2}
I20260709 13:06:43.890417 23121 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.890471 23228 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:43.890744 23203 tablet_copy_client.cc:806] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:43.804641 21064 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:43.824636 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:43.863837 23217 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:43.869944 22904 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.851397 23218 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:43.873528 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78323.1us
I20260709 13:06:43.870712 23297 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.892555 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.898072 21765 leader_election.cc:400] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7.
I20260709 13:06:43.898450 21765 leader_election.cc:403] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:06:43.898775 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.895622 23144 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:43.891475 23203 tablet_copy_client.cc:670] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:43.900898 23132 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:43.894979 21213 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.885694 23321 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:43.908321 23234 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7.
I20260709 13:06:43.910422 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:43.914150 23302 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.902573 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.919000 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.926038 21065 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.926728 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.926927 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.929297 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.930675 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:43.917001 23254 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:43.935256 21216 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:43.939479 23236 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.931058 23072 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:43.929962 23295 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.924504 23132 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:43.944696 23132 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:43.944768 21139 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:43.945524 23302 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.947166 23297 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.948562 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:43.950397 23306 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.951813 23297 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.952323 23237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:54618 (request call id 12) took 4355 ms. Trace:
I20260709 13:06:43.952425 23297 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.955956 22814 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.953675 23254 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:43.956688 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.956610 23254 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.957930 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.958353 21213 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:43.958652 23254 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:43.930219 22930 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:43.925060 23298 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:06:43.954461 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:43.955569 23039 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:43.952812 23237 rpcz_store.cc:276] 0709 13:06:39.597139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.600368 (+  3229us) service_pool.cc:224] Handling call
0709 13:06:39.753255 (+152887us) log.cc:844] Serialized 513 byte log entry
0709 13:06:39.871750 (+118495us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b5400092ca0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4249cb26 00007f7f4249649a 00007f7f42495714 00007f7f42475eaa 00007f7f4247cd53 00007f7f4247ccd9 00007f7f4247cc69 00007f7f4247cc31 00007f7f4247bdfd 000000000038f354
0709 13:06:43.952256 (+4080506us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2008,"raft.queue_time_us":14248,"raft.run_cpu_time_us":4056,"raft.run_wall_time_us":82420,"spinlock_wait_cycles":6119296,"thread_start_us":53195,"threads_started":1,"wal-append.queue_time_us":58676,"wal-append.run_cpu_time_us":1685,"wal-append.run_wall_time_us":1001690}
I20260709 13:06:43.966300 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:43.935626 23321 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:06:43.960528 23297 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.959185 21218 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:43.968639 23321 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:43.961834 23203 tablet_copy_client.cc:538] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:43.964254 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:43.956853 23144 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.967774 21215 raft_consensus.cc:2368] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:43.971246 22904 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:43.977782 23265 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:43.971580 23144 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.978881 23230 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:43.979249 23230 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:43.972568 23298 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:06:43.980260 23144 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:43.981819 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:43.978336 23265 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:43.983956 23265 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:43.985100 23265 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:43.987383 23132 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.988615 21066 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:43.064837 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:43.996762 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:43.997381 21071 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:43.999538 23293 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.716s	user 0.003s	sys 0.001s
I20260709 13:06:43.974677 21065 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:43.955184 23093 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.002053 23230 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.003388 23093 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.004422 23225 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.004886 23312 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:44.005338 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:44.006423 22830 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:44.006793 22830 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.007637 22830 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:43.975613 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:43.974810 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:44.010385 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 233) took 1281 ms. Trace:
I20260709 13:06:44.019301 22830 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.023022 23107 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:44.024078 23107 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:44.024434 23107 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.025220 23107 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.026293 23107 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:44.020311 23318 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:44.021054 23216 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:44.021850 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2303) took 1111 ms. Trace:
I20260709 13:06:43.968990 23321 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:44.041822 21795 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:44.042959 23009 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.028105 21066 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:06:44.030246 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:44.044014 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.044570 21139 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:44.032277 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.045622 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:44.042052 23321 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.028589 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:44.032927 21144 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:44.046643 23321 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.046854 23218 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.031371 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:44.037333 21217 rpcz_store.cc:276] 0709 13:06:42.910380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.274318 (+363938us) service_pool.cc:224] Handling call
0709 13:06:44.021697 (+747379us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b5400442ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:44.021798 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46481792}
I20260709 13:06:44.019019 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1790) took 1305 ms. Trace:
I20260709 13:06:44.003089 23230 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:43.976409 23039 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.048734 21142 rpcz_store.cc:276] 0709 13:06:42.713600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.743942 (+ 30342us) service_pool.cc:224] Handling call
0709 13:06:42.750499 (+  6557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:42.750808 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.750831 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:42.828167 (+ 77336us) log.cc:844] Serialized 513 byte log entry
0709 13:06:43.268715 (+440548us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:43.268777 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.268890 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.704738 (+435848us) raft_consensus.cc:1685] finished
0709 13:06:43.704765 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.018753 (+313988us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b540030a550. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:44.018956 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13887360,"thread_start_us":253283,"threads_started":1,"wal-append.queue_time_us":378290}
I20260709 13:06:44.038852 23216 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:44.054513 23216 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.037106 23318 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:44.056092 23318 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.057785 23318 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:43.976238 23313 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:44.039759 23249 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:44.059669 21067 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:44.060295 23249 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:43.987898 23203 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:06:44.004433 23113 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:44.060850 23249 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:44.061071 23113 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:44.040036 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:44.061873 23113 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.043678 21070 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:44.063526 21070 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:44.064436 21070 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.065634 21070 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:44.010859 21063 rpcz_store.cc:276] 0709 13:06:42.728819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.149231 (+420412us) service_pool.cc:224] Handling call
0709 13:06:43.207384 (+ 58153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.316877 (+109493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:43.320205 (+  3328us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:43.328269 (+  8064us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:43.371477 (+ 43208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.010169 (+638692us) spinlock_profiling.cc:243] Waited 99.6 ms on lock 0x7b54002ad0d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:44.010218 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.010330 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229032192}
I20260709 13:06:44.063001 23113 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:44.062884 23216 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.069375 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 652) took 1216 ms. Trace:
I20260709 13:06:44.069823 21220 rpcz_store.cc:276] 0709 13:06:42.852843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.855584 (+  2741us) service_pool.cc:224] Handling call
0709 13:06:44.069122 (+1213538us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54004434d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:44.069299 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052188544}
W20260709 13:06:44.072837 23299 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.513s	user 0.003s	sys 0.000s
I20260709 13:06:44.073686 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.075304 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.076205 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.076788 21144 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:44.077071 21144 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.077780 21144 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.078641 21144 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:44.079519 21069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 603) took 1343 ms (client timeout 1554 ms). Trace:
I20260709 13:06:44.079988 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:44.079931 21069 rpcz_store.cc:269] 0709 13:06:42.736002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.755608 (+ 19606us) service_pool.cc:224] Handling call
0709 13:06:44.079377 (+1323769us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54000fb254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:44.079467 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230855552}
I20260709 13:06:44.061873 23249 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.084018 23303 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.084134 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:44.084915 21147 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:44.085355 23249 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:44.085889 23216 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:44.089296 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62876.1us
I20260709 13:06:44.023175 21071 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:06:44.092121 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:43.972872 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:44.093542 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:44.094905 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.095184 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:44.097685 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.098269 21217 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:44.099876 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.026618 23228 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:44.101074 21218 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:44.103206 21070 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
W20260709 13:06:44.103204 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 130391us
I20260709 13:06:44.105901 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:44.110865 21144 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:44.113548 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:44.114462 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:44.110253 21185 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.117648 21139 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:06:44.118336 21147 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:44.128221 21036 leader_election.cc:400] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:44.129072 21212 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:44.107940 23093 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:44.125561 23136 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:06:44.130306 21767 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:44.129464 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:44.090556 23241 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:06:44.115896 23296 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:06:44.131609 23265 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:44.138556 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.138861 23050 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:06:44.152035 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 142465us
I20260709 13:06:44.141793 23093 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:44.101619 23273 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:44.131894 23009 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:44.145503 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.106928 23114 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:44.180121 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:44.150933 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.132718 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:44.110775 23225 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:44.184839 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:44.185720 21070 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:44.144745 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51357us
I20260709 13:06:44.190501 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:44.191208 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:44.132827 23228 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:44.192122 23329 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:44.192677 23329 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:44.193535 23329 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.195704 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.123698 23276 raft_consensus.cc:2764] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:06:44.227573 21071 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:44.229324 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 47038.9us
I20260709 13:06:44.229380 21068 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: [3.5-4.6]
I20260709 13:06:44.149159 23265 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:44.171710 23237 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:44.171808 23093 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:44.181702 23308 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:44.194962 22814 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:44.223699 23314 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:44.178937 23009 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:44.171952 23106 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:06:44.151188 23241 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.238046 23039 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:44.232648 21797 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:44.250965 23132 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.241228 23093 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.232959 22836 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:44.260121 21068 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:44.282846 23324 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:44.283505 23121 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.260311 21070 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:44.234277 23313 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:44.264542 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 74121us
I20260709 13:06:44.267509 23265 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.241741 23306 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.242201 22814 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:44.257459 23009 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:44.257515 23237 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:44.296753 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66234.1us
I20260709 13:06:44.313628 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:44.265199 21064 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:44.317706 21072 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:44.267165 23300 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:44.327553 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:44.313697 22814 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.331151 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.332442 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:44.334046 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:44.334306 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:44.334714 21144 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:44.334918 21138 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:44.335196 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.337575 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.338901 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.339709 21143 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:44.341212 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.344220 23329 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:44.338733 23242 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:44.347790 23242 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:44.347617 23329 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.348232 23242 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:44.344252 23217 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:06:44.339264 23273 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:44.344385 23319 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:44.327306 23300 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:44.355576 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.359081 23241 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:44.344480 23225 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:44.360018 21064 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:44.355670 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:44.361899 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:44.363554 21801 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:44.364223 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.365727 21217 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:06:44.343111 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:44.367197 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:44.368623 21068 raft_consensus.cc:2376] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6. Re-sending same reply.
I20260709 13:06:44.367838 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:44.373838 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.372179 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:44.347370 23309 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.377674 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:44.378930 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:44.381498 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:44.389801 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 244361us
I20260709 13:06:44.392479 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:44.367439 23314 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:44.367085 23121 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.367596 23273 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.371392 23132 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:44.417390 23132 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:44.417815 23132 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:44.419078 21212 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:44.419566 23050 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:44.420113 23050 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:44.420986 23050 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.397823 23241 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:06:44.396768 23300 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.424574 23314 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:44.424984 23314 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:44.425177 23300 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.426030 23106 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:44.425894 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.398778 23225 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:44.426488 23106 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.397122 22791 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:44.427132 22791 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.427065 23225 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:44.427417 23106 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:44.399163 23307 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:44.351352 23311 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.161s
I20260709 13:06:44.371472 23313 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:44.397341 23217 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:06:44.386670 23039 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:44.431736 23039 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.431911 23217 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:44.432139 23039 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:44.433804 21796 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
W20260709 13:06:44.407879 21018 auto_rebalancer.cc:629] Failed to schedule move for tablet 72edcb8f7f3d437b8e7bb24c612600ab: Illegal state: Replica 5e5e65640c8b4f5ca7eadc6a14312c55 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:06:44.436298 23093 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.397668 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.438416 21219 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:44.387131 23205 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:06:44.386683 23303 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:44.443624 22904 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:44.432385 23217 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.446473 23273 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.409255 23312 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:44.447235 23217 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.447777 23312 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.448863 23217 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:44.450954 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:44.451766 21068 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:44.409644 23009 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.412061 23329 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.363327 23237 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:06:44.415030 23249 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:44.458407 23249 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.397168 23190 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:44.416805 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:44.425767 23314 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.428277 23225 consensus_queue.cc:1243] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:44.429226 23106 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.482537 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1586) took 1772 ms. Trace:
I20260709 13:06:44.371110 23319 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.482950 21072 rpcz_store.cc:276] 0709 13:06:42.709748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.077635 (+1367887us) service_pool.cc:224] Handling call
0709 13:06:44.317544 (+239909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.318192 (+   648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.482322 (+164130us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54003cad50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:44.482479 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551086976}
I20260709 13:06:44.435436 21139 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:06:44.486294 21139 pending_rounds.cc:85] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 5
I20260709 13:06:44.486585 21139 pending_rounds.cc:107] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:06:44.388518 23309 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.387087 23339 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:06:44.440490 21212 raft_consensus.cc:2376] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6. Re-sending same reply.
I20260709 13:06:44.445233 21071 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:44.459977 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.459493 23190 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.463333 21141 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:44.480480 23314 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:44.482184 21142 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:06:44.431030 23313 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.486634 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:44.487509 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.486842 21139 raft_consensus.cc:2889] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:44.488161 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 95776.2us
I20260709 13:06:44.511389 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.494275 23346 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:06:44.454972 23303 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.498338 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:44.481532 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:44.508944 21097 heartbeater.cc:646] Failed to heartbeat to 127.19.127.254:37743 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.127.254:37743 timed out after 15.000s (SENT)
I20260709 13:06:44.521248 23225 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:06:44.460537 22904 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:06:44.525278 23349 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:44.525668 23307 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:44.526032 23349 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:44.526489 23307 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:44.527669 22964 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:44.527890 21071 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:44.528240 22964 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:44.523697 23348 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:44.529142 22964 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.520009 23211 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:06:44.428620 22791 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.524940 23319 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.529315 23348 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.538638 21071 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:44.536206 20967 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 256) took 12594 ms (client timeout 15000 ms). Trace:
I20260709 13:06:44.463819 23242 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.572566 23242 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:44.572108 22814 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.573909 21146 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:44.554675 23211 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:44.574689 23211 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.497031 23312 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.576035 23211 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.568487 22904 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:44.577999 22904 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:44.579209 22904 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.570020 23205 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:44.581453 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:44.580796 23093 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.491405 23046 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:06:44.513798 23313 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:44.634071 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75016.6us
I20260709 13:06:44.644568 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:44.560053 21797 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:44.649335 23249 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.635125 23211 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.645934 23190 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.650550 23348 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.654171 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:44.560708 20967 rpcz_store.cc:269] 0709 13:06:31.925471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.007369 (+ 81898us) service_pool.cc:224] Handling call
0709 13:06:44.519500 (+12512131us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:41.616473 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:41.617114 (+   641us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:41.617156 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:06:41.617177 (+    21us) tablet.cc:662] Decoding operations
0709 13:06:41.617911 (+   734us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:41.617978 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:41.618009 (+    31us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:06:41.618550 (+   541us) tablet.cc:701] Row locks acquired
0709 13:06:41.618569 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:06:41.618713 (+   144us) write_op.cc:270] Start()
0709 13:06:41.618826 (+   113us) write_op.cc:276] Timestamp: P: 1783602401618667 usec, L: 0
0709 13:06:41.618846 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:06:41.637668 (+ 18822us) log.cc:844] Serialized 2109 byte log entry
0709 13:06:42.416762 (+779094us) op_driver.cc:464] REPLICATION: finished
0709 13:06:42.543600 (+126838us) write_op.cc:301] APPLY: starting
0709 13:06:42.543688 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:42.543836 (+   148us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:42.545895 (+  2059us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:42.576949 (+ 31054us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:42.577175 (+   226us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:06:42.577232 (+    57us) write_op.cc:312] APPLY: finished
0709 13:06:42.577522 (+   290us) log.cc:844] Serialized 47 byte log entry
0709 13:06:42.577697 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:42.577813 (+   116us) write_op.cc:454] Released schema lock
0709 13:06:42.577861 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:42.608164 (+ 30303us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:42.608211 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":115012,"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":125244,"apply.run_cpu_time_us":3141,"apply.run_wall_time_us":64517,"num_ops":5,"prepare.queue_time_us":215710,"prepare.run_cpu_time_us":15111,"prepare.run_wall_time_us":188519,"raft.queue_time_us":512606,"raft.run_cpu_time_us":26897,"raft.run_wall_time_us":141403,"replication_time_us":771549,"thread_start_us":639388,"threads_started":4,"wal-append.queue_time_us":208348,"wal-append.run_cpu_time_us":18983,"wal-append.run_wall_time_us":1720362}]]}
I20260709 13:06:44.647158 21799 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:44.662926 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 151190us
I20260709 13:06:44.672351 23309 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.668745 23190 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.667425 23303 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.674592 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:44.672466 23144 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:44.675429 21070 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:44.679541 21767 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:44.675588 23144 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.689237 23144 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:44.695539 23309 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:44.696071 23309 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.696450 23309 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:44.664547 22904 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:44.718070 21070 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:06:44.699465 22814 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:44.664759 23305 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:06:44.759121 23273 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:06:44.784770 23237 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:44.785288 23237 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:44.786092 23237 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.787904 23305 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:44.788424 23305 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:44.773527 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 262104us
I20260709 13:06:44.665917 23218 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:06:44.710816 23230 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.793680 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.812158 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.813306 21212 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:44.704552 23132 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.815253 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:44.817544 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 335388us
I20260709 13:06:44.818320 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.818783 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.820217 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.820566 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.822196 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.828554 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:44.828933 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.830152 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:44.830957 21072 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:44.719725 23039 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.831974 21067 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:44.832520 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:44.833300 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:44.832629 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:44.835567 21216 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:06:44.835927 21069 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:44.836526 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.837280 21065 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:44.837327 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:44.838848 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:44.839808 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:44.812868 23349 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:44.842949 21070 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:44.835013 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:44.825220 23237 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:44.834103 23345 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:06:44.846913 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:44.847856 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:44.853606 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.880239 21218 raft_consensus.cc:2368] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4.
I20260709 13:06:44.831344 23205 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:44.881385 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46447.1us
I20260709 13:06:44.883092 21220 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:44.885661 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:44.815642 23230 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.827399 23242 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:06:44.844817 23093 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:44.888952 21219 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:44.887854 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.847776 23228 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:44.890571 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:44.891093 23239 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 13:06:44.893945 23350 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 13:06:44.900106 23353 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:06:44.900846 23218 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:44.909682 23132 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:44.910552 23009 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:44.911588 23308 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:44.965018 23308 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.965870 23308 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.886898 23356 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:44.914798 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:44.966789 23356 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:44.918830 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:44.967195 23356 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.968885 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:44.965890 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:44.970305 21797 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:44.970422 23239 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:44.971431 23300 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 13:06:44.971652 21035 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:44.932344 23039 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:44.935998 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:44.936988 21214 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:44.937402 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46909.8us
I20260709 13:06:44.972872 21214 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:44.975179 21801 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:44.975907 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:44.957809 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.886689 23296 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:44.841120 23305 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:44.920758 23230 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:44.973954 21214 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.983471 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:44.985836 23309 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.945272 23205 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.959007 23035 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:45.000676 23035 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:44.997535 23265 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:45.001847 23035 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:44.959887 23093 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:45.005507 23273 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:44.959218 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:45.006132 23273 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:45.007007 23305 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:44.991094 23132 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:45.013409 23132 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.013968 23132 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:45.014819 21035 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:44.892541 23346 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 13:06:45.016304 23313 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.017146 23346 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:45.017541 23346 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:45.017787 23205 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.018466 23346 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.020851 23228 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:45.017158 23309 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.024039 21212 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:45.025215 23241 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:45.007009 23273 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.027798 21221 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:45.089565 21068 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:45.031417 21070 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:45.087679 21146 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:45.035858 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:45.039367 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:45.091938 21142 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:06:45.091740 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55988.8us
I20260709 13:06:45.040607 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:45.040988 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:45.040671 21143 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:44.891769 23303 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.102s
W20260709 13:06:45.054459 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.041932 21138 raft_consensus.cc:2955] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:06:45.099947 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:45.122396 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:45.123235 21146 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:45.055827 22830 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:45.051049 23338 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.088943 23346 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:45.107191 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:45.130700 22830 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.113739 22904 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:45.118002 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:45.137285 23313 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.140944 23218 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:45.057231 23241 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:45.144155 23348 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:45.147982 21070 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:45.149286 21064 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:45.167465 23107 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:45.128971 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:45.055569 23249 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.119s
I20260709 13:06:45.162995 23356 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.203326 21217 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:45.204493 23050 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.205613 23050 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.216804 23305 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.217624 23305 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.220471 21146 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:45.171069 23309 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:06:45.224087 23303 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:45.189577 23218 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.190163 22904 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:45.229434 22904 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.178711 22836 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:45.234419 21219 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:45.234833 21219 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:45.239033 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.239755 21065 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:45.240334 21067 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:45.239806 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:45.182155 21144 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:45.055219 23265 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.124s
W20260709 13:06:45.190538 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 304944us
I20260709 13:06:45.054314 23228 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:45.187565 21063 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:45.244699 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:45.257478 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:45.258096 21219 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.259675 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:45.264252 21219 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:45.262158 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:45.269333 21072 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:45.270263 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:45.273587 21066 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:45.274554 21220 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:45.170598 23132 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.266820 21796 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:45.260210 23228 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:45.263788 23308 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:45.343190 21146 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:45.231751 23241 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:45.269527 23313 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.272256 23312 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:45.343604 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:45.326683 23349 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:45.364127 21219 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:06:45.367193 21219 pending_rounds.cc:85] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 2
I20260709 13:06:45.368984 21219 pending_rounds.cc:107] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:06:45.275803 23230 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:45.273952 23296 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:45.196734 23107 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.267171 23295 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:45.327319 23009 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.327597 23211 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.338754 23304 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.266605 23339 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 13:06:45.352555 21146 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:45.267331 22836 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.181051 23338 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.427150 21219 raft_consensus.cc:2889] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:45.427454 21147 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:45.343585 23121 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:45.353232 23144 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:45.360307 23349 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.359936 23363 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:06:45.418996 23203 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:45.450358 21146 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: [4.5-5.6]
I20260709 13:06:45.355518 23249 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:06:45.418968 22814 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:45.459162 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 181805us
I20260709 13:06:45.358879 23313 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.433045 23242 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:06:45.446453 23265 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:45.453135 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 534) took 1106 ms. Trace:
I20260709 13:06:45.566283 21138 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:45.576411 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.465942 23349 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:45.454082 23225 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.113s
I20260709 13:06:45.448715 23009 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.453410 23211 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.457077 23304 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:45.608235 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.568367 23093 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:45.578763 23230 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.461346 23121 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.464407 23144 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:45.641382 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.631400 23093 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.566792 22814 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:45.620402 23237 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.763s	user 0.011s	sys 0.002s
I20260709 13:06:45.458118 23295 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.578473 23265 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.617377 23132 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.474669 23363 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.642777 23375 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:45.622498 23321 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:45.644757 23376 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:45.651398 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:45.645138 23230 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.661710 22836 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.752219 23265 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.665621 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:45.753482 21792 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:45.753871 23304 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.668278 23321 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.756927 23321 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.662473 23375 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:45.760268 23375 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.761869 23375 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:45.680896 23093 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:45.729522 23366 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 13:06:45.681334 23338 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.680858 23376 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.763712 23093 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:45.743939 23368 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:06:45.764310 23093 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.766911 23363 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.767460 23363 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.767939 23363 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:45.768172 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 120520us
I20260709 13:06:45.769145 23203 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:06:45.770012 23363 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.771049 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:45.772913 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.773020 21063 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:06:45.773528 23295 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.775852 23295 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.776408 23295 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.778378 23241 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:45.778661 23295 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.787863 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:45.789268 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:45.789676 23039 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:45.790448 23039 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:45.790555 21185 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:45.790983 23039 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:45.791486 22904 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:45.791965 23039 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.794667 23339 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:06:45.796836 23357 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.788s	user 0.004s	sys 0.009s
I20260709 13:06:45.798209 23349 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.799340 23374 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:45.800549 23242 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:45.800968 23242 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:45.792006 22904 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:45.801761 23242 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.802810 23295 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:45.802996 22904 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.744863 23009 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.659847 23296 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.746412 23228 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.784179 23144 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.747035 22814 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:45.747618 23367 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:06:45.748188 23377 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.749372 23211 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:45.749504 23346 consensus_meta.cc:287] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.568s	user 0.004s	sys 0.005s
W20260709 13:06:45.749941 23372 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.074s	user 0.001s	sys 0.003s
I20260709 13:06:45.676797 23382 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.751765 23121 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.744661 23313 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:45.824860 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.807202 23093 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.815587 23382 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.803642 23295 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.812664 23228 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.805528 23376 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.814337 23377 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.832564 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.833747 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.834328 21064 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:45.831718 23295 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.803604 23242 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:45.811072 23296 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.820775 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.821648 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.837476 21070 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:45.837841 23356 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.806973 23381 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.838819 23311 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.839371 23311 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:45.840902 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:45.831946 22814 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.818691 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.834385 23377 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.842229 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:45.843729 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:45.822283 21067 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:45.829206 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.857089 21767 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:45.829948 23382 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:45.829366 21797 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:45.859650 21142 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:06:45.813448 23371 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.206s	user 0.002s	sys 0.008s
I20260709 13:06:45.839017 23381 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.873940 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:45.831025 21792 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:45.876178 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.817293 23313 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.828852 23305 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.884663 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:45.827831 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:45.879796 21142 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:45.805397 23230 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.882830 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.883019 23296 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.800495 23349 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.901899 21072 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:45.874022 23377 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:45.880321 23203 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:06:45.569039 21794 rpcz_store.cc:276] 0709 13:06:44.346692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.361089 (+ 14397us) service_pool.cc:224] Handling call
0709 13:06:44.437305 (+ 76216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.481789 (+ 44484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.484752 (+  2963us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:44.491646 (+  6894us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:44.497440 (+  5794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.450849 (+953409us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b54002c3654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:45.452919 (+  2070us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:45.453073 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37171072}
I20260709 13:06:45.914134 23203 ts_tablet_manager.cc:1403] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 1.926s	user 0.030s	sys 0.031s
I20260709 13:06:45.881852 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:45.883745 23241 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.922978 23241 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.925779 23241 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.818789 23121 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.927650 23121 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.853017 23376 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.930330 23121 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:45.906340 21799 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:45.906911 21035 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:45.953454 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:45.957322 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:45.923552 23349 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:45.947800 23203 raft_consensus.cc:359] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.931092 23376 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.854036 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.932878 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:45.904609 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:45.899780 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:45.974187 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:45.975205 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:45.975805 21142 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:45.976809 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:45.977439 21142 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:45.934540 23381 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:45.935930 23211 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:45.937680 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:45.859627 22814 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:45.938228 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:45.892313 23313 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.941175 23311 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:45.943038 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:46.018447 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 80280.8us
I20260709 13:06:46.019783 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:45.944370 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:45.945024 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 536) took 1573 ms. Trace:
I20260709 13:06:45.945304 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:45.946118 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 246) took 1363 ms. Trace:
I20260709 13:06:45.943025 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:46.041854 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:45.958873 21069 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:45.971841 23295 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:46.041527 21068 rpcz_store.cc:276] 0709 13:06:44.582801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.582976 (+   175us) service_pool.cc:224] Handling call
0709 13:06:44.583381 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:45.090087 (+506706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.945867 (+855780us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b540026e154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:45.946055 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163327360}
I20260709 13:06:46.047011 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:46.048151 21798 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
W20260709 13:06:46.046891 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 103971us
I20260709 13:06:45.921976 23234 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.961381 23121 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:45.961388 23039 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.091599 23385 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:46.091480 23039 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.955109 21767 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:46.013430 23303 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.967646 23356 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.967743 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:45.968191 23203 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:06:45.968248 23198 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:45.970213 23376 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:45.972328 21070 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:06:45.933681 23377 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:45.962190 23349 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:45.979481 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.087682 21796 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
W20260709 13:06:45.981650 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.023572 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:46.186203 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 162718us
I20260709 13:06:46.022934 21801 rpcz_store.cc:276] 0709 13:06:44.371810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.371948 (+   138us) service_pool.cc:224] Handling call
0709 13:06:45.944839 (+1572891us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b54006612d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:45.944958 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1375813376}
I20260709 13:06:45.959774 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:46.187605 21142 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:45.961467 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:46.049006 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:46.092242 23385 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.189452 21798 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:45.978634 22964 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:45.972820 23296 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:46.046751 23381 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:46.050540 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:45.995823 23339 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.997916 23230 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.047379 23009 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:46.047051 23035 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:46.096766 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 141789us
I20260709 13:06:45.966401 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:46.047799 23348 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:46.047189 22791 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:46.151166 22904 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.089675 23295 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:46.088027 23121 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:46.295481 23121 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:46.190704 21035 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:46.299871 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 109241us
I20260709 13:06:46.107662 23356 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.301003 23385 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.301143 23296 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:46.301605 23296 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.301982 23296 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:46.302807 23356 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.187708 21067 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:06:46.186915 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:46.304525 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 117703us
I20260709 13:06:46.304688 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:46.301630 23385 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.306169 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:46.306411 23385 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:06:46.207693 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 157231us
I20260709 13:06:46.095700 23321 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:06:46.305887 23228 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:46.308337 23228 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:46.308682 23228 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:46.216266 21800 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:46.180228 21795 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:46.261013 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.309762 21142 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:06:46.311101 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.311686 21142 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:46.312477 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.312724 23385 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.313120 21142 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:46.313273 23385 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:46.313879 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.314465 21142 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:46.193585 23376 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:46.224566 23230 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.315253 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.316684 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.317281 23230 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:46.317278 23296 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.319887 23296 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:46.096940 23234 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.325789 23234 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.108561 23303 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.226758 23203 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:46.328966 23303 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.198815 23249 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:46.294479 22904 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.337934 21035 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:46.179426 23198 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.339906 23296 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:46.229182 22791 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:46.347124 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.380057 23241 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.229206 23035 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.294960 23295 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.226502 23339 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.381541 23228 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.384927 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:46.381808 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1823) took 1339 ms. Trace:
I20260709 13:06:46.229676 23009 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:46.395812 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 179588us
I20260709 13:06:46.359323 23356 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:46.350554 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:46.383107 23242 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.547s	user 0.005s	sys 0.005s
I20260709 13:06:46.367265 23230 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:46.380178 23203 ts_tablet_manager.cc:1434] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.466s	user 0.015s	sys 0.005s
I20260709 13:06:46.379576 23273 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:46.224718 23348 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.374922 23373 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:46.381909 23198 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.404775 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:46.390800 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:46.397277 23039 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.211397 22964 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.398614 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:46.414520 21147 rpcz_store.cc:276] 0709 13:06:45.042861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.042991 (+   130us) service_pool.cc:224] Handling call
0709 13:06:45.043263 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:45.457234 (+413971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.381579 (+924345us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54002599d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:46.381749 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":883010304}
I20260709 13:06:46.415635 23313 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.416918 23339 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.403863 23296 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.418975 23035 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.419363 23228 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.401921 22791 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:46.439857 23009 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:46.420524 23396 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
W20260709 13:06:46.421250 23395 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:06:46.403723 23211 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:46.430048 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 79582.6us
I20260709 13:06:46.370790 23377 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.432659 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:46.483731 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51181us
I20260709 13:06:46.484707 21217 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:46.426270 22814 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:46.483737 23039 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.487617 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.487908 23039 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.411377 23241 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.489750 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.490059 23039 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.490198 23241 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.490547 21220 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:06:46.461149 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 61845.6us
I20260709 13:06:46.461668 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.427862 23329 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:06:46.491766 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:46.436772 23203 ts_tablet_manager.cc:1467] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Tablet startup took 522ms
I20260709 13:06:46.450903 23348 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:46.450037 23230 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:46.460294 23405 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:46.481863 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:46.485839 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.486604 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.499791 21219 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:46.488369 21212 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:46.500736 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.488224 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.428489 23356 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.435365 23132 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:46.516610 23132 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.517031 23132 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:46.518482 23356 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:46.519104 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1826) took 1474 ms. Trace:
I20260709 13:06:46.519419 21797 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:06:46.519541 21144 rpcz_store.cc:276] 0709 13:06:45.044280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.250898 (+206618us) service_pool.cc:224] Handling call
0709 13:06:45.251204 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:45.453491 (+202287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:45.453527 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:45.453569 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:45.453606 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.518834 (+1065228us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b540020add4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:46.518921 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:46.519048 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464647424}
I20260709 13:06:46.466707 23121 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.521034 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.521780 21144 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:46.522583 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.522876 23121 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:46.491603 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.491716 21184 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:46.467336 21795 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:46.492506 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.530048 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.530577 21220 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:46.460798 23198 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:46.466830 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.469213 23398 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:46.494927 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.535243 23198 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.535523 21147 raft_consensus.cc:2376] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6. Re-sending same reply.
I20260709 13:06:46.535866 21217 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:46.536545 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.501006 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:46.507781 23241 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:46.537384 23411 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.537699 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.452620 23399 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:06:46.537823 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:46.537899 23411 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.538602 21796 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:46.508989 23228 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:46.507567 23313 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:46.510768 23405 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.511744 22964 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.477897 23349 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.497189 22904 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:46.565107 23402 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.565796 23402 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:46.568832 23360 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.199s	user 0.004s	sys 0.022s
I20260709 13:06:46.570573 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 645) took 1779 ms. Trace:
I20260709 13:06:46.516074 23303 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:46.523176 21144 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:46.523634 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:46.627254 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 72552us
I20260709 13:06:46.628259 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:46.591542 21216 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:46.531244 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.608489 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1832) took 1431 ms. Trace:
I20260709 13:06:46.637210 21146 rpcz_store.cc:276] 0709 13:06:45.176589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.466768 (+290179us) service_pool.cc:224] Handling call
0709 13:06:45.466995 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:45.662173 (+195178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:45.662205 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:45.662246 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:45.662277 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.608258 (+945981us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54002954d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:46.608303 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:46.608427 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448424832}
I20260709 13:06:46.457984 23373 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:46.491616 23339 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.497522 23400 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:46.532496 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.593257 23405 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.705591 23400 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.705963 21145 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:46.532732 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:46.533142 21064 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:46.533442 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:46.695226 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.707819 21144 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 1.
I20260709 13:06:46.534633 23407 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:46.535547 23398 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:46.536625 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:46.711910 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 57635us
I20260709 13:06:46.712265 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:46.712989 21217 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:46.509891 23230 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.510219 23296 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.537424 23241 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.563326 23242 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:46.563999 23009 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.715596 23121 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:46.494971 23211 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.713995 23398 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:46.709774 21213 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:46.724817 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.725139 23211 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:46.637247 21220 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:46.725819 21147 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:46.726347 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.727073 21220 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
I20260709 13:06:46.706004 23400 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:46.615358 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:46.727928 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 112676us
I20260709 13:06:46.727737 23348 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.610967 23309 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:46.728897 21142 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:46.728974 23309 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:06:46.700948 23359 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.028s	user 0.000s	sys 0.008s
I20260709 13:06:46.729928 23309 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.701642 23198 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.702646 23339 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.736567 23339 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.737201 21213 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:46.737358 21797 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:46.591066 23303 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.738868 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:46.739775 23303 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:46.740603 23228 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:46.741174 23228 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:46.704639 21796 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:46.741773 21796 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:46.574146 23385 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:46.572425 23313 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:46.706498 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 78331.4us
I20260709 13:06:46.714861 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:46.716176 21067 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:06:46.570981 21071 rpcz_store.cc:276] 0709 13:06:44.791575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.791776 (+   201us) service_pool.cc:224] Handling call
0709 13:06:44.792205 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:44.808944 (+ 16739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.808992 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:44.826636 (+ 17644us) log.cc:844] Serialized 26 byte log entry
0709 13:06:45.032259 (+205623us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:45.032317 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:45.033544 (+  1227us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:46.568227 (+1534683us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:46.570252 (+  2025us) raft_consensus.cc:1685] finished
0709 13:06:46.570274 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:46.570506 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16232,"raft.queue_time_us":17093,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":12659,"spinlock_wait_cycles":21098880,"thread_start_us":152523,"threads_started":1,"wal-append.queue_time_us":427969}
I20260709 13:06:46.723143 23121 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.728564 23241 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.639074 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:46.732102 21215 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:46.708580 23381 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:46.733063 21139 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:06:46.709786 23407 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:46.734287 23347 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:46.742717 21141 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:46.825208 21185 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 2.137s (SENT) [suppressed 5 similar messages]
I20260709 13:06:46.732805 23411 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.714167 23376 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:46.856195 23388 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 13:06:46.496868 23203 ts_tablet_manager.cc:1469] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Trace:
0709 13:06:40.402354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.402593 (+   239us) service_pool.cc:224] Handling call
0709 13:06:40.929010 (+526417us) ts_tablet_manager.cc:934] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
0709 13:06:41.909812 (+980802us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f3f4120ac 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42bcaa 00007f7f3f42bbd2 00007f7f47490131
0709 13:06:43.527198 (+1617386us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:06:43.834034 (+306836us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:43.834155 (+   121us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:06:43.861071 (+ 26916us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:06:43.861360 (+   289us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:43.861680 (+   320us) ts_tablet_manager.cc:989] Replica registered.
0709 13:06:43.861803 (+   123us) inbound_call.cc:177] Queueing success response
0709 13:06:43.987541 (+125738us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:06:45.914599 (+1927058us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:06:45.917095 (+  2496us) tablet_replica.cc:261] Starting instrumentation
0709 13:06:45.917826 (+   731us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":28848,"mutex_wait_us":27524,"raft.queue_time_us":19109,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":25246,"spinlock_wait_cycles":2444032,"tablet-copy.queue_time_us":251021,"tablet-download-pool.queue_time_us":20928,"tablet-download-pool.run_cpu_time_us":1383,"tablet-download-pool.run_wall_time_us":20514,"thread_start_us":1450254,"threads_started":2}
I20260709 13:06:46.866941 23107 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.869560 23356 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.851660 23121 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:46.731475 23242 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.871925 23298 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.742039 23228 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.853907 23402 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:46.744079 21795 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:06:46.837950 23113 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:46.910375 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 84984us
I20260709 13:06:46.871467 23352 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:46.743748 23009 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.859746 23407 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:46.854144 23385 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:46.913682 21218 raft_consensus.cc:2368] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7.
I20260709 13:06:46.894815 23347 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.907697 22830 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:46.924281 23347 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:46.913353 23343 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.897161 23376 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.919881 23352 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:46.900440 21796 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:46.908427 23298 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:46.915257 23113 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:46.926380 23385 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:47.076846 23113 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:47.050873 21796 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:47.027510 23416 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.051730 23348 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:46.976346 23343 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:47.226989 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:47.355190 23414 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:06:47.147917 23416 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:47.322472 22830 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:47.343451 23348 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:47.414006 23400 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.686s	user 0.005s	sys 0.005s
I20260709 13:06:47.417905 22830 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:47.465778 23399 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:47.466382 23399 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:47.467417 22830 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:47.413276 23242 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:47.454227 23009 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:47.478180 23083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.002s	sys 0.007s
I20260709 13:06:47.437284 23348 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:47.609972 23241 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:06:47.638355 23241 consensus_queue.cc:1243] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:06:47.649266 23121 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.000s
I20260709 13:06:47.655391 23348 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:47.613714 22791 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.121s	user 0.008s	sys 0.004s
I20260709 13:06:47.755273 22791 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:47.757054 21138 raft_consensus.cc:2368] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:47.657387 23298 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:47.761356 22791 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:47.773511 23121 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:47.673069 21796 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.000s
I20260709 13:06:47.607568 23376 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:47.683517 23343 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:47.684224 23399 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:47.470288 23400 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:47.781116 23400 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:47.687440 23416 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:47.787395 21796 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:06:47.788038 21796 pending_rounds.cc:85] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 5
I20260709 13:06:47.788422 21796 pending_rounds.cc:107] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:06:47.788802 21796 raft_consensus.cc:2889] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:47.789858 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 557) took 1165 ms. Trace:
W20260709 13:06:47.707716 23083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.003s	sys 0.007s
I20260709 13:06:47.717489 23356 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:47.718235 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:47.724972 23132 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.207s	user 0.007s	sys 0.005s
I20260709 13:06:47.888643 23132 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:47.732404 23347 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.785s	user 0.011s	sys 0.000s
I20260709 13:06:47.893617 23347 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:47.897771 23347 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:47.845230 23121 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:47.903163 21796 rpcz_store.cc:276] 0709 13:06:46.624819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.695841 (+ 71022us) service_pool.cc:224] Handling call
0709 13:06:46.704511 (+  8670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:47.789489 (+1084978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.789776 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:47.878548 23407 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.944s	user 0.011s	sys 0.001s
I20260709 13:06:47.916877 23407 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:47.918349 21212 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:47.925464 21212 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:06:47.891705 23132 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:47.834208 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 336704us
W20260709 13:06:47.693630 23398 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.970s	user 0.006s	sys 0.004s
I20260709 13:06:47.927057 23398 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:47.928519 21219 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:06:47.934461 23398 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:47.976181 23407 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:48.009138 21036 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.005s	sys 0.005s
W20260709 13:06:48.010180 21036 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.005s	sys 0.005s
W20260709 13:06:48.010826 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 571115us
W20260709 13:06:48.012406 21035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.103s	user 0.012s	sys 0.013s
W20260709 13:06:48.012957 21035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.104s	user 0.012s	sys 0.013s
I20260709 13:06:48.018260 21035 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:48.031159 23298 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.035624 23407 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.037377 23407 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.039623 23376 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:48.040988 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 654) took 2222 ms. Trace:
I20260709 13:06:48.040710 23376 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.041465 21063 rpcz_store.cc:276] 0709 13:06:45.817986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.871701 (+ 53715us) service_pool.cc:224] Handling call
0709 13:06:48.040404 (+2168703us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400215054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.040734 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062040960}
I20260709 13:06:48.043011 23407 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:48.033412 23298 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.048817 23298 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.053192 23376 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:48.054688 22791 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.055689 22791 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.061247 23416 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:48.062484 23416 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.065948 23416 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.067108 23348 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.113584 23348 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.068492 23009 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.115869 23009 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.119935 23425 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:48.120879 23425 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.073809 23132 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.074573 23418 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:48.076212 22791 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:48.090598 21216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2325) took 1604 ms (client timeout 1966 ms). Trace:
I20260709 13:06:48.047114 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 665) took 2220 ms. Trace:
I20260709 13:06:48.125365 23348 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.131319 23416 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.136158 23416 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.136503 23418 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:48.128455 21216 rpcz_store.cc:269] 0709 13:06:46.485874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.488005 (+  2131us) service_pool.cc:224] Handling call
0709 13:06:48.089925 (+1601920us) spinlock_profiling.cc:243] Waited 89.9 ms on lock 0x7b5400040e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.090259 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206812032}
I20260709 13:06:48.127667 21065 rpcz_store.cc:276] 0709 13:06:45.826858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.837835 (+ 10977us) service_pool.cc:224] Handling call
0709 13:06:45.838137 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:45.840534 (+  2397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:45.840562 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:45.853021 (+ 12459us) log.cc:844] Serialized 26 byte log entry
0709 13:06:46.047818 (+194797us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:46.186686 (+138868us) log.cc:844] Serialized 14 byte log entry
0709 13:06:46.223606 (+ 36920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.045869 (+1822263us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b18002a4d20. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3fd2a6 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419
0709 13:06:48.046507 (+   638us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b540041c654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:48.046560 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.046660 (+   100us) raft_consensus.cc:1685] finished
0709 13:06:48.046670 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.046896 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36608,"prepare.queue_time_us":323088,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":1705,"raft.queue_time_us":5833,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":18723,"spinlock_wait_cycles":7877888,"thread_start_us":258083,"threads_started":2,"wal-append.queue_time_us":306070,"wal-append.run_cpu_time_us":3230,"wal-append.run_wall_time_us":1349886}
I20260709 13:06:48.139081 23416 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.111367 23376 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:48.142370 21212 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2337) took 1644 ms (client timeout 1886 ms). Trace:
I20260709 13:06:48.147346 23050 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:48.151849 23376 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.110989 23298 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.159037 23121 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:48.125938 23009 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:48.142431 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 2.05313e+06us
W20260709 13:06:48.154671 21212 rpcz_store.cc:269] 0709 13:06:46.497856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.500777 (+  2921us) service_pool.cc:224] Handling call
0709 13:06:48.141821 (+1641044us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54000bc7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.142096 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3176289664}
W20260709 13:06:48.164484 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 63601.1us
I20260709 13:06:48.165904 23009 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.166570 23009 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.167640 23121 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.168874 23009 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.170825 23121 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.139204 23418 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:48.173874 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 155999us
I20260709 13:06:48.176504 23427 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:48.177261 23296 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:48.177351 23427 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.178037 23296 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:48.180269 23425 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:48.180975 23425 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:48.183030 23249 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.183524 23132 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.183378 23298 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.186692 23132 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.187222 23400 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.196026 23400 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.196738 23352 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.198354 23352 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:48.187391 23380 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:48.200079 23307 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:48.199820 23380 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.200592 23307 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:48.202533 23380 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:48.202131 23307 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.188114 23218 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.188789 23399 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:48.199039 23400 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.193677 23321 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:48.204794 23321 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:48.206097 23321 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.203892 23399 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.209054 23399 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:48.162123 23376 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.210098 23321 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:48.179745 23296 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.189647 23427 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.181941 23425 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:48.223829 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.183667 23249 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:48.245402 23427 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.245477 23430 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:48.246441 23427 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.246723 23430 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:48.249218 21793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 549) took 2296 ms (client timeout 1981 ms). Trace:
I20260709 13:06:48.249876 23427 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.250284 23430 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.251066 23427 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.247301 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2476) took 2405 ms. Trace:
I20260709 13:06:48.251572 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:48.252617 21799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 653) took 2409 ms (client timeout 2706 ms). Trace:
I20260709 13:06:48.251992 21144 rpcz_store.cc:276] 0709 13:06:45.842106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.708403 (+866297us) service_pool.cc:224] Handling call
0709 13:06:46.708694 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:46.741152 (+ 32458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:46.741182 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:46.741220 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:46.741250 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.246584 (+1505334us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b540027a454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:48.246666 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.246975 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74194816}
W20260709 13:06:48.253285 21799 rpcz_store.cc:269] 0709 13:06:45.843368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.843500 (+   132us) service_pool.cc:224] Handling call
0709 13:06:48.252135 (+2408635us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b5400701a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.252395 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120695680}
W20260709 13:06:48.249953 21793 rpcz_store.cc:269] 0709 13:06:45.952557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.962839 (+ 10282us) service_pool.cc:224] Handling call
0709 13:06:48.248590 (+2285751us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54005429d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.248901 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1685973888}
I20260709 13:06:48.254947 23218 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:48.255493 23218 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.255906 23218 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:48.255990 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.260010 23298 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.259753 23401 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:48.259050 23239 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:48.262969 23239 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.259954 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.265976 23239 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:48.266544 21138 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2487) took 2337 ms (client timeout 1884 ms). Trace:
W20260709 13:06:48.267079 21138 rpcz_store.cc:269] 0709 13:06:45.928481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.304439 (+375958us) service_pool.cc:224] Handling call
0709 13:06:48.266045 (+1961606us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540032cdd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.266307 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3338098432}
W20260709 13:06:48.267202 21765 connection.cc:256] RPC call timeout handler was delayed by -2.2356368829999838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
I20260709 13:06:48.267472 23431 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:48.271214 23431 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.271831 21793 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:48.273736 23431 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.273805 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.274652 23431 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.275209 23431 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.279474 23431 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:48.196065 23093 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:48.287744 23307 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:48.299108 21141 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:48.300501 23093 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.302897 23093 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.303786 23228 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:48.304620 23228 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.305158 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:48.306046 21221 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:48.307366 23228 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:48.307792 21072 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:48.310472 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.311419 21066 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:48.313233 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.320896 21138 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:48.325500 23009 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.325691 23400 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:48.326218 23009 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.326821 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1821) took 3285 ms. Trace:
I20260709 13:06:48.326534 23400 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.327201 21140 rpcz_store.cc:276] 0709 13:06:45.040844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.040949 (+   105us) service_pool.cc:224] Handling call
0709 13:06:45.098837 (+ 57888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:45.478514 (+379677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:45.478545 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:45.565337 (+ 86792us) log.cc:844] Serialized 26 byte log entry
0709 13:06:46.378108 (+812771us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b1800087a60. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:06:46.465794 (+ 87686us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:46.465853 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.326250 (+1860397us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b540036cc54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:48.326350 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.326435 (+    85us) raft_consensus.cc:1685] finished
0709 13:06:48.326454 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.326652 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96637,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10871,"spinlock_wait_cycles":857733376,"thread_start_us":87443,"threads_started":1,"wal-append.queue_time_us":916670,"wal-append.run_cpu_time_us":1600,"wal-append.run_wall_time_us":1052718}
I20260709 13:06:48.328744 23400 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.319787 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.329378 23429 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.319878 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.332338 21063 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:46.582232 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:48.334357 21799 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:48.334456 23385 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:48.337069 23373 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:48.338315 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.339242 21143 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
W20260709 13:06:48.340860 23413 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.438s	user 0.005s	sys 0.018s
I20260709 13:06:48.342024 21141 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:48.343523 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:48.345624 21219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2336) took 1848 ms (client timeout 1885 ms). Trace:
I20260709 13:06:48.296130 23427 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:48.346135 21219 rpcz_store.cc:269] 0709 13:06:46.497545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.500522 (+  2977us) service_pool.cc:224] Handling call
0709 13:06:48.345198 (+1844676us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54002ebb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.345458 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3281840896}
I20260709 13:06:48.348168 23427 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.348838 23429 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:48.350378 21147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2483) took 2444 ms (client timeout 2706 ms). Trace:
I20260709 13:06:48.351193 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:48.351367 23429 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.352674 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:48.353720 21217 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:48.353961 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.350446 21212 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
W20260709 13:06:48.306594 21221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2329) took 1819 ms (client timeout 2083 ms). Trace:
I20260709 13:06:48.354568 21067 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:48.355130 21212 pending_rounds.cc:85] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 2
I20260709 13:06:48.360623 21141 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:06:48.420958 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:48.355875 21221 rpcz_store.cc:269] 0709 13:06:46.486936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.516375 (+ 29439us) service_pool.cc:224] Handling call
0709 13:06:48.304848 (+1788473us) spinlock_profiling.cc:243] Waited 9 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:48.306535 (+  1687us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20695808}
I20260709 13:06:48.379105 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.586232 21217 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:06:48.350944 21147 rpcz_store.cc:269] 0709 13:06:45.905986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.537482 (+631496us) service_pool.cc:224] Handling call
0709 13:06:48.350025 (+1812543us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54001ead54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.350218 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423433600}
W20260709 13:06:48.381285 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.296162 23431 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:48.453019 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:48.381059 21142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2484) took 2365 ms (client timeout 2083 ms). Trace:
I20260709 13:06:48.597510 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.602979 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.383221 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:48.603018 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.604982 21138 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:48.596385 21212 pending_rounds.cc:107] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:06:48.607987 21212 raft_consensus.cc:2889] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:48.604398 21142 rpcz_store.cc:269] 0709 13:06:45.927577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.316483 (+388906us) service_pool.cc:224] Handling call
0709 13:06:48.219824 (+1903341us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b540030aa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.293047 (+ 73223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":946411904}
W20260709 13:06:48.544359 21218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 215) took 2059 ms (client timeout 2249 ms). Trace:
W20260709 13:06:48.586979 23415 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.434s	user 0.006s	sys 0.042s
I20260709 13:06:48.490115 21063 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:06:48.614099 21215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2326) took 2127 ms (client timeout 1701 ms). Trace:
I20260709 13:06:48.378620 23425 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:48.610628 21218 rpcz_store.cc:269] 0709 13:06:46.484722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.485596 (+   874us) service_pool.cc:224] Handling call
0709 13:06:48.543818 (+2058222us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54002e5254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.544127 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949546752}
W20260709 13:06:48.615536 21765 connection.cc:256] RPC call timeout handler was delayed by -1.4797934179999856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
I20260709 13:06:48.614836 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.617218 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.424863 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:48.619182 21070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 254) took 2740 ms (client timeout 1641 ms). Trace:
W20260709 13:06:48.615437 21215 rpcz_store.cc:269] 0709 13:06:46.486148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.491378 (+  5230us) service_pool.cc:224] Handling call
0709 13:06:48.613727 (+2122349us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54002ee5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.613930 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472558336}
I20260709 13:06:48.623987 23009 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:48.526137 23309 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:48.628154 21143 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
W20260709 13:06:48.625197 21070 rpcz_store.cc:269] 0709 13:06:45.875722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.875871 (+   149us) service_pool.cc:224] Handling call
0709 13:06:48.603907 (+2728036us) spinlock_profiling.cc:243] Waited 91.2 ms on lock 0x7b54000dd254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.616077 (+ 12170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209678592}
I20260709 13:06:48.628976 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.633455 21069 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:06:48.632645 20969 catalog_manager.cc:5697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 5, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 5 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.634961 23321 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.535163 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:48.588510 23313 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:48.635521 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.634852 21145 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:06:48.637663 23321 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.636880 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.642973 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.641311 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:48.643965 21139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1819) took 3586 ms (client timeout 2137 ms). Trace:
I20260709 13:06:48.644369 23318 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.627552 23249 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:48.648166 21139 rpcz_store.cc:269] 0709 13:06:45.040255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.040393 (+   138us) service_pool.cc:224] Handling call
0709 13:06:48.609009 (+3568616us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540032c3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:48.626270 (+ 17261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3711760640}
I20260709 13:06:48.650590 21146 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:48.654297 23402 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:48.654053 23318 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.656700 23318 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.657836 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.658584 21145 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:48.659042 21793 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:48.659446 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.632228 21796 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:48.626863 23431 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.665832 21140 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:48.673266 21147 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:48.654963 23402 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.684427 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:48.688426 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 261) took 2146 ms. Trace:
I20260709 13:06:48.688894 21068 rpcz_store.cc:276] 0709 13:06:46.541466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.541626 (+   160us) service_pool.cc:224] Handling call
0709 13:06:46.542042 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:46.542462 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:46.542506 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:46.717626 (+175120us) log.cc:844] Serialized 26 byte log entry
0709 13:06:46.872947 (+155321us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:46.872989 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.873060 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.585876 (+1712816us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:48.687944 (+102068us) raft_consensus.cc:1685] finished
0709 13:06:48.687965 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.688250 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1198,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":246009088,"thread_start_us":132981,"threads_started":1,"wal-append.queue_time_us":231979}
I20260709 13:06:48.695433 21214 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:48.696633 23321 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:48.697355 21113 connection.cc:256] RPC call timeout handler was delayed by -1.2929273290000083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
W20260709 13:06:48.697926 21113 connection.cc:256] RPC call timeout handler was delayed by -1.1336273289999781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
I20260709 13:06:48.698933 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:48.700315 23218 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.700875 23338 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:48.684160 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.684008 21141 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:48.712337 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.719401 21765 leader_election.cc:400] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4.
I20260709 13:06:48.715488 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:48.719614 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:48.723312 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:48.727957 21217 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:06:48.738289 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.744127 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.745600 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.746991 23249 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:48.746796 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.709379 21795 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:48.757718 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:48.758396 21139 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:48.760994 21147 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:48.765961 23352 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.004s
I20260709 13:06:48.767426 23318 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.768050 23318 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:48.768352 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.768874 21214 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:06:48.746830 23009 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:06:48.769873 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:48.783126 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:48.784330 20970 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 222) took 19324 ms (client timeout 15000 ms). Trace:
I20260709 13:06:48.784518 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.777133 23402 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:48.785403 23402 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.785935 23431 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:48.786059 23402 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:48.777809 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.787789 21071 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:48.788750 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.789386 21071 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:48.790270 21147 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:06:48.772151 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:48.790661 21071 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:48.778486 21064 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:48.780812 20969 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.784603 23365 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:48.784958 20970 rpcz_store.cc:269] 0709 13:06:29.459636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.459841 (+   205us) service_pool.cc:224] Handling call
0709 13:06:48.784007 (+19324166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2055107,"reported_tablets":15}
I20260709 13:06:48.777244 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.780967 23318 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:48.793768 21072 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:48.797717 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:48.770769 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.754997 21067 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:48.795074 21138 raft_consensus.cc:2376] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7. Re-sending same reply.
W20260709 13:06:48.800261 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 99439.7us
W20260709 13:06:48.801719 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 49977.5us
I20260709 13:06:48.781924 21214 pending_rounds.cc:85] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 5
I20260709 13:06:48.802894 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:48.803357 21214 pending_rounds.cc:107] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:06:48.803666 21214 raft_consensus.cc:2889] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:48.811074 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 200063us
I20260709 13:06:48.827169 21066 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:48.831727 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:48.832760 21221 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:48.834154 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:48.835379 21218 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:06:48.865212 21144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.007s	sys 0.011s
W20260709 13:06:49.005645 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 89932.2us
I20260709 13:06:48.836212 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:48.980365 21145 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:48.877437 21140 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
W20260709 13:06:49.008196 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 198272us
W20260709 13:06:48.880030 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 57271.8us
I20260709 13:06:48.824460 20969 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.934232 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 209) took 3974 ms. Trace:
I20260709 13:06:48.858165 21212 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:06:48.999782 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 330092us
W20260709 13:06:48.997628 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 83983.4us
I20260709 13:06:48.867863 21069 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:48.836568 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:48.999742 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.916261 21142 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:48.837180 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.892669 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.011171 21214 rpcz_store.cc:276] 0709 13:06:44.935970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.936591 (+   621us) service_pool.cc:224] Handling call
0709 13:06:44.936874 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.359712 (+3422838us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b58004658d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f424780aa 00007f7f424fa447 00007f7f42512fe7 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4
0709 13:06:48.879443 (+519731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.909815 (+ 30372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1829365120}
W20260709 13:06:49.007691 21144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.014s	sys 0.013s
I20260709 13:06:49.023312 21072 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:49.024317 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:49.025167 21218 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:48.913838 23337 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:06:49.000581 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:48.966326 23218 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:49.005811 23436 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.036479 21219 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:48.934095 23401 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.021051 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:48.999051 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.034392 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.043469 21221 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:06:49.038131 23433 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
I20260709 13:06:49.039852 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.040529 23198 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.006002 23307 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:06:49.042652 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.050642 21070 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:49.028635 20969 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 4, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.046228 21799 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:49.052477 21792 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:49.017246 23425 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:06:49.030521 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:06:49.056929 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 143139us
I20260709 13:06:49.050958 21065 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:48.959098 23365 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.069638 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.032902 23298 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 13:06:49.078671 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.080291 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:49.107448 21187 connection.cc:256] RPC call timeout handler was delayed by -1.0158770069999719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
I20260709 13:06:49.107622 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.108281 21219 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:49.108397 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.080499 23365 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:49.109452 21219 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:49.109815 23365 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.110412 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.110380 23365 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.048098 23373 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:49.111735 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-8b6dd6862dd14baba4a82c5a837bc802 received from {username='slave'} at 127.0.0.1:54620
I20260709 13:06:49.112377 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-8b6dd6862dd14baba4a82c5a837bc802 on tablet 8b6dd6862dd14baba4a82c5a837bc802 with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:49.112260 23373 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.112799 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.112967 23365 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:49.114006 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.114172 21212 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:49.083449 21793 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:06:49.114673 21216 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:49.115645 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.073877 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:49.116252 21217 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:49.116703 21800 raft_consensus.cc:2368] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7.
I20260709 13:06:49.117146 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.117257 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.118316 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.118420 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.072959 23402 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:49.114500 23373 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:49.087862 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.090217 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.120290 21070 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 2.
I20260709 13:06:49.121210 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.121773 21070 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:49.122607 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.084895 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:49.090415 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:49.123701 21069 raft_consensus.cc:2368] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5.
I20260709 13:06:49.124423 21765 leader_election.cc:400] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7.
I20260709 13:06:49.124768 21765 leader_election.cc:403] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:49.090922 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.116812 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.126077 21219 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:49.126255 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.123679 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.091022 21063 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:49.127640 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.127969 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.128223 21068 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:49.124643 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:49.092025 23307 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:49.129067 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.055833 23239 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:49.054059 23218 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:49.130355 23239 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:49.130545 23218 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:49.131227 21071 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:49.132270 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.133044 21799 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:49.133185 23218 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:49.093223 23434 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.376s	user 0.000s	sys 0.001s
I20260709 13:06:49.133630 23239 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:49.052508 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.092525 23441 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:49.099732 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.134307 23249 raft_consensus.cc:2764] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:06:49.134488 21213 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:49.134910 21065 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:49.100414 21215 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:49.135720 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.054651 23198 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:49.136423 21217 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:49.054467 23436 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.129074 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:49.137619 21063 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:49.138924 23436 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:49.092139 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:49.103401 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:49.114086 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.123713 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.135460 23401 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:49.102061 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.140923 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:49.142031 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.142784 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.102656 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.144984 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.141515 21797 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:49.145820 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.147415 21799 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:49.095436 21066 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:49.111155 21139 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:06:49.160351 21067 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:06:49.164204 21063 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:49.185060 21212 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:49.188602 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.188086 20969 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.245251 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.253959 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:49.242712 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:49.254786 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.254540 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.314357 21221 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
W20260709 13:06:49.246441 21111 connection.cc:256] RPC call timeout handler was delayed by -2.0803420779999442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
I20260709 13:06:49.317358 21072 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:49.202375 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.315212 21215 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
W20260709 13:06:49.260142 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 168139us
I20260709 13:06:49.316227 21214 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:06:49.322221 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:49.196578 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 166389us
I20260709 13:06:49.253556 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:49.319934 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.325840 21071 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:49.253177 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:49.326793 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.313727 21220 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:49.156127 23296 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:49.326984 21217 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:49.311898 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 206380us
I20260709 13:06:49.324950 21216 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:06:49.323536 21068 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:49.331889 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.352582 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.352728 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.352819 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.241400 21064 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:49.354456 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:49.367695 21068 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:49.372009 21063 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:06:49.372092 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.373435 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.374105 21212 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
W20260709 13:06:49.202018 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.341318 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.354977 21066 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:49.378603 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.361238 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.375398 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.379346 21067 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:49.275944 21795 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:06:49.276412 23337 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:49.376605 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 123191us
I20260709 13:06:49.379400 21220 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:49.376972 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.246665 23445 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.260970 23401 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:49.380089 21072 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:49.381086 21767 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:49.376420 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.330828 23425 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.241518 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.376442 20969 catalog_manager.cc:5697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 4, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.388717 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.331265 23442 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:49.398756 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.407418 21139 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.380055 21071 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:49.404412 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:49.375026 23313 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.384320 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:49.331455 21793 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:06:49.409875 21069 raft_consensus.cc:2410] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:49.354836 23198 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:49.384323 23337 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:49.449414 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 71719.6us
I20260709 13:06:49.449594 21212 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.450480 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:49.452173 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69836.9us
I20260709 13:06:49.452386 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.450675 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:49.456491 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.460875 21072 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:49.395905 23428 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:06:49.462988 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 242266us
W20260709 13:06:49.457464 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 69450.3us
W20260709 13:06:49.462642 23439 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.405s	user 0.001s	sys 0.000s
I20260709 13:06:49.463145 21140 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:49.408687 23309 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:49.452028 23249 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7.
I20260709 13:06:49.462922 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.464741 21145 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:49.464619 23296 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:49.449688 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:49.456784 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:49.476289 21139 raft_consensus.cc:2368] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:49.395402 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.467109 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.468127 21035 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:49.471871 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:49.471014 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66129.3us
I20260709 13:06:49.460798 23337 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:49.482844 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.459492 23453 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:49.488097 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:49.484512 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 71969.9us
I20260709 13:06:49.481699 20969 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 6, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 6 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:06:49.397728 23445 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.469702 23428 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.471305 23309 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.497279 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.498222 21794 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:49.499153 23450 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:49.499711 23450 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:49.500167 23450 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:49.486950 23401 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.504225 23401 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:49.495687 23428 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:49.495020 20969 catalog_manager.cc:5697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 5, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 5 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:06:49.508481 20969 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.467262 23298 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:06:49.459628 23318 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:06:49.447597 23425 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:49.499471 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 128114us
I20260709 13:06:49.512061 23309 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:49.483481 23198 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:49.539616 23198 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:49.540141 23198 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:49.540352 23457 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:49.540815 23457 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:49.541191 23457 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:49.510829 20969 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.528491 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.543716 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.543998 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:49.544956 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.529080 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.529512 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.546190 23441 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:49.530831 21218 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:49.532572 23296 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:49.482228 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.487764 23453 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:49.589663 23453 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:49.459381 23442 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:06:49.589744 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 109018us
I20260709 13:06:49.590631 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:49.592641 23307 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:49.593026 23307 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.593130 23352 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:49.593456 23307 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:49.545573 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:49.594312 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 48819.9us
I20260709 13:06:49.594394 23239 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:49.594843 23239 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.595069 21184 leader_election.cc:400] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7.
I20260709 13:06:49.595222 23239 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:49.595434 21184 leader_election.cc:403] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:06:49.596308 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:49.596729 23365 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7.
I20260709 13:06:49.598520 23444 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:49.598656 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:49.598985 23444 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.603947 23444 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:49.604964 23307 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.606207 23445 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:49.606690 21142 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.593528 23428 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:49.609279 23402 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:49.609889 23402 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:49.610191 23428 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:49.610842 23402 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:49.529297 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.546312 21138 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.588384 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:49.599592 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 211) took 1345 ms. Trace:
I20260709 13:06:49.546360 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.599684 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:49.600764 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:49.600823 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.618920 21143 raft_consensus.cc:2376] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6. Re-sending same reply.
I20260709 13:06:49.460558 23313 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:49.602738 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:49.622124 21147 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:49.603462 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:49.607512 23307 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:49.588384 21216 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.600224 21146 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:49.601572 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.543469 20969 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 4, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 4 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:49.644251 20969 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 4, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.642975 21795 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:06:49.650061 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.629457 21798 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:49.653401 21139 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.499190 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:49.657518 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 56870.2us
I20260709 13:06:49.616598 21144 rpcz_store.cc:276] 0709 13:06:48.254115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.262168 (+  8053us) service_pool.cc:224] Handling call
0709 13:06:48.262642 (+   474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:48.265943 (+  3301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.265976 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:48.267301 (+  1325us) log.cc:844] Serialized 52 byte log entry
0709 13:06:49.026278 (+758977us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:49.035841 (+  9563us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.198990 (+163149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.199118 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:49.599348 (+400230us) raft_consensus.cc:1685] finished
0709 13:06:49.599376 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.599532 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5800,"prepare.queue_time_us":7641,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":65874,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":336,"thread_start_us":912515,"threads_started":2,"wal-append.queue_time_us":778644}
I20260709 13:06:49.546671 23441 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.668886 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.670703 23442 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:49.683607 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.684409 21139 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.685350 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.686311 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:49.686488 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.687778 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.687104 21139 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:49.688679 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.688798 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.689412 21142 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.690521 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:49.692323 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52451.2us
I20260709 13:06:49.692706 21142 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:49.632594 23425 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:49.702682 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.707679 21147 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.628636 21796 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:49.712823 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.714061 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.714100 21138 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:49.715322 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:49.711464 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 212565us
I20260709 13:06:49.712401 23444 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.716516 23442 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:49.717425 20969 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 3, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 3 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:49.720970 23313 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:49.736198 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 137651us
I20260709 13:06:49.717449 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.737228 21143 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:49.722131 23457 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:49.727643 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 259769us
I20260709 13:06:49.744231 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:49.756973 21139 raft_consensus.cc:2368] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:49.713027 23307 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:49.632135 23445 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:06:49.621327 23309 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:49.690132 23441 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:49.735870 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.767591 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:49.721236 23321 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.683606 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:49.738444 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.738623 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.741096 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:49.628348 23401 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:49.771545 21144 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
W20260709 13:06:49.744851 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1856071900000416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
I20260709 13:06:49.771662 21147 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:49.772733 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.691799 23436 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:06:49.766283 21141 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:49.766294 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 93129.8us
I20260709 13:06:49.730527 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:49.747083 20969 catalog_manager.cc:5697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 4, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.767107 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:49.772092 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1856071900000416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
I20260709 13:06:49.769838 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:49.767635 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:49.776679 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1854910509999854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2634s for a response
I20260709 13:06:49.710788 23451 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:49.775597 21071 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
W20260709 13:06:49.777139 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1611660400000119s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
I20260709 13:06:49.775713 21142 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:49.787006 23402 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:49.789791 23402 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:49.776203 23373 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:06:49.790918 23402 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.792936 21792 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:49.795485 23296 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:49.795981 23296 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:49.796916 23296 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:49.798101 23296 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:49.799963 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 69532.9us
I20260709 13:06:49.806308 23444 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:49.806747 23309 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:49.808342 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.783480 20969 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:49.814790 21113 connection.cc:256] RPC call timeout handler was delayed by -1.4038645530000622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 13:06:49.816030 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 129801us
I20260709 13:06:49.819561 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.820253 21068 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:49.774900 23307 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:49.819360 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:49.821560 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:49.826076 21111 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:49.825419 23249 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:49.829025 23444 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:49.830103 23450 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.831458 20969 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.832617 21140 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:49.851459 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.852612 21065 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:49.817487 23459 raft_consensus.cc:2764] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:06:49.830549 23453 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:49.833616 23457 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:49.780351 23321 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:49.863932 21113 connection.cc:256] RPC call timeout handler was delayed by -1.2923610589999726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2288s for a response
I20260709 13:06:49.856247 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:49.856492 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 109862us
I20260709 13:06:49.778965 23198 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:49.865329 21069 raft_consensus.cc:2368] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:49.865475 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:06:49.855125 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0251410599998962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2768s for a response
I20260709 13:06:49.860246 20969 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:06:49.868817 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 99281.5us
I20260709 13:06:49.851069 23309 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:49.848516 23249 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:49.884620 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:49.890007 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:49.858613 23009 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:49.896682 21066 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:49.893214 20969 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 4, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.898906 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:49.888494 23352 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:49.912851 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 86836.2us
I20260709 13:06:49.903620 23307 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:49.917519 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:49.920953 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.921741 21065 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:49.891875 23309 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.923135 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:49.922926 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.927713 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.829350 23466 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:06:49.913705 23373 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:49.929776 20969 catalog_manager.cc:5697] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.937583 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:49.937695 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:49.938917 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:49.890652 23401 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:06:49.937418 23445 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:49.919567 23444 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:49.945591 21799 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:49.937472 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:49.935150 23373 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:49.943631 21110 leader_election.cc:336] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.681s (SENT)
I20260709 13:06:49.948040 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:49.943579 21217 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.007s
I20260709 13:06:49.919147 23352 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:49.929483 23309 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:49.957141 20969 catalog_manager.cc:5697] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:50.025964 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.001246 23373 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:50.006996 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:50.011035 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:49.952833 23444 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.013381 21217 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
W20260709 13:06:50.045955 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 109235us
W20260709 13:06:50.017648 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 92674.9us
I20260709 13:06:50.018827 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:50.040341 20969 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:50.047672 21801 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:50.037714 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:06:50.041530 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 127106us
I20260709 13:06:49.940316 23337 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:50.050866 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.053907 21065 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:50.054375 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.054080 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:50.048436 20969 catalog_manager.cc:5697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 6, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.020642 23352 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:50.057549 21069 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:06:50.001641 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 112588us
I20260709 13:06:50.056823 20969 catalog_manager.cc:5697] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:50.067745 21066 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:06:50.089301 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.089327 20969 catalog_manager.cc:5697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 4 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:06:50.093200 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:50.097288 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:50.111425 21065 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:50.065287 23436 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:50.099628 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59012.3us
W20260709 13:06:50.068456 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 124851us
I20260709 13:06:50.070181 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.115890 21063 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:50.050530 23444 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:50.079844 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 185305us
I20260709 13:06:50.095909 23402 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:50.099355 21767 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:50.057811 23457 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:06:50.099565 23321 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:50.102180 23198 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:50.109653 23239 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.001s
I20260709 13:06:50.096869 23466 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:50.139168 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.112638 20969 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.051442 23451 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:50.130828 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51410.2us
I20260709 13:06:50.143927 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:50.132535 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.053278 23249 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:50.078882 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:50.146421 21066 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:50.133634 23458 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:50.160168 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:50.156787 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:50.162456 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:50.156800 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 102764us
I20260709 13:06:50.163578 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:50.164211 21138 raft_consensus.cc:2410] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:06:50.141134 23474 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:50.168330 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:50.172133 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:50.169157 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:50.146166 23466 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.168011 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:50.172513 21147 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:50.180248 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:50.180886 21139 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:50.178283 21140 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:50.168475 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:50.145469 23451 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.145321 23239 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:50.182402 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:50.182967 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.158738 23321 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:50.156883 23402 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:50.195514 21147 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:50.182767 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.157220 23469 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:06:50.158842 23009 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:50.167270 23298 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:50.211535 23298 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.211805 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.211992 23298 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:50.207779 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:50.212538 21212 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:50.212852 21069 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:50.213272 21221 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:50.213646 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.214178 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:50.200222 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.214494 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.197846 21066 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:50.199733 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.211988 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.183632 23466 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:50.210124 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.134583 23441 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:50.214452 21219 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:06:50.231961 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.216341 21215 raft_consensus.cc:2376] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5. Re-sending same reply.
I20260709 13:06:50.147225 23436 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:06:50.282518 21218 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
W20260709 13:06:50.281476 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 111169us
I20260709 13:06:50.195099 23451 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:50.280064 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 80942.7us
I20260709 13:06:50.186575 23474 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:50.208750 23009 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:50.312374 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 98262.4us
I20260709 13:06:50.197431 23321 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:50.295890 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.314180 21066 raft_consensus.cc:2368] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5.
I20260709 13:06:50.346396 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:50.347150 21144 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 1.
W20260709 13:06:50.295065 23475 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:06:50.349421 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.350512 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.280776 23198 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:50.292413 23337 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:50.347507 23239 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:50.354609 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:50.356007 21071 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:06:50.356070 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 197839us
I20260709 13:06:50.356416 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.359951 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.360782 21071 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:50.361749 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:50.362037 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:50.362581 21066 raft_consensus.cc:2410] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:06:50.362649 21071 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:50.364464 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:50.293658 23441 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:50.317703 23009 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:50.318392 23321 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:50.372133 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:50.372807 21144 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:50.375264 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 230182us
I20260709 13:06:50.376116 21111 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:50.379941 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:50.397437 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:50.398483 21066 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:50.278952 23488 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.170s
I20260709 13:06:50.409536 23450 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:50.410130 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 48654.5us
I20260709 13:06:50.421540 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.427102 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:50.424018 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:50.439040 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65729.1us
W20260709 13:06:50.441601 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44317.7us
I20260709 13:06:50.442489 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.447960 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:50.443087 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:50.450695 21800 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:50.430871 23239 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:50.361377 21215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:50.456189 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:50.466243 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:50.466993 21796 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:50.475546 23491 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 13:06:50.475869 23487 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:06:50.478258 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.452942 23239 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:50.473068 23296 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.675s	user 0.006s	sys 0.007s
I20260709 13:06:50.454218 23469 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:50.514994 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 72423.9us
W20260709 13:06:50.482748 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 102857us
I20260709 13:06:50.459734 23441 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:50.515362 21070 raft_consensus.cc:2410] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:06:50.516238 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:50.516917 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:50.512683 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44074.1us
W20260709 13:06:50.528095 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 73324.9us
W20260709 13:06:50.542685 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 188071us
I20260709 13:06:50.543987 21147 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:50.512279 23239 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:50.483525 23198 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:06:50.553032 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 119081us
I20260709 13:06:50.521988 23441 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.559229 21066 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:50.565636 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.561892 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:50.568715 21216 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:50.566694 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:50.566943 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.560274 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:50.570447 21142 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:50.564303 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:50.588702 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 140844us
I20260709 13:06:50.590090 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:50.564777 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.445479 23402 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:50.598908 21066 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:50.600867 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:50.600414 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56265.5us
W20260709 13:06:50.601483 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 85363.1us
W20260709 13:06:50.602252 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49473.3us
I20260709 13:06:50.476725 23465 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:06:50.605926 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:50.607097 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:50.614423 21071 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:50.562531 23461 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:50.589960 23441 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:50.619177 21147 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:06:50.665845 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49165.3us
I20260709 13:06:50.617694 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.477456 23474 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:50.690845 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 73206us
I20260709 13:06:50.691740 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.692677 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.601033 23486 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:06:50.704401 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 140250us
I20260709 13:06:50.709712 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.615387 23505 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:50.671092 23461 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.723730 21066 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:06:50.725497 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 53597.4us
I20260709 13:06:50.678987 23198 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:50.670567 23499 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:50.788259 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 171845us
I20260709 13:06:50.705317 23313 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:50.792186 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:50.794312 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 205819us
W20260709 13:06:50.798004 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 181998us
I20260709 13:06:50.804684 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:50.805660 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 96071.6us
I20260709 13:06:50.806443 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.758948 23505 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.763842 23461 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:50.710196 23318 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:06:50.807874 23488 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:50.804795 23313 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:50.818075 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:50.823153 23446 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:06:50.834874 21142 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
W20260709 13:06:50.853976 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 61880.6us
I20260709 13:06:50.854898 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:50.822643 23505 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:50.826726 23488 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:50.857944 21069 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.000s
I20260709 13:06:50.861306 21069 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
W20260709 13:06:50.863516 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 76749.2us
I20260709 13:06:50.865178 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:50.838145 23313 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:50.878096 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60136us
I20260709 13:06:50.837329 23495 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:50.841012 23480 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:50.898594 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 92227.6us
W20260709 13:06:50.927764 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 62712.8us
I20260709 13:06:50.928548 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:50.930224 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:50.874776 23465 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:50.863493 23457 raft_consensus.cc:2764] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:50.864768 23488 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:50.935546 23531 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:50.942148 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:50.927788 21147 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
W20260709 13:06:50.959084 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 104262us
I20260709 13:06:50.893919 23479 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:50.929440 23502 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
W20260709 13:06:50.943460 21793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.004s	sys 0.008s
I20260709 13:06:50.967134 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:50.899057 23480 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:50.954186 23484 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:06:50.895644 23450 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.961690 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:50.894732 23501 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:50.897687 23518 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:50.932301 23499 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:50.972200 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:50.973837 23494 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:50.969055 21793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.006s	sys 0.008s
I20260709 13:06:50.954478 23510 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:50.980233 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:50.948709 23465 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:06:51.029587 23451 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.000s
I20260709 13:06:51.029810 23465 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.012635 21113 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:50.960218 23531 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.944056 23503 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.189s
I20260709 13:06:50.973176 23480 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:51.035343 23531 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:51.029644 23506 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:51.004755 23450 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:51.039670 23508 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.358s	user 0.000s	sys 0.001s
I20260709 13:06:51.041133 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:51.042667 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1680) took 1037 ms. Trace:
W20260709 13:06:51.014999 23452 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.004s	sys 0.000s
I20260709 13:06:51.025929 23511 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:06:51.026527 23498 raft_consensus.cc:2764] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:51.030095 23451 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:51.025236 21144 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.000s
I20260709 13:06:50.976603 23515 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:51.059772 23515 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.060274 23515 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:51.061061 23373 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.994s	user 0.010s	sys 0.000s
I20260709 13:06:51.061149 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:51.061707 23321 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.000s
I20260709 13:06:51.061604 23373 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.062144 23321 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.063227 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.063800 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.064164 21147 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:51.064384 21142 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:51.063522 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.065904 21146 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 1.
I20260709 13:06:51.045940 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:51.067451 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.068039 21142 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:06:51.043182 21064 rpcz_store.cc:276] 0709 13:06:50.005100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.396676 (+391576us) service_pool.cc:224] Handling call
0709 13:06:50.432157 (+ 35481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.461272 (+ 29115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:50.461311 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:50.461778 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:06:51.015271 (+553493us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:51.015358 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.015437 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:51.042385 (+ 26948us) raft_consensus.cc:1685] finished
0709 13:06:51.042414 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:51.042592 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6984,"raft.queue_time_us":20672,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":3880,"thread_start_us":495498,"threads_started":1,"wal-append.queue_time_us":530978}
I20260709 13:06:51.050908 21113 leader_election.cc:400] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:51.070535 21113 leader_election.cc:403] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:51.046955 21072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.002s	sys 0.001s
W20260709 13:06:51.046051 23452 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.005s	sys 0.000s
I20260709 13:06:51.052443 23536 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:06:51.053737 23309 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.008s	user 0.009s	sys 0.002s
I20260709 13:06:51.054641 21138 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:06:51.056908 23444 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.912s	user 0.009s	sys 0.002s
I20260709 13:06:51.065063 23239 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:51.075461 21072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.003s	sys 0.001s
W20260709 13:06:51.081168 23352 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.998s	user 0.011s	sys 0.001s
I20260709 13:06:51.074774 23507 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:51.074867 23516 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:51.080447 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:51.102975 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 130853us
I20260709 13:06:51.103926 21142 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:51.021559 23510 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:51.047811 23492 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:51.026954 23526 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:51.085558 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:51.085938 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.107671 21146 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:50.969658 23479 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.113641 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:51.088821 21065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.002s	sys 0.006s
I20260709 13:06:51.091341 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:51.129580 21065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.002s	sys 0.008s
I20260709 13:06:51.093878 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:51.094877 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:51.095685 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:51.051916 23506 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:51.020511 23453 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:51.051170 21144 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:51.105525 23492 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.084329 23536 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:51.097473 23499 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:51.098452 23309 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.105453 23507 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:51.133185 21800 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.670s	user 0.010s	sys 0.007s
I20260709 13:06:51.102320 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 687) took 1078 ms. Trace:
I20260709 13:06:51.140213 21142 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:51.140435 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.148054 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.104707 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:51.141314 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:51.128098 23373 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:51.154443 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:51.155444 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:51.141470 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1681) took 1136 ms. Trace:
I20260709 13:06:51.142429 21071 raft_consensus.cc:2368] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:51.019390 23501 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.108765 23526 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.144793 21797 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:51.145857 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:51.146080 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:51.146381 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:51.147050 21796 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.001s
I20260709 13:06:51.105093 23352 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:51.148315 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:51.175873 21216 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:51.145848 23518 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:51.176185 21066 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:51.101336 23444 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.141423 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:51.148123 23536 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:51.177511 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:51.178254 21219 raft_consensus.cc:2368] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:51.180279 21216 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:51.180871 23444 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:51.181180 21113 leader_election.cc:400] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
W20260709 13:06:51.182801 23298 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.904s	user 0.008s	sys 0.003s
I20260709 13:06:51.147364 23492 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:51.183012 23506 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:51.152047 23507 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:51.155848 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 51238.6us
I20260709 13:06:51.169961 23479 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:51.170383 23510 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.171507 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:51.156805 23321 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:51.161497 21794 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:51.163030 23309 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:51.145699 23452 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:51.188519 23479 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:51.172338 23505 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.182933 21113 leader_election.cc:403] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:06:51.178994 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:51.153905 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:51.190249 21219 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:51.149883 21800 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:51.173247 23307 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:51.187983 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:51.168710 21212 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:51.176517 21796 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:51.152340 23453 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:51.198738 21800 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:51.182638 21213 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:51.168897 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:51.224869 21111 leader_election.cc:400] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:51.230486 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:51.224997 21218 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:51.188871 23452 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.192854 23298 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.190289 23526 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:51.235545 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.240520 23352 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:51.240715 23509 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4.
I20260709 13:06:51.241532 23509 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:51.233671 23479 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:51.250402 21071 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:51.252075 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.252195 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:51.256327 21797 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:51.262207 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 76126.2us
I20260709 13:06:51.262931 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:51.263953 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:51.244167 23445 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:51.265961 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:51.266893 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.268249 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:51.275972 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:51.254978 23479 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:51.285815 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:51.266307 23453 raft_consensus.cc:2764] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:06:51.293429 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 140344us
I20260709 13:06:51.228366 23518 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.279568 23524 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.281347 23401 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:51.244863 23488 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.292641 23373 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:51.311401 23542 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:51.311756 23373 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:51.288147 23239 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:51.288961 23479 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:51.293010 23307 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:51.339918 23512 raft_consensus.cc:2764] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:06:51.341276 23541 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:06:51.343900 23492 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.345340 21144 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:51.347036 23452 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:51.317132 23524 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.351043 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1687) took 1135 ms. Trace:
I20260709 13:06:51.351591 21065 rpcz_store.cc:276] 0709 13:06:50.215433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.455114 (+239681us) service_pool.cc:224] Handling call
0709 13:06:50.467623 (+ 12509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.467946 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:50.467972 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:50.512507 (+ 44535us) log.cc:844] Serialized 26 byte log entry
0709 13:06:51.130147 (+617640us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:51.130208 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.130308 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:51.350735 (+220427us) raft_consensus.cc:1685] finished
0709 13:06:51.350767 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:51.350966 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64124,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":11454,"thread_start_us":570043,"threads_started":1,"wal-append.queue_time_us":554655}
I20260709 13:06:51.288920 23510 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:06:51.358325 21146 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:51.312811 23401 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:51.359616 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 93724.1us
W20260709 13:06:51.392089 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 102877us
I20260709 13:06:51.393157 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:51.395823 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:51.398113 21221 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:06:51.400408 23450 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:51.401705 23470 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.301s	user 0.000s	sys 0.004s
I20260709 13:06:51.420477 23451 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:51.437126 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:51.438925 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:51.442045 21071 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:51.443835 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:51.445029 23495 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.448030 23495 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:51.448182 23530 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.287s	user 0.004s	sys 0.000s
I20260709 13:06:51.455271 21071 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:51.362844 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:51.463887 23501 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:51.331116 23552 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.146s
I20260709 13:06:51.464996 23540 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:51.465282 23552 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:51.465775 23552 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:51.468436 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 75395us
W20260709 13:06:51.468281 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 175861us
W20260709 13:06:51.476969 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 114229us
I20260709 13:06:51.477843 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:51.362668 23401 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:51.360530 23307 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:51.383841 23466 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.026s	user 0.010s	sys 0.002s
I20260709 13:06:51.356513 23524 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:51.374614 23505 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:51.349582 23531 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.498504 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:51.502323 23531 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:51.503172 23531 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:51.505409 21064 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:06:51.493863 23298 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:51.375722 23539 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:51.482963 23492 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:51.538875 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:51.513954 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:51.495142 23296 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:51.466722 23552 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.153337 21793 rpcz_store.cc:276] 0709 13:06:50.017274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.020906 (+  3632us) service_pool.cc:224] Handling call
0709 13:06:50.090192 (+ 69286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:50.099702 (+  9510us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:50.286695 (+186993us) log.cc:844] Serialized 14 byte log entry
0709 13:06:51.025279 (+738584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:51.045829 (+ 20550us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:51.049388 (+  3559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.081360 (+ 31972us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:51.095985 (+ 14625us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51896,"prepare.queue_time_us":237910,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":17523,"raft.queue_time_us":46683,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":6637,"thread_start_us":751125,"threads_started":2}
W20260709 13:06:51.548380 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49756.5us
I20260709 13:06:51.507694 23453 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:51.514639 23539 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.507531 23239 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:51.560421 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 89727.8us
I20260709 13:06:51.571345 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:51.560783 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46936.9us
I20260709 13:06:51.574741 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:51.568464 23313 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.660s	user 0.011s	sys 0.000s
I20260709 13:06:51.508437 23466 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:51.541357 23542 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:51.578377 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 397306us
I20260709 13:06:51.583732 21187 leader_election.cc:400] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5.
I20260709 13:06:51.584959 21187 leader_election.cc:403] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:51.586884 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:51.587569 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.588711 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:51.537210 23517 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:06:51.568537 23539 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:51.572283 23524 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:06:51.599720 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45584.8us
I20260709 13:06:51.572489 23296 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:51.577309 23552 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:51.601944 21071 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:51.603726 21144 raft_consensus.cc:2368] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:51.511305 23373 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.570179 23555 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:51.577929 23547 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:51.600896 23402 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 13:06:51.629684 23009 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.228s	user 0.010s	sys 0.001s
I20260709 13:06:51.630195 23009 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.630697 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:51.633939 21064 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:51.618377 23499 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3.
I20260709 13:06:51.584440 23512 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:51.620512 23515 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.003s
I20260709 13:06:51.620087 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 683) took 1699 ms. Trace:
I20260709 13:06:51.631301 23401 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:51.632023 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1879) took 1467 ms. Trace:
I20260709 13:06:51.671321 21138 rpcz_store.cc:276] 0709 13:06:50.165140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.171901 (+  6761us) service_pool.cc:224] Handling call
0709 13:06:51.631779 (+1459878us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b5400424ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:51.631937 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026929024}
I20260709 13:06:51.600561 23445 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 13:06:51.654695 23441 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.964s	user 0.011s	sys 0.001s
I20260709 13:06:51.620173 23557 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:51.588824 23239 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:51.665150 21145 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:51.736089 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:51.582198 23313 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:51.616400 23480 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.000s
I20260709 13:06:51.604565 23510 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:51.668903 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:52.028224 23566 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:06:51.589725 23501 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:06:52.264580 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:52.306560 23526 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.048s	user 0.012s	sys 0.000s
I20260709 13:06:52.031497 23239 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.780668 23557 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.670112 23542 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.731036 23512 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.807879 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:51.611515 23451 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.147s
I20260709 13:06:51.775807 23458 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.671780 23561 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.772893 23453 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:06:52.118906 23495 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.670s	user 0.004s	sys 0.006s
I20260709 13:06:51.884933 23524 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:52.021673 23533 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.000s	sys 0.010s
W20260709 13:06:52.024366 23479 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.661s	user 0.009s	sys 0.000s
I20260709 13:06:51.821398 23296 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:51.835072 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 259205us
I20260709 13:06:52.395516 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:52.191326 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51584.3us
W20260709 13:06:51.853399 21036 connection.cc:256] RPC call timeout handler was delayed by -2.059123201000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 13:06:51.816202 23503 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:52.306453 23536 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.129s	user 0.011s	sys 0.001s
W20260709 13:06:52.299268 23539 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.686s	user 0.006s	sys 0.002s
I20260709 13:06:51.822290 23401 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:52.392372 23533 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.222s	user 0.010s	sys 0.013s
I20260709 13:06:52.035391 23522 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:51.758622 23441 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:51.758975 23459 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:51.595237 23307 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:51.945225 23461 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.094s	user 0.006s	sys 0.004s
I20260709 13:06:51.821875 23482 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5.
I20260709 13:06:51.827464 23402 consensus_queue.cc:1243] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:06:51.820344 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 237167us
I20260709 13:06:51.670202 23515 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:51.670784 21794 rpcz_store.cc:276] 0709 13:06:49.920451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.922330 (+  1879us) service_pool.cc:224] Handling call
0709 13:06:51.619807 (+1697477us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400633dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:51.619990 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2832359680}
I20260709 13:06:51.675348 23488 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:51.611543 23507 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:52.320998 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2504) took 2723 ms (client timeout 1715 ms). Trace:
I20260709 13:06:52.331583 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 751) took 1172 ms. Trace:
I20260709 13:06:52.371439 23526 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:52.332679 23480 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:52.334633 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:52.417392 21064 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:52.411252 21212 rpcz_store.cc:276] 0709 13:06:51.144803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.237162 (+ 92359us) service_pool.cc:224] Handling call
0709 13:06:51.237593 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:51.250398 (+ 12805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.250738 (+   340us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:51.251423 (+   685us) log.cc:844] Serialized 26 byte log entry
0709 13:06:51.363381 (+111958us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:51.363420 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.363497 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:51.615938 (+252441us) raft_consensus.cc:1685] finished
0709 13:06:51.615959 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.317466 (+701507us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8353,"raft.queue_time_us":16263,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":2832,"thread_start_us":83712,"threads_started":1,"wal-append.queue_time_us":352134}
I20260709 13:06:52.337702 21065 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:52.338671 23313 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.384665 23458 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:52.422457 23576 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.423023 23576 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:52.424379 23458 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.401713 23536 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:52.425822 21071 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:52.425343 23576 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:52.352684 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 222) took 1287 ms. Trace:
I20260709 13:06:52.357400 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:52.378404 21221 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:52.403841 23539 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:52.426836 21147 rpcz_store.cc:276] 0709 13:06:51.065713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.075206 (+  9493us) service_pool.cc:224] Handling call
0709 13:06:51.096730 (+ 21524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:51.166943 (+ 70213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.166990 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:51.167046 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:51.174926 (+  7880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.352393 (+1177467us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b5400258fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:52.352447 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.352598 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160865152}
I20260709 13:06:52.428555 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:52.406039 23527 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:52.429322 21147 raft_consensus.cc:2368] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:52.413636 23522 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:52.430152 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:52.396395 21036 connection.cc:256] RPC call timeout handler was delayed by -2.0564751870000464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:06:52.431319 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.635s (SENT) [suppressed 49 similar messages]
I20260709 13:06:52.401036 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 692) took 2615 ms. Trace:
I20260709 13:06:52.404280 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 695) took 2590 ms. Trace:
W20260709 13:06:52.433343 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.405136 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:52.318101 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:52.411041 21140 rpcz_store.cc:269] 0709 13:06:49.597964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.603176 (+  5212us) service_pool.cc:224] Handling call
0709 13:06:52.320763 (+2717587us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b540036d654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:52.320898 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482505984}
I20260709 13:06:52.418087 21214 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:52.432226 21068 rpcz_store.cc:276] 0709 13:06:49.813615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.850352 (+ 36737us) service_pool.cc:224] Handling call
0709 13:06:49.850756 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.861314 (+ 10558us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:50.052356 (+191042us) log.cc:844] Serialized 14 byte log entry
0709 13:06:52.403700 (+2351344us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b1800038da0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a
0709 13:06:52.403881 (+   181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:52.403928 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:52.403993 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.404067 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.404199 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20973,"prepare.queue_time_us":226377,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":9246,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":122,"spinlock_wait_cycles":4371712,"thread_start_us":232692,"threads_started":2,"wal-append.queue_time_us":119205,"wal-append.run_cpu_time_us":2365,"wal-append.run_wall_time_us":1297876}
I20260709 13:06:52.403491 23373 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:52.405514 23501 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:52.419543 23524 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:52.381661 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:52.390270 23561 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:52.336340 23577 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.087s	user 0.000s	sys 0.002s
I20260709 13:06:52.437559 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:52.437295 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:52.476083 21142 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:52.476142 21215 raft_consensus.cc:606] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:52.394913 23479 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:52.478156 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:52.513936 21215 raft_consensus.cc:614] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:52.481184 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:52.515231 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1697) took 1069 ms. Trace:
I20260709 13:06:52.482085 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:52.515697 21068 rpcz_store.cc:276] 0709 13:06:51.446236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.477826 (+1031590us) service_pool.cc:224] Handling call
0709 13:06:52.515157 (+ 37331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:52.516039 21147 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:06:52.426759 23536 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.391968 23495 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:52.519985 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 82511.4us
I20260709 13:06:52.434931 21067 rpcz_store.cc:276] 0709 13:06:49.785725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.786436 (+   711us) service_pool.cc:224] Handling call
0709 13:06:49.822787 (+ 36351us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:06:49.822970 (+   183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.830768 (+  7798us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:52.322477 (+2491709us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b180034fa80. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f45f143c7 00007f7f425269a7 00007f7f425360bf
0709 13:06:52.330863 (+  8386us) log.cc:844] Serialized 14 byte log entry
0709 13:06:52.400622 (+ 69759us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:52.400705 (+    83us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:52.400782 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.400861 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.400978 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":228400,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":11595,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":5061,"spinlock_wait_cycles":19039104,"thread_start_us":275985,"threads_started":2,"wal-append.queue_time_us":66465}
I20260709 13:06:52.409364 23459 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:52.437489 23576 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.522684 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:52.477175 23482 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:52.409183 23461 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:52.522804 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:52.532222 21220 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:52.533098 21068 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:52.528757 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:52.530386 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 99116.7us
W20260709 13:06:52.535625 21036 connection.cc:256] RPC call timeout handler was delayed by -1.942175187000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
I20260709 13:06:52.480189 23581 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:52.408826 23503 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:52.542677 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:52.543197 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:52.545042 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:52.552224 21063 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:52.552588 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:52.526795 23402 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:52.563578 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:52.573339 21067 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:52.483837 23373 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:52.540892 23539 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.518273 23453 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:52.569293 23461 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:52.816982 23561 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:52.840823 23515 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.627521 23522 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:52.626612 23562 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.289s
I20260709 13:06:52.842984 23507 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.843804 23512 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.844592 23479 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:52.543473 23517 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:52.848498 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 312430us
I20260709 13:06:52.595836 21220 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:52.603704 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 752) took 1458 ms. Trace:
I20260709 13:06:52.879392 21218 rpcz_store.cc:276] 0709 13:06:51.145032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.238171 (+ 93139us) service_pool.cc:224] Handling call
0709 13:06:51.238480 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:51.348775 (+110295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.348806 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:51.349272 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:06:51.606502 (+257230us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:51.606548 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.624920 (+ 18372us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b4800281958. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:06:52.563209 (+938289us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540015cf54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:52.563303 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:52.603382 (+ 40079us) raft_consensus.cc:1685] finished
0709 13:06:52.603414 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.603610 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46678,"raft.queue_time_us":112856,"raft.run_cpu_time_us":2551,"raft.run_wall_time_us":156696,"spinlock_wait_cycles":261665280,"thread_start_us":256869,"threads_started":1,"wal-append.queue_time_us":888763}
I20260709 13:06:52.560582 23578 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:52.569547 23557 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:52.565234 23402 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:52.543678 23513 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:52.564122 23581 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:52.556473 23523 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:52.589584 23503 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.750350 23488 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.749979 23451 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:06:52.619681 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 62516.1us
I20260709 13:06:52.915755 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:52.622797 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:52.817917 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1893) took 1239 ms. Trace:
I20260709 13:06:52.916997 21147 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:52.861685 23526 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.844838 23480 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:52.896190 21064 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:52.922875 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 715) took 1162 ms. Trace:
I20260709 13:06:52.855921 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:52.560233 23575 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:06:52.923580 21064 rpcz_store.cc:276] 0709 13:06:51.760185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.330798 (+570613us) service_pool.cc:224] Handling call
0709 13:06:52.922798 (+592000us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:52.925772 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 708) took 1832 ms. Trace:
I20260709 13:06:52.926062 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2533) took 1268 ms. Trace:
W20260709 13:06:52.908650 21070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 690) took 3123 ms (client timeout 3186 ms). Trace:
I20260709 13:06:52.917604 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2537) took 1259 ms. Trace:
I20260709 13:06:52.862063 23441 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:52.777621 23510 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.916415 21144 rpcz_store.cc:276] 0709 13:06:51.578711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.586555 (+  7844us) service_pool.cc:224] Handling call
0709 13:06:52.817644 (+1231089us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b540036c754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:52.817830 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36441344}
I20260709 13:06:52.927017 21138 rpcz_store.cc:276] 0709 13:06:51.657345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.849910 (+1192565us) service_pool.cc:224] Handling call
0709 13:06:52.925983 (+ 76073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:52.547602 23576 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:52.850699 23495 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.627588 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 707) took 1534 ms. Trace:
I20260709 13:06:52.816205 23459 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:52.909652 23402 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:52.918560 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:52.952021 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:52.953756 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2535) took 1295 ms. Trace:
I20260709 13:06:52.954330 21138 rpcz_store.cc:276] 0709 13:06:51.657975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.945370 (+1287395us) service_pool.cc:224] Handling call
0709 13:06:52.945781 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.946027 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.946057 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:52.953359 (+  7302us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:52.953441 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.953524 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.953681 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:52.947341 21067 rpcz_store.cc:276] 0709 13:06:51.092735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.626437 (+1533702us) service_pool.cc:224] Handling call
0709 13:06:52.626890 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.627126 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.627164 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:52.627221 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:52.627272 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.627370 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.627516 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:52.947698 21147 rpcz_store.cc:276] 0709 13:06:51.658532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.555016 (+896484us) service_pool.cc:224] Handling call
0709 13:06:52.917536 (+362520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:52.914085 23578 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:52.920569 23562 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:06:52.961620 21036 connection.cc:256] RPC call timeout handler was delayed by -1.9138428730000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 13:06:52.942806 21070 rpcz_store.cc:269] 0709 13:06:49.785142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.785247 (+   105us) service_pool.cc:224] Handling call
0709 13:06:49.821348 (+ 36101us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:52.905613 (+3084265us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54002ad5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:52.908581 (+  2968us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1595275392}
I20260709 13:06:52.965869 21144 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:52.970060 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1896) took 1388 ms. Trace:
I20260709 13:06:52.970544 21144 rpcz_store.cc:276] 0709 13:06:51.581420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.948616 (+1367196us) service_pool.cc:224] Handling call
0709 13:06:52.948988 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.969730 (+ 20742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.969987 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:52.927759 23479 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.926257 21064 rpcz_store.cc:276] 0709 13:06:51.093058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.924641 (+1831583us) service_pool.cc:224] Handling call
0709 13:06:52.925089 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.925341 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.925378 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:52.925436 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:52.925500 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.925570 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.925704 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:52.973340 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:52.973423 21036 connection.cc:256] RPC call timeout handler was delayed by -1.7693232010000202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
I20260709 13:06:52.973914 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:52.976213 21036 connection.cc:256] RPC call timeout handler was delayed by -1.726123201000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
I20260709 13:06:52.974351 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:52.976711 21036 connection.cc:256] RPC call timeout handler was delayed by -1.6469751870000664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260709 13:06:52.976899 21221 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
W20260709 13:06:52.977152 21036 connection.cc:256] RPC call timeout handler was delayed by -1.6447428730000411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
I20260709 13:06:52.976565 21212 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:06:52.977622 21036 connection.cc:256] RPC call timeout handler was delayed by -1.619923201000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 13:06:52.978219 21036 connection.cc:256] RPC call timeout handler was delayed by -1.6093096030000424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20350000000000001s for a response
W20260709 13:06:52.978603 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4804232010000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 13:06:52.979002 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4705232009999918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
W20260709 13:06:52.979413 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4341428730000416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
W20260709 13:06:52.979785 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3360428729999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
W20260709 13:06:52.980198 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3054428730000609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
I20260709 13:06:52.995200 21219 raft_consensus.cc:2410] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:06:52.963449 23561 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:52.997421 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:52.997187 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:53.001439 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:53.002140 21064 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:53.003930 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:53.005018 23461 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:53.005764 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:53.006486 21212 raft_consensus.cc:2376] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7. Re-sending same reply.
I20260709 13:06:53.008502 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:53.009690 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.010527 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:52.970551 23501 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:52.921222 23488 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.005062 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.013336 21221 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:53.014091 21064 raft_consensus.cc:2410] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:52.970753 23480 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:52.931427 23451 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.017285 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:52.951740 23510 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:52.998303 23561 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:53.113976 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:53.115820 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:53.116989 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 112388us
I20260709 13:06:53.118464 21138 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:53.016669 23373 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:53.144028 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.326982 21067 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:52.998381 23479 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.024725 23501 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:53.117267 23561 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:53.185375 23582 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.232s	user 0.001s	sys 0.001s
I20260709 13:06:53.192502 23578 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:53.342626 23594 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:06:53.289844 21064 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:53.024219 23480 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.345484 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:53.351763 21067 raft_consensus.cc:2368] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
W20260709 13:06:53.339044 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 217640us
I20260709 13:06:53.368759 23488 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:53.370123 23581 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:53.376672 23451 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:53.390775 23501 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:53.461496 23510 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:53.483457 23581 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.484246 23581 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.486850 23581 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:53.492142 23561 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:53.494740 23561 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:53.495767 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.505304 21064 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:53.544658 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1692) took 2110 ms. Trace:
I20260709 13:06:53.545120 21064 rpcz_store.cc:276] 0709 13:06:51.434625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.504600 (+2069975us) service_pool.cc:224] Handling call
0709 13:06:53.505051 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.544368 (+ 39317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.544597 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:53.530764 21146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 226) took 2463 ms (client timeout 2141 ms). Trace:
W20260709 13:06:53.531107 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 241132us
I20260709 13:06:53.533010 23576 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:53.547770 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:53.547137 21146 rpcz_store.cc:269] 0709 13:06:51.067056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.134567 (+ 67511us) service_pool.cc:224] Handling call
0709 13:06:53.530513 (+2395946us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54002bbe54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:53.530680 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461514112}
I20260709 13:06:53.557164 21187 leader_election.cc:400] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:53.557667 21187 leader_election.cc:403] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:06:53.565434 23452 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:53.565980 23452 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:53.567781 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 227838us
W20260709 13:06:53.560427 23402 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.003s
I20260709 13:06:53.552002 23480 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:53.577286 23465 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:53.568544 23562 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:53.570483 23526 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:53.578027 23402 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:53.575683 23452 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:53.617868 21110 leader_election.cc:336] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 3.231s (SENT)
I20260709 13:06:53.618440 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:53.619238 21064 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:53.619004 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1903) took 1218 ms. Trace:
I20260709 13:06:53.623641 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1693) took 2188 ms. Trace:
I20260709 13:06:53.624171 21064 rpcz_store.cc:276] 0709 13:06:51.435018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.570188 (+2135170us) service_pool.cc:224] Handling call
0709 13:06:53.618959 (+ 48771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.619864 (+   905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.623581 (+  3717us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:53.622458 21144 rpcz_store.cc:276] 0709 13:06:52.400789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.971594 (+570805us) service_pool.cc:224] Handling call
0709 13:06:52.971884 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.972317 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:53.256687 (+284370us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.618658 (+361971us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.618707 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:53.618769 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.618838 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.618951 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":429053,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":10935,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":590593,"threads_started":2}
I20260709 13:06:53.629156 23479 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:53.630286 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 82623.5us
I20260709 13:06:53.631337 23543 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:53.632201 23569 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:53.632586 23569 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:53.633420 23569 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.634037 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:53.634744 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 706) took 2542 ms. Trace:
I20260709 13:06:53.634871 23569 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:53.635048 23543 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:53.635828 23543 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:53.636343 23543 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:53.639868 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1888) took 2488 ms. Trace:
I20260709 13:06:53.640337 21142 rpcz_store.cc:276] 0709 13:06:51.151856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.410125 (+258269us) service_pool.cc:224] Handling call
0709 13:06:51.410431 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.516478 (+1106047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.639582 (+1123104us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540032c3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:53.639773 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407880064}
I20260709 13:06:53.642393 23527 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:53.642915 23527 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:53.650058 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1701) took 2179 ms. Trace:
I20260709 13:06:53.620848 23498 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:53.650553 21067 rpcz_store.cc:276] 0709 13:06:51.470807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.649215 (+2178408us) service_pool.cc:224] Handling call
0709 13:06:53.649535 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.649724 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.649750 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.649794 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:53.649835 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.649890 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.649996 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:53.652989 21067 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:53.653584 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1702) took 2170 ms. Trace:
I20260709 13:06:53.653945 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:53.653882 21067 rpcz_store.cc:276] 0709 13:06:51.482993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.652474 (+2169481us) service_pool.cc:224] Handling call
0709 13:06:53.652761 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.653370 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.653535 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:53.656363 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:53.635192 21069 rpcz_store.cc:276] 0709 13:06:51.092457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.426802 (+334345us) service_pool.cc:224] Handling call
0709 13:06:51.427215 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:51.633203 (+205988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.633262 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:51.633325 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:51.633387 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.634518 (+2001131us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54003bf454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:53.634560 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.634688 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473147776}
I20260709 13:06:53.657156 21142 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:53.658493 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 711) took 2057 ms. Trace:
I20260709 13:06:53.659830 21069 rpcz_store.cc:276] 0709 13:06:51.601419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.657259 (+2055840us) service_pool.cc:224] Handling call
0709 13:06:53.657783 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.658045 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.658085 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.658141 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:53.658200 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.658274 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.658421 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:53.662292 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:53.662655 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 713) took 1994 ms. Trace:
I20260709 13:06:53.663496 21069 rpcz_store.cc:276] 0709 13:06:51.667983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.661528 (+1993545us) service_pool.cc:224] Handling call
0709 13:06:53.661955 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.662199 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.662237 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.662297 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:53.662370 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.662441 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.662576 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:53.656384 23441 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.665906 23441 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:53.668608 23441 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:53.669325 23458 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.662756 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:53.671305 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1698) took 2224 ms. Trace:
I20260709 13:06:53.671950 21142 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:53.673620 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:53.674616 21144 raft_consensus.cc:2368] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:53.672030 21064 rpcz_store.cc:276] 0709 13:06:51.446544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.625972 (+2179428us) service_pool.cc:224] Handling call
0709 13:06:53.670756 (+ 44784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.670943 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.670968 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.671010 (+    42us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:53.671047 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.671102 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.671226 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:53.655905 23452 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:53.679585 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:53.669850 23458 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.679844 21069 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:53.689689 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 716) took 1370 ms. Trace:
I20260709 13:06:53.690011 21069 rpcz_store.cc:276] 0709 13:06:52.319356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.675473 (+1356117us) service_pool.cc:224] Handling call
0709 13:06:53.675770 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.689402 (+ 13632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.689627 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:53.692157 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 290) took 1275 ms. Trace:
I20260709 13:06:53.692191 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:53.693135 21069 rpcz_store.cc:276] 0709 13:06:52.416630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.691404 (+1274774us) service_pool.cc:224] Handling call
0709 13:06:53.691684 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.691858 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.691873 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.691921 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:53.691959 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.692010 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.692108 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:53.680615 23607 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:06:53.683606 23569 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:53.714643 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:53.698288 23548 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 2.356s	user 0.015s	sys 0.039s
I20260709 13:06:53.699399 21147 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:53.673269 23526 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:53.701593 23580 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.247s	user 0.000s	sys 0.019s
I20260709 13:06:53.721251 23526 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:53.721416 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:53.706203 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.722126 21147 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
W20260709 13:06:53.722851 21113 connection.cc:256] RPC call timeout handler was delayed by -1.1614921950000507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
I20260709 13:06:53.722966 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:53.723536 23498 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:53.724112 23498 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.726097 23469 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:53.726670 23469 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.692924 21144 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
W20260709 13:06:53.729122 23590 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.868s	user 0.007s	sys 0.007s
I20260709 13:06:53.729276 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:53.711730 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:53.713205 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50996.2us
I20260709 13:06:53.715637 23402 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:53.716137 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 699) took 2626 ms (client timeout 1627 ms). Trace:
W20260709 13:06:53.735574 21063 rpcz_store.cc:269] 0709 13:06:51.090162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.093507 (+  3345us) service_pool.cc:224] Handling call
0709 13:06:53.715899 (+2622392us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540019d2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:53.716055 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3269364096}
I20260709 13:06:53.697718 21146 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:53.717325 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.721789 23526 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:53.723737 21147 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:06:53.723321 21113 connection.cc:256] RPC call timeout handler was delayed by -1.1519600139999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3186s for a response
I20260709 13:06:53.708814 23527 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:53.746762 23505 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:53.847496 23552 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:53.847890 23505 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:53.848554 23552 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:53.848984 23552 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:53.849153 23505 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.849903 23552 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:53.850723 23505 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:51.156126 21072 rpcz_store.cc:276] 0709 13:06:50.005322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.431564 (+426242us) service_pool.cc:224] Handling call
0709 13:06:50.453389 (+ 21825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:50.453654 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:50.479796 (+ 26142us) log.cc:844] Serialized 14 byte log entry
0709 13:06:51.141147 (+661351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:51.141195 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:51.141248 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.141309 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:51.141417 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18921,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":2798,"raft.queue_time_us":56952,"thread_start_us":660917,"threads_started":2,"wal-append.queue_time_us":616888}
I20260709 13:06:53.712543 23543 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.861123 21063 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:53.867357 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 117800us
W20260709 13:06:53.868810 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 133803us
I20260709 13:06:53.870081 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:53.869068 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1880) took 3685 ms. Trace:
I20260709 13:06:53.870767 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:53.871467 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:53.853919 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 293) took 1421 ms. Trace:
I20260709 13:06:53.874640 21072 rpcz_store.cc:276] 0709 13:06:52.432717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.853024 (+1420307us) service_pool.cc:224] Handling call
0709 13:06:53.853377 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.853565 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.853594 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.853639 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:53.853680 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.853735 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.853858 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:53.876304 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 288) took 1460 ms. Trace:
I20260709 13:06:53.876827 21064 rpcz_store.cc:276] 0709 13:06:52.416063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.678997 (+1262934us) service_pool.cc:224] Handling call
0709 13:06:53.720741 (+ 41744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.870928 (+150187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.870958 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.875997 (+  5039us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:53.876063 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.876130 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.876251 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15240,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":113}
I20260709 13:06:53.872787 23198 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:06:53.876183 23453 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:53.871461 21141 rpcz_store.cc:276] 0709 13:06:50.183807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.196330 (+ 12523us) service_pool.cc:224] Handling call
0709 13:06:50.207837 (+ 11507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.465005 (+257168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:50.465042 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:50.479270 (+ 14228us) log.cc:844] Serialized 26 byte log entry
0709 13:06:52.519178 (+2039908us) spinlock_profiling.cc:243] Waited 9.27 ms on lock 0x7b18000bd340. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:06:52.555627 (+ 36449us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:52.555688 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.555797 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:53.728507 (+1172710us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:53.730061 (+  1554us) raft_consensus.cc:1685] finished
0709 13:06:53.730089 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.869005 (+138916us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":206933,"raft.queue_time_us":311580,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":24656,"spinlock_wait_cycles":34326272,"thread_start_us":174293,"threads_started":2,"wal-append.queue_time_us":2333357}
I20260709 13:06:53.863186 23009 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:53.859402 23543 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:53.865980 23458 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:53.917708 23546 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 2.422s	user 0.011s	sys 0.029s
I20260709 13:06:53.917948 23498 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:53.872920 23402 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:06:53.851256 23552 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:53.919262 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 747) took 2776 ms. Trace:
I20260709 13:06:53.878396 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:53.922492 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 44190.4us
I20260709 13:06:53.923434 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:53.923861 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:53.924592 21072 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:53.856719 23527 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:53.921895 21216 rpcz_store.cc:276] 0709 13:06:51.142910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.181040 (+ 38130us) service_pool.cc:224] Handling call
0709 13:06:51.181356 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:51.181851 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.181887 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:51.184082 (+  2195us) log.cc:844] Serialized 52 byte log entry
0709 13:06:51.275192 (+ 91110us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:51.275257 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.275404 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:53.917095 (+2641691us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:53.918910 (+  1815us) raft_consensus.cc:1685] finished
0709 13:06:53.918931 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.919192 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2720,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":15179520,"thread_start_us":90609,"threads_started":1,"wal-append.queue_time_us":177307}
I20260709 13:06:53.876408 23569 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:53.876575 23451 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:53.929255 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.931763 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:53.932938 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.934015 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.934566 21064 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:53.927009 21036 leader_election.cc:400] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:53.935447 23469 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:53.927022 23543 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:53.938072 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.926450 23542 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:53.938869 21064 raft_consensus.cc:2368] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7.
I20260709 13:06:53.938927 23542 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:53.940112 23542 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.941304 23542 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:53.875813 23562 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:06:53.942003 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 705) took 2849 ms. Trace:
I20260709 13:06:53.942444 23562 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:53.942404 21065 rpcz_store.cc:276] 0709 13:06:51.092164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.362232 (+270068us) service_pool.cc:224] Handling call
0709 13:06:51.384095 (+ 21863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.418561 (+1034466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.941745 (+1523184us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b54000db954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:53.941939 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192630528}
I20260709 13:06:53.943709 23451 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:53.944291 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.944730 23451 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.942988 23562 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:53.945461 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.946750 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:53.946461 23562 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.928110 23453 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:53.937357 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:53.948724 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:53.949491 23453 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:53.949857 23453 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:53.950007 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:53.950246 23453 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:53.950758 23543 raft_consensus.cc:2764] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:06:53.951086 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:53.951449 23543 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:53.951849 21064 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:53.952739 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.947314 21065 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:53.953301 21214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 230) took 3386 ms (client timeout 1609 ms). Trace:
I20260709 13:06:53.953730 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:53.953747 21214 rpcz_store.cc:269] 0709 13:06:50.566602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.566732 (+   130us) service_pool.cc:224] Handling call
0709 13:06:53.953100 (+3386368us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b54003fa7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:53.953229 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4249925376}
I20260709 13:06:53.955462 21221 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:06:53.955863 21212 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:53.956193 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:53.953334 21064 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:53.956203 21212 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:53.957508 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:53.960484 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:53.961650 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:53.964632 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 92132.9us
I20260709 13:06:53.963982 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:53.958249 21212 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:53.968228 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:53.968681 21212 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:53.963498 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:53.966364 23569 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:53.982065 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:53.951851 23543 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:54.022940 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 53480us
W20260709 13:06:54.038570 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 70239.1us
W20260709 13:06:54.040525 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 121006us
I20260709 13:06:54.046175 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:53.971397 23509 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:54.041304 21036 leader_election.cc:400] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:54.052930 21036 leader_election.cc:403] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:06:54.052649 23509 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:54.054459 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:54.055406 23557 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:06:54.056088 23557 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:54.056449 23557 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.056809 23557 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:54.041087 23543 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:54.064817 23523 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.065497 23523 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:54.070168 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1876) took 3914 ms. Trace:
I20260709 13:06:54.070785 21139 rpcz_store.cc:276] 0709 13:06:50.155568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.181525 (+ 25957us) service_pool.cc:224] Handling call
0709 13:06:50.198555 (+ 17030us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.401082 (+202527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:50.401113 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.690351 (+3289238us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b2c00010828. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:06:53.692877 (+  2526us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.920184 (+227307us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:53.920229 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.920496 (+   267us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.069747 (+149251us) raft_consensus.cc:1685] finished
0709 13:06:54.069773 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.070067 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":203477,"raft.queue_time_us":304233,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":24944,"spinlock_wait_cycles":43102592,"thread_start_us":273287,"threads_started":2,"wal-append.queue_time_us":354004}
I20260709 13:06:54.089380 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:54.107247 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.112800 21212 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:06:54.113484 21212 pending_rounds.cc:85] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 6
I20260709 13:06:54.113901 21212 pending_rounds.cc:107] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:06:54.114215 21212 raft_consensus.cc:2889] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:54.118049 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:54.118963 21065 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:06:54.120684 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.038467 23444 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:54.121938 21068 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:06:54.122622 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1902) took 1714 ms. Trace:
I20260709 13:06:54.123003 21138 rpcz_store.cc:276] 0709 13:06:52.398041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.968718 (+570677us) service_pool.cc:224] Handling call
0709 13:06:53.004382 (+ 35664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.290451 (+286069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.112740 (+822289us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54001eb4d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:54.112940 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261250304}
W20260709 13:06:54.123456 21145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1870) took 3972 ms (client timeout 3325 ms). Trace:
I20260709 13:06:54.041908 23562 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:54.124004 21145 rpcz_store.cc:269] 0709 13:06:50.151061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.151174 (+   113us) service_pool.cc:224] Handling call
0709 13:06:54.123152 (+3971978us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54001ec3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:54.123357 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3414071168}
I20260709 13:06:54.135186 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.126312 23444 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:54.139302 23444 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:54.121129 23569 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:54.145840 21070 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:54.147400 21070 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:54.148444 21070 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:54.149590 21070 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:54.149459 23523 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.074772 23526 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:54.155004 21036 connection.cc:256] RPC call timeout handler was delayed by -2.0933018769999308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2844s for a response
W20260709 13:06:54.152189 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:54.162765 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:54.183815 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 132081us
I20260709 13:06:54.156087 21140 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
W20260709 13:06:54.189646 21765 connection.cc:256] RPC call timeout handler was delayed by -2.3914489909999475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 13:06:54.190773 21765 connection.cc:256] RPC call timeout handler was delayed by -2.3914489909999475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 13:06:54.191192 21765 connection.cc:256] RPC call timeout handler was delayed by -2.33744899099986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
I20260709 13:06:54.133472 23373 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:54.194391 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:54.199433 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.207422 21069 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:54.235942 21113 leader_election.cc:400] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7.
W20260709 13:06:54.236575 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.236588 21113 leader_election.cc:403] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:54.237599 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.139766 23461 raft_consensus.cc:2764] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:06:54.242403 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:54.192564 21765 connection.cc:256] RPC call timeout handler was delayed by -2.2393489909999289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
W20260709 13:06:54.245960 21765 connection.cc:256] RPC call timeout handler was delayed by -2.1763489909999407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 13:06:54.247769 21765 connection.cc:256] RPC call timeout handler was delayed by -2.1529489909999029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
I20260709 13:06:54.181478 23520 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:54.249555 21765 connection.cc:256] RPC call timeout handler was delayed by -2.1529489909999029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
I20260709 13:06:54.181176 23569 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:54.253295 21765 connection.cc:256] RPC call timeout handler was delayed by -2.1250489909998578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 13:06:54.253649 21765 connection.cc:256] RPC call timeout handler was delayed by -2.1025489909999351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
W20260709 13:06:54.254355 21765 connection.cc:256] RPC call timeout handler was delayed by -2.0512489909998521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
W20260709 13:06:54.254711 21765 connection.cc:256] RPC call timeout handler was delayed by -1.961248990999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
W20260709 13:06:54.255041 21765 connection.cc:256] RPC call timeout handler was delayed by -1.9126489909998554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2141s for a response
W20260709 13:06:54.255395 21765 connection.cc:256] RPC call timeout handler was delayed by -1.8631489909998891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 13:06:54.255707 21765 connection.cc:256] RPC call timeout handler was delayed by -1.5211677249999411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
W20260709 13:06:54.256036 21765 connection.cc:256] RPC call timeout handler was delayed by -1.4896677250000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
W20260709 13:06:54.256326 21765 connection.cc:256] RPC call timeout handler was delayed by -1.4716677249999748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
I20260709 13:06:54.204258 23523 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.191658 23458 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:54.265483 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.262023 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.248559 23499 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:06:54.256706 21765 connection.cc:256] RPC call timeout handler was delayed by -1.4716677249999748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
I20260709 13:06:54.262023 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.269208 21139 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:54.244580 23373 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.266254 21138 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:54.248880 23488 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:54.260015 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:54.268535 21765 connection.cc:256] RPC call timeout handler was delayed by -1.4689489909999338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2634s for a response
I20260709 13:06:54.271624 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1707) took 1305 ms. Trace:
W20260709 13:06:54.268317 21220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 759) took 1934 ms (client timeout 2027 ms). Trace:
I20260709 13:06:54.257134 23498 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:54.274303 21220 rpcz_store.cc:269] 0709 13:06:52.340346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.357055 (+ 16709us) service_pool.cc:224] Handling call
0709 13:06:54.268136 (+1911081us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400442ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:54.268246 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241559424}
W20260709 13:06:54.273531 21765 connection.cc:256] RPC call timeout handler was delayed by -1.4509677249999413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 13:06:54.271410 23527 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:54.278573 21765 connection.cc:256] RPC call timeout handler was delayed by -1.4509677249999413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 13:06:54.264189 23461 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.279222 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.252012 23469 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 13:06:54.280658 21111 connection.cc:256] RPC call timeout handler was delayed by -1.0604310480000549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
I20260709 13:06:54.257740 23453 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.271550 23488 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.281643 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:54.283814 21765 connection.cc:256] RPC call timeout handler was delayed by -1.1692677249999406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20640000000000003s for a response
I20260709 13:06:54.261075 23569 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:54.291739 21765 connection.cc:256] RPC call timeout handler was delayed by -1.1494677250000223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
W20260709 13:06:54.292229 21765 connection.cc:256] RPC call timeout handler was delayed by -1.0351677249999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
I20260709 13:06:54.286046 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.259155 23520 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:54.292536 21765 connection.cc:256] RPC call timeout handler was delayed by -1.0351677249999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
I20260709 13:06:54.295529 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.292877 21138 raft_consensus.cc:2376] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5. Re-sending same reply.
I20260709 13:06:54.268879 23458 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:54.290107 23613 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.324640 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.325549 21141 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:54.291541 23505 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.287268 23452 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.328603 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.293883 23488 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:54.297020 23451 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.291217 23498 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:54.309321 23547 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.291721 23461 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:54.338402 23584 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.478s	user 0.015s	sys 0.007s
W20260709 13:06:54.360107 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 64692.9us
I20260709 13:06:54.360984 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.363997 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1712) took 1340 ms. Trace:
I20260709 13:06:54.347321 23459 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:54.364421 21072 rpcz_store.cc:276] 0709 13:06:53.023357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.039454 (+1016097us) service_pool.cc:224] Handling call
0709 13:06:54.081285 (+ 41831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.081566 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:54.269986 (+188420us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.363641 (+ 93655us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:54.363696 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:54.363754 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.363817 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.363937 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":197253,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":2925,"thread_start_us":232054,"threads_started":2}
I20260709 13:06:54.333196 23613 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.371239 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.372376 21138 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:54.335218 23505 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:54.339730 23451 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:54.390281 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1425843429999532s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:06:54.339936 23452 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:54.391032 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:54.390527 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 101347us
I20260709 13:06:54.345147 23547 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.399698 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.384629 23621 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.401778 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:54.408735 21138 raft_consensus.cc:2368] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:54.395084 23625 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:54.418229 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1874) took 4259 ms. Trace:
W20260709 13:06:54.421319 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54133.5us
I20260709 13:06:54.426887 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.427697 21111 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.423499 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.425022 23526 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:54.421348 21143 rpcz_store.cc:276] 0709 13:06:50.154896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.155040 (+   144us) service_pool.cc:224] Handling call
0709 13:06:50.155515 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.312138 (+156623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:50.312192 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:50.361330 (+ 49138us) log.cc:844] Serialized 26 byte log entry
0709 13:06:52.322637 (+1961307us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b18003d88e0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:06:52.534184 (+211547us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:52.534222 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.534302 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.257409 (+1723107us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:54.413660 (+156251us) raft_consensus.cc:1685] finished
0709 13:06:54.413689 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.413856 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9204,"raft.queue_time_us":218487,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":26096,"spinlock_wait_cycles":79313536,"thread_start_us":354093,"threads_started":2,"wal-append.queue_time_us":2439081}
I20260709 13:06:54.436509 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:54.436724 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:54.439011 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 48071.5us
I20260709 13:06:54.419745 23453 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.439998 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.442684 21072 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:54.399758 23498 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.443981 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:54.412719 23621 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:54.468390 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:54.454844 23620 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:06:54.456947 23627 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:06:54.438951 23625 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:54.442337 23373 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:54.441473 23552 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:54.460440 23556 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:54.463491 23453 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:06:54.463954 21071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1700) took 3001 ms (client timeout 1656 ms). Trace:
I20260709 13:06:54.464556 21067 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:54.466351 23543 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.435791 23569 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.448480 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:54.444022 23617 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.138s	user 0.000s	sys 0.002s
I20260709 13:06:54.438290 23520 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:54.458410 23444 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:54.450466 23542 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.507s	user 0.005s	sys 0.006s
I20260709 13:06:54.476260 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:06:54.500701 23568 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:06:54.500993 23451 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.501612 23453 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:54.501986 23444 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:54.490404 23552 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:54.502358 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:54.503271 23457 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:54.503069 23552 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.503561 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:54.472007 23527 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:54.505144 23547 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.505360 23527 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:54.505646 23518 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:54.475152 21138 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:06:54.480801 23621 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:54.507592 23625 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:54.518272 23625 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:54.518443 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.518707 23625 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:54.519100 21138 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:54.520093 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.520622 21138 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:54.482412 23516 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.526319 23516 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.526937 21138 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:54.528180 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:54.528455 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.475862 23556 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:54.529301 23466 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:06:54.529037 21138 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:54.453125 23526 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:54.531267 23526 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:54.531711 23526 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:54.531488 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:54.532311 21794 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:54.494733 21213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2366) took 4283 ms (client timeout 2844 ms). Trace:
W20260709 13:06:54.535491 21213 rpcz_store.cc:269] 0709 13:06:50.210990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.211698 (+   708us) service_pool.cc:224] Handling call
0709 13:06:54.491795 (+4280097us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b5400103454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:54.494616 (+  2821us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2072931200}
I20260709 13:06:54.495088 21065 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:54.536501 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.537385 21797 raft_consensus.cc:1217] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Deduplicated request from leader. Original: 1.4->[3.5-5.7]   Dedup: 4.6->[5.7-5.7]
I20260709 13:06:54.537922 21797 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: [3.5-5.7]
W20260709 13:06:54.538599 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2532) took 2905 ms (client timeout 1922 ms). Trace:
W20260709 13:06:54.539004 21140 rpcz_store.cc:269] 0709 13:06:51.633009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.436961 (+803952us) service_pool.cc:224] Handling call
0709 13:06:54.538432 (+2101471us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540030a554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:54.538534 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3756658560}
I20260709 13:06:54.540063 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.540583 21140 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:54.540911 23531 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:54.541868 21141 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:54.542394 23531 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:54.542905 23531 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:54.544915 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 710) took 2954 ms. Trace:
I20260709 13:06:54.545287 21797 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:54.545325 21067 rpcz_store.cc:276] 0709 13:06:51.590731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.654679 (+2063948us) service_pool.cc:224] Handling call
0709 13:06:54.464361 (+809682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.489909 (+ 25548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.544735 (+ 54826us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54003cad50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:54.544857 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861295872}
I20260709 13:06:54.547585 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:54.548194 21067 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:06:54.549038 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:54.550185 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:54.551187 23520 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:54.551604 23520 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.551663 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.551977 23520 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:54.553038 21147 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:54.553373 21139 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:54.511811 23461 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.549572 21067 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:54.556200 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:54.556419 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:54.557240 21801 raft_consensus.cc:2410] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:06:54.497920 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:54.558480 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:54.559069 21072 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:06:54.561590 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:54.453020 23458 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.504166 23452 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:54.479724 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1709) took 1480 ms. Trace:
I20260709 13:06:54.481815 23629 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.478063 23613 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.524659 23513 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.530099 23556 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:54.525707 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:54.529158 21793 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:54.488060 21071 rpcz_store.cc:269] 0709 13:06:51.462207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.604098 (+141891us) service_pool.cc:224] Handling call
0709 13:06:54.463711 (+2859613us) spinlock_profiling.cc:243] Waited 93.1 ms on lock 0x7b54003cad54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:54.463873 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214231296}
I20260709 13:06:54.275624 21064 rpcz_store.cc:276] 0709 13:06:52.966781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.958364 (+991583us) service_pool.cc:224] Handling call
0709 13:06:53.958618 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.958921 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:54.174661 (+215740us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.271092 (+ 96431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:54.271145 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:54.271201 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.271445 (+   244us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.271570 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":299156,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"thread_start_us":268119,"threads_started":2}
I20260709 13:06:54.543712 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:54.544761 23516 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.573169 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:54.542824 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.562855 21068 rpcz_store.cc:276] 0709 13:06:52.999585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.039073 (+1039488us) service_pool.cc:224] Handling call
0709 13:06:54.119662 (+ 80589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.122510 (+  2848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.479520 (+357010us) spinlock_profiling.cc:243] Waited 72.7 ms on lock 0x7b54000f9e50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:54.479657 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167203200}
I20260709 13:06:54.574779 21067 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:54.562687 21799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 584) took 2221 ms. Trace:
I20260709 13:06:54.540469 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 704) took 3448 ms. Trace:
I20260709 13:06:54.564061 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:54.562543 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:54.565927 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.566859 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:54.583348 21071 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:06:54.583864 21219 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:54.585721 21072 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:54.564116 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46560 (request call id 7) took 4281 ms. Trace:
I20260709 13:06:54.565356 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:54.542939 23526 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:54.570849 21064 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:54.562150 23549 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.563103 23629 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.591640 23495 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:06:54.653142 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 87859.7us
I20260709 13:06:54.653313 23495 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:06:54.653759 23623 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
I20260709 13:06:54.653170 23453 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:54.654642 21036 leader_election.cc:400] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4.
I20260709 13:06:54.592418 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.585608 23616 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:54.657624 21146 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:06:54.657636 23616 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:54.585500 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:54.658717 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 73274.7us
I20260709 13:06:54.569234 23520 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:54.659075 23616 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.594239 21795 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:54.573051 23461 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:54.661734 23516 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:54.563350 23513 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:54.662212 23516 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.642947 23579 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.662609 23516 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:54.662868 23579 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.665015 23579 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:54.597072 23009 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:54.668951 23629 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.669828 21067 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:54.671947 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:54.672163 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:54.673027 21071 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:54.674312 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.596314 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1713) took 1237 ms. Trace:
W20260709 13:06:54.496585 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48237.5us
I20260709 13:06:54.577863 21066 rpcz_store.cc:276] 0709 13:06:51.091872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.182749 (+ 90877us) service_pool.cc:224] Handling call
0709 13:06:51.183044 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:51.183275 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:51.254286 (+ 71011us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.540006 (+3285720us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b180034f960. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287
0709 13:06:54.540102 (+    96us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:54.540152 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:54.540221 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.540297 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.540408 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7649,"prepare.queue_time_us":115948,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":35784,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":11380,"spinlock_wait_cycles":143907200,"thread_start_us":205702,"threads_started":3,"wal-append.queue_time_us":371440,"wal-append.run_cpu_time_us":1215,"wal-append.run_wall_time_us":1001251}
I20260709 13:06:54.675640 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:54.620285 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 291) took 2203 ms. Trace:
I20260709 13:06:54.620500 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 292) took 2188 ms. Trace:
I20260709 13:06:54.576155 21799 rpcz_store.cc:276] 0709 13:06:52.341598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.352921 (+ 11323us) service_pool.cc:224] Handling call
0709 13:06:52.354119 (+  1198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:52.530741 (+176622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.530776 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:52.531449 (+   673us) log.cc:844] Serialized 26 byte log entry
0709 13:06:52.949934 (+418485us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:52.949975 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.562247 (+1612272us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b54004934d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:54.562298 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.562348 (+    50us) raft_consensus.cc:1685] finished
0709 13:06:54.562362 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.562593 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":68130,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":3196,"spinlock_wait_cycles":89031040,"thread_start_us":458454,"threads_started":2,"wal-append.queue_time_us":468643,"wal-append.run_cpu_time_us":2033,"wal-append.run_wall_time_us":1161551}
I20260709 13:06:54.676815 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.562687 21792 raft_consensus.cc:2410] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:06:54.586405 23557 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
W20260709 13:06:54.623098 23618 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 13:06:54.623463 21147 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:54.563678 23556 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:54.624668 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 816871us
I20260709 13:06:54.641481 21215 rpcz_store.cc:276] 0709 13:06:50.316966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.345625 (+ 28659us) service_pool.cc:224] Handling call
0709 13:06:54.563897 (+4218272us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54003fa7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42516853 00007f7f474e460a 00007f7f40a6a7d4 00007f7f40a6a762 00007f7f40a6a691 00007f7f40a6a60c 00007f7f40a698c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:54.564008 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4651925248}
W20260709 13:06:54.641458 21219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2376) took 1638 ms (client timeout 1540 ms). Trace:
I20260709 13:06:54.550758 23488 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:54.650864 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 212940us
W20260709 13:06:54.651932 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.652381 21140 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
W20260709 13:06:54.689235 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.656031 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:54.660750 23520 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:54.660808 23526 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:54.661414 23549 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.651170 23479 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:06:54.676498 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:54.614852 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.562963 23505 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:54.592324 23586 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.728s	user 0.006s	sys 0.021s
I20260709 13:06:54.713169 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.675305 21065 rpcz_store.cc:276] 0709 13:06:53.358767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.126104 (+767337us) service_pool.cc:224] Handling call
0709 13:06:54.140490 (+ 14386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.536619 (+396129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.595987 (+ 59368us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54000f9954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:54.596235 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814924160}
I20260709 13:06:54.676653 21069 rpcz_store.cc:276] 0709 13:06:52.432424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.737764 (+1305340us) service_pool.cc:224] Handling call
0709 13:06:53.746667 (+  8903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.235388 (+488721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.620201 (+384813us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400214654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:54.620427 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058885376}
I20260709 13:06:54.680362 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.681676 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:54.684495 21219 rpcz_store.cc:269] 0709 13:06:53.003175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.003627 (+   452us) service_pool.cc:224] Handling call
0709 13:06:54.536360 (+1532733us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:54.641368 (+105008us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2555136}
I20260709 13:06:54.771487 21142 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
W20260709 13:06:54.772143 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 90571.7us
I20260709 13:06:54.772194 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.772580 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.772944 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.773339 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:54.773586 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.690133 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:54.685479 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:54.660857 23616 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:54.695494 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.695822 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.776262 21217 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:54.689781 23452 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:54.694749 23461 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:54.653791 23495 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:54.561760 23531 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:54.732295 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.797421 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:54.797709 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.707840 23579 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:54.676304 21063 rpcz_store.cc:276] 0709 13:06:52.416924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.736819 (+1319895us) service_pool.cc:224] Handling call
0709 13:06:53.847772 (+110953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.868067 (+ 20295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.619915 (+751848us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b54003ca854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:54.620196 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30118272}
I20260709 13:06:54.713439 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.803785 21220 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:54.765637 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56596.3us
I20260709 13:06:54.737001 21221 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:06:54.765669 21067 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:54.768008 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.808466 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.773497 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:54.774212 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 84159us
I20260709 13:06:54.689404 23557 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.809784 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:54.692366 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.810647 21146 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:06:54.811653 23557 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:54.812415 20969 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 240) took 12075 ms (client timeout 15000 ms). Trace:
I20260709 13:06:54.812867 23531 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.812609 23495 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.779798 23452 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.813808 23452 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:54.814214 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.684899 23576 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:06:54.688836 23532 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:54.775367 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.634145 23458 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:06:54.777925 21141 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:54.689551 23499 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7.
I20260709 13:06:54.669850 23009 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.780893 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:54.569253 23625 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:54.783535 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:54.791210 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:54.796252 21147 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:54.792131 21212 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:54.708237 20970 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 4, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 4 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.791680 21138 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:54.789830 21071 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:54.800350 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.689002 23555 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:06:54.709158 23307 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:54.844883 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64074.2us
I20260709 13:06:54.816031 21111 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.692260 23465 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:06:54.816593 21070 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:54.774716 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 43486.2us
I20260709 13:06:54.766201 23518 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.815181 23461 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:54.786183 23526 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:54.785051 23543 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.796474 23588 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:54.797500 23520 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:54.851768 23318 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:54.823694 21069 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:54.824146 23532 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.692396 23627 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:54.842064 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:54.939962 21799 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:54.715076 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.940948 21214 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:54.941905 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.942407 21214 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:54.848083 21220 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:54.944133 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.944845 21220 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:54.815801 21217 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:54.852370 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.833284 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:54.945655 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 112446us
I20260709 13:06:54.945909 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.946364 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:54.946741 21217 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:54.947140 23296 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:54.947570 23296 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.947913 23296 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:54.853664 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.949954 21794 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:54.854229 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:54.951557 23635 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:06:54.952051 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.952265 23452 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.851415 20970 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 4, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 4 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:54.953208 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.954229 21069 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:06:54.859470 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.808609 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:54.882359 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:54.863184 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 69163.6us
I20260709 13:06:54.891770 23526 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.896236 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:54.812824 20969 rpcz_store.cc:269] 0709 13:06:42.736693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.736905 (+   212us) service_pool.cc:224] Handling call
0709 13:06:54.812329 (+12075424us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:50.940059 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:50.940880 (+   821us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:50.940940 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:06:50.940984 (+    44us) tablet.cc:662] Decoding operations
0709 13:06:50.942879 (+  1895us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:50.942969 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:50.942995 (+    26us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:06:51.164641 (+221646us) tablet.cc:701] Row locks acquired
0709 13:06:51.164694 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:06:51.164865 (+   171us) write_op.cc:270] Start()
0709 13:06:51.165023 (+   158us) write_op.cc:276] Timestamp: P: 1783602411164795 usec, L: 0
0709 13:06:51.165045 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:06:51.169594 (+  4549us) log.cc:844] Serialized 9880 byte log entry
0709 13:06:51.655866 (+486272us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f3f4120ac 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b
0709 13:06:53.149769 (+1493903us) op_driver.cc:464] REPLICATION: finished
0709 13:06:53.534243 (+384474us) write_op.cc:301] APPLY: starting
0709 13:06:53.534370 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:53.535194 (+   824us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:53.535251 (+    57us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:53.570815 (+ 35564us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:53.673384 (+102569us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=25
0709 13:06:53.679359 (+  5975us) write_op.cc:312] APPLY: finished
0709 13:06:53.704005 (+ 24646us) log.cc:844] Serialized 170 byte log entry
0709 13:06:53.704228 (+   223us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:53.704426 (+   198us) write_op.cc:454] Released schema lock
0709 13:06:53.704505 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:53.705235 (+   730us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:53.705279 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1388733,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":384022,"apply.run_cpu_time_us":11425,"apply.run_wall_time_us":171004,"num_ops":25,"prepare.queue_time_us":50882,"prepare.run_cpu_time_us":31875,"prepare.run_wall_time_us":1471560,"raft.queue_time_us":156068,"raft.run_cpu_time_us":18888,"raft.run_wall_time_us":485414,"replication_time_us":1853455,"spinlock_wait_cycles":10504960,"thread_start_us":1703213,"threads_started":4,"wal-append.queue_time_us":1017651,"wal-append.run_cpu_time_us":14619,"wal-append.run_wall_time_us":2535833}]]}
I20260709 13:06:54.813869 23495 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:54.846187 23009 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:54.839833 23531 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:54.819332 23498 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:54.836822 23488 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.883095 23628 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:54.835572 23569 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:06:54.823073 23459 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:06:54.979584 23526 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:54.983516 23498 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.984129 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.984866 21212 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:54.985707 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.985814 23498 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:54.989104 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.990269 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.991480 23505 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:06:54.992056 23636 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:54.992838 23531 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:54.893278 23499 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
W20260709 13:06:54.994094 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 185651us
W20260709 13:06:54.996305 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 143982us
I20260709 13:06:54.997408 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:54.998392 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:54.999037 21217 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:54.954430 23452 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:55.001863 21214 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:06:55.002254 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:55.002904 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:55.003401 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:55.006302 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:55.006964 21212 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:06:55.008713 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.820516 23579 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:54.858429 23542 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:06:54.854647 23576 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:06:54.937089 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:54.857399 23307 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:54.903741 23520 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:54.952684 23543 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:54.777113 23485 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:06:54.942888 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:54.917270 23532 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.958312 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:54.957731 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.922777 23588 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:06:55.046742 21147 raft_consensus.cc:2376] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3. Re-sending same reply.
I20260709 13:06:54.957398 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:54.967348 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 107939us
I20260709 13:06:54.954249 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:54.953794 20970 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 3, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 3 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:54.930141 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83674.1us
I20260709 13:06:55.009881 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:54.897977 23518 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.012279 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:55.027586 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:55.029090 21214 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:55.032058 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:55.032335 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:55.043120 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:55.004609 23627 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:55.047881 21140 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:55.440650 23639 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.394s	user 0.003s	sys 0.000s
I20260709 13:06:55.690024 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 300) took 1359 ms. Trace:
W20260709 13:06:55.690979 21035 leader_election.cc:336] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.550s (SENT)
W20260709 13:06:55.643694 23485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.002s	sys 0.010s
W20260709 13:06:55.691732 23485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.003s	sys 0.011s
I20260709 13:06:55.690512 21068 rpcz_store.cc:276] 0709 13:06:54.330745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.575515 (+244770us) service_pool.cc:224] Handling call
0709 13:06:54.575798 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.642390 (+ 66592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:54.642420 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:54.684152 (+ 41732us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.839795 (+155643us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:54.839834 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.839906 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.439868 (+599962us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:55.689696 (+249828us) raft_consensus.cc:1685] finished
0709 13:06:55.689720 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.689935 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12906368,"thread_start_us":70910,"threads_started":1,"wal-append.queue_time_us":194442}
I20260709 13:06:55.694114 23373 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:55.694612 23373 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:55.644868 23616 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.860s	user 0.007s	sys 0.004s
I20260709 13:06:55.695580 23373 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.696698 23373 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:55.696101 20970 catalog_manager.cc:5697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 2 to 6, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 6 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.697770 20970 catalog_manager.cc:5697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 3, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:55.049664 23520 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:55.700187 21212 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:55.029229 21792 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:55.702235 23518 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:55.110488 23543 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:55.081502 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:55.048185 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:55.125849 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:55.006476 23479 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:55.011369 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:55.716553 21797 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.017s	sys 0.036s
W20260709 13:06:55.722636 21797 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.018s	sys 0.036s
I20260709 13:06:55.121644 21221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:55.723266 21221 raft_consensus.cc:606] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:55.033452 23458 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:55.041393 23401 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:55.121306 23579 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:55.129146 23562 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:55.086268 23318 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:55.033577 21798 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:06:55.095660 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 44758.5us
I20260709 13:06:55.054535 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:55.758639 21146 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:55.759418 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:06:55.707615 21187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.003s	sys 0.012s
W20260709 13:06:55.760706 21187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.003s	sys 0.012s
W20260709 13:06:55.761089 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 818237us
I20260709 13:06:55.708436 21215 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:55.002393 23517 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:55.709002 21801 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
W20260709 13:06:54.955173 23634 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:06:55.055805 21218 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:55.125566 21067 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:06:55.394726 23556 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.702s	user 0.008s	sys 0.002s
I20260709 13:06:55.710224 23549 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:55.120011 23625 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:55.033924 23461 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.013775 23630 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:55.843182 23452 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.843405 23549 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.844004 23452 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.844300 23549 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.846264 23452 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:55.373768 21145 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.005s
I20260709 13:06:55.846881 21145 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
W20260709 13:06:55.147243 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.126266 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:55.847149 23549 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:55.088362 23513 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.849153 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:55.849941 23513 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:55.850232 21069 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:55.712716 20970 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 4 to 5, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 5 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.044050 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:55.851042 21794 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:55.851270 21216 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:06:55.242240 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:55.852216 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:55.852742 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.072429 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:55.853629 23549 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:55.854118 21798 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:55.091198 23296 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.455251 23009 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.854314 23549 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:55.856189 23296 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:55.643989 21185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.005s	sys 0.011s
I20260709 13:06:55.856549 23009 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:55.856798 21185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.006s	sys 0.011s
W20260709 13:06:55.857108 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 851379us
W20260709 13:06:55.653497 21138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.005s	sys 0.007s
I20260709 13:06:55.721643 21217 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:55.073163 21213 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:55.715348 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:06:55.642889 23542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.005s	sys 0.008s
I20260709 13:06:55.745410 23543 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.746053 21140 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:06:55.258157 23569 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.747413 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:55.052867 23637 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:06:55.073733 23516 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.707921 23627 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.848387 21070 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:06:55.857872 21138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.006s	sys 0.007s
I20260709 13:06:55.857779 23549 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:55.861685 23516 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:55.090216 23531 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:55.858676 23542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.005s	sys 0.009s
W20260709 13:06:55.859985 21145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2545) took 1605 ms (client timeout 1513 ms). Trace:
I20260709 13:06:55.798553 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:06:55.862969 21145 rpcz_store.cc:269] 0709 13:06:54.254820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.273904 (+ 19084us) service_pool.cc:224] Handling call
0709 13:06:55.859832 (+1585928us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b5400279a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:55.859925 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1295313152}
I20260709 13:06:55.793208 23307 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:55.799422 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:55.746836 23401 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:55.864078 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 731) took 1375 ms. Trace:
I20260709 13:06:55.859357 23569 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:55.864866 21064 rpcz_store.cc:276] 0709 13:06:54.488951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.644726 (+155775us) service_pool.cc:224] Handling call
0709 13:06:54.645032 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.649587 (+  4555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:54.649619 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:54.650094 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.842155 (+192061us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:54.842198 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.842267 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.863875 (+1021608us) raft_consensus.cc:1685] finished
0709 13:06:55.863895 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.864025 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4183,"raft.queue_time_us":5981,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":186412,"threads_started":1,"wal-append.queue_time_us":248617}
I20260709 13:06:55.866484 23569 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:55.771121 23318 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:55.869978 23318 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:55.870961 23318 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:55.757494 23562 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:55.825201 23465 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:55.835703 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:55.839550 23616 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:06:55.841616 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 793459us
I20260709 13:06:55.842630 23552 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.842952 21067 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:55.843180 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.126662 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:55.851431 20970 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.857383 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:55.858088 21065 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:55.858191 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2384) took 1011 ms. Trace:
I20260709 13:06:55.864886 23444 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:55.871357 23401 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:55.859058 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 761) took 1146 ms. Trace:
I20260709 13:06:55.873164 23627 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:55.991268 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 281764us
W20260709 13:06:55.876664 23532 consensus_meta.cc:287] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.831s	user 0.007s	sys 0.003s
I20260709 13:06:55.728083 23373 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.843611 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:55.864172 21799 raft_consensus.cc:2376] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5. Re-sending same reply.
I20260709 13:06:55.747956 23479 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.012871 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:56.013583 21066 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:56.012879 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:55.903017 21146 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:55.909590 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 775) took 1117 ms. Trace:
I20260709 13:06:56.015650 21216 rpcz_store.cc:276] 0709 13:06:54.791813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.800108 (+  8295us) service_pool.cc:224] Handling call
0709 13:06:55.909420 (+1109312us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001257d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:55.909532 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308285312}
I20260709 13:06:55.727873 23458 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:55.952965 21063 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:55.913173 21215 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:56.022734 21220 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:56.023768 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2385) took 1176 ms. Trace:
I20260709 13:06:56.024396 21215 rpcz_store.cc:276] 0709 13:06:54.847380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.954036 (+106656us) service_pool.cc:224] Handling call
0709 13:06:56.023685 (+1069649us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:55.912384 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 195128us
I20260709 13:06:56.019812 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 697) took 1114 ms. Trace:
I20260709 13:06:56.027557 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.028224 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:55.910754 23465 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:55.339409 23526 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:55.988502 21213 rpcz_store.cc:276] 0709 13:06:54.712352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.771803 (+ 59451us) service_pool.cc:224] Handling call
0709 13:06:55.858911 (+1087108us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b540034b2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:55.859005 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593879808}
I20260709 13:06:55.915081 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2547) took 1659 ms. Trace:
I20260709 13:06:55.990124 21072 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
W20260709 13:06:55.992187 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.993063 20970 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 7, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 7 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:55.988375 21217 rpcz_store.cc:276] 0709 13:06:54.847138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.008337 (+161199us) service_pool.cc:224] Handling call
0709 13:06:55.858113 (+849776us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:55.994791 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 137107us
I20260709 13:06:55.995414 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1733) took 1051 ms. Trace:
I20260709 13:06:56.005582 23459 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:55.910431 23562 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:55.938333 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.008008 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 140464us
I20260709 13:06:56.015050 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:56.080204 21071 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:56.017516 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2567) took 1060 ms. Trace:
I20260709 13:06:56.080631 21146 rpcz_store.cc:276] 0709 13:06:54.957288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.958441 (+  1153us) service_pool.cc:224] Handling call
0709 13:06:55.054396 (+ 95955us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:56.017396 (+963000us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3792256}
I20260709 13:06:56.019335 21069 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:56.014379 21064 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:06:56.081981 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:55.988016 23444 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:55.914448 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 199184us
I20260709 13:06:56.025831 21797 rpcz_store.cc:276] 0709 13:06:54.905775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.905966 (+   191us) service_pool.cc:224] Handling call
0709 13:06:54.906267 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.906476 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:55.723073 (+816597us) log.cc:844] Serialized 14 byte log entry
0709 13:06:56.019392 (+296319us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.019447 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:56.019518 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.019619 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.019732 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":962560,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"thread_start_us":1097730,"threads_started":2}
I20260709 13:06:56.063536 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:56.069666 20970 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 2 to 5, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 5 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:06:55.990940 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:56.070096 21065 rpcz_store.cc:276] 0709 13:06:54.944426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.944556 (+   130us) service_pool.cc:224] Handling call
0709 13:06:54.955366 (+ 10810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.989330 (+1033964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.995131 (+  5801us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b540041d554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:55.995353 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075564928}
I20260709 13:06:56.071318 21143 rpcz_store.cc:276] 0709 13:06:54.255402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.439753 (+184351us) service_pool.cc:224] Handling call
0709 13:06:55.801492 (+1361739us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b540024a2a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42493dc0 00007f7f4251fe18 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6
0709 13:06:55.914827 (+113335us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54004236d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:55.915002 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2960292480}
I20260709 13:06:56.001330 23549 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:56.015053 23547 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:56.030481 23401 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:56.086333 21070 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:06:56.078100 21185 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:56.075670 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:56.069664 23642 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:56.087422 20970 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 2, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 2 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:56.093473 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 102594us
I20260709 13:06:56.089126 21066 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:06:56.083921 21138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:06:56.009101 23373 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.094982 21138 raft_consensus.cc:606] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:56.022614 23518 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:06:56.019609 23542 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:06:56.007398 23516 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:06:56.005191 23532 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.006728 23552 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.031361 23009 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:06:56.096767 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:56.098227 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:56.105134 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 1.09383e+06us
I20260709 13:06:56.106278 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 783) took 1294 ms. Trace:
I20260709 13:06:56.112875 21220 rpcz_store.cc:276] 0709 13:06:54.811891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.751565 (+939674us) service_pool.cc:224] Handling call
0709 13:06:56.022505 (+270940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.023255 (+   750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.106010 (+ 82755us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540015cf50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:56.106196 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":622406528}
I20260709 13:06:56.116943 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:56.117825 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:56.128233 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:56.098701 23616 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:06:56.133193 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 373890us
I20260709 13:06:56.090090 23556 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:06:56.100227 23642 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:06:56.104055 23555 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:56.095144 23520 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.110957 23499 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:56.115706 23547 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:06:56.157969 21038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.013s	sys 0.013s
I20260709 13:06:56.136440 23629 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:56.107828 23549 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.134729 23451 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:56.152277 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:56.136219 23628 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:56.158346 21038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.013s	sys 0.013s
I20260709 13:06:56.110356 23494 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:06:56.103700 23465 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:56.145774 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59887.2us
I20260709 13:06:56.146605 23568 raft_consensus.cc:2764] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:06:56.146149 23517 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 13:06:56.172657 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 44518.1us
W20260709 13:06:56.177425 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.162797 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 133595us
I20260709 13:06:56.161118 23009 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:56.125972 23648 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:06:56.156926 23499 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:56.155719 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52361.2us
I20260709 13:06:56.175616 23629 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:56.183540 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 77221us
I20260709 13:06:56.098858 23458 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.168437 23451 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.189121 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:56.202095 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:56.171239 23628 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.153613 23649 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:06:56.179621 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:56.208977 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:56.209831 23296 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:56.210618 23543 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:56.216647 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:56.206589 21110 leader_election.cc:336] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.671s (SENT)
I20260709 13:06:56.190819 23660 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:56.217747 23660 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.218204 23556 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:56.207546 21138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:06:56.219187 21138 raft_consensus.cc:606] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:06:56.219457 23661 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:06:56.191946 23616 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:56.226431 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.202993 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:56.218225 23660 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:56.197825 23499 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:56.205823 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:56.218336 23459 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:56.208856 23628 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:56.242904 23459 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.236984 23494 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.203155 23465 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:56.202471 23479 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:56.200829 23009 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:56.209878 23667 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:06:56.241415 21184 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:56.228368 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:56.242951 23579 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:56.240607 23543 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:56.204507 23451 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:56.204841 23627 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:56.231423 23662 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:06:56.208454 23670 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:06:56.265537 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 187514us
I20260709 13:06:56.308322 21065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
mode: GRACEFUL
new_leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
 from {username='slave'} at 127.0.0.1:58210
I20260709 13:06:56.308560 21185 leader_election.cc:400] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6.
I20260709 13:06:56.308959 21065 raft_consensus.cc:606] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Received request to transfer leadership to TS b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:56.309326 21065 raft_consensus.cc:614] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:56.309479 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:56.268743 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.208485 23568 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:06:56.237694 23660 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.210937 23461 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
W20260709 13:06:56.314106 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 85823.5us
I20260709 13:06:56.320775 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:56.320957 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:56.315788 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 74448.3us
I20260709 13:06:56.241462 23616 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.322252 21185 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:56.199630 23642 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.324085 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:56.308014 23659 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:06:56.335511 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:56.310652 23373 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:56.342649 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:56.346082 21071 raft_consensus.cc:2410] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:56.343874 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.314050 23526 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:56.317701 23652 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:56.320478 23479 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:56.314716 23532 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:56.318267 23579 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:56.357517 23658 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:06:56.323300 23669 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:56.299652 23494 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.336638 23670 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:56.337061 23667 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:56.341395 23627 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:56.343917 23556 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:56.381944 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.342756 23650 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:56.350790 23307 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:56.390257 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 54830.8us
I20260709 13:06:56.393404 21795 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:06:56.393961 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:56.394491 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:56.394948 21796 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:56.341645 23485 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:06:56.358016 23660 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:56.244270 23547 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:06:56.368608 23670 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.352206 23588 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:56.387108 23556 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.407439 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:56.387957 23627 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:56.354844 23479 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:56.344645 23629 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.430101 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:56.412112 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 90016.6us
I20260709 13:06:56.392352 23532 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:56.417229 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:56.391641 23307 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:06:56.417629 23318 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.000s
I20260709 13:06:56.413866 23556 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:56.456393 23579 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.438899 23401 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.459769 23401 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.461870 23401 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:56.418340 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.418447 23444 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.396123 23650 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.420002 23495 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:56.422616 23616 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:56.428435 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:56.426738 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:56.366989 23520 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:56.435318 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:56.435467 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.463271 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.435951 23459 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:56.407151 23552 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:56.438584 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:56.449474 23562 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.425047 23627 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:56.358398 23543 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.399320 23009 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:06:56.445055 23307 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:56.460222 21800 raft_consensus.cc:2410] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:56.466702 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.435093 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.433336 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:56.488261 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55067.2us
I20260709 13:06:56.464289 23495 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:56.489494 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:56.408177 23670 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:56.477244 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 380505us
I20260709 13:06:56.480041 21795 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:56.454074 23588 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:06:56.465591 21143 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:06:56.464123 23444 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:56.465301 23650 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:56.500065 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:56.506942 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:56.508126 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:06:56.511669 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 301470us
I20260709 13:06:56.512436 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.489777 23401 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:56.519483 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.517776 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:56.496673 23654 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.475816 23451 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.518786 21216 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:56.520985 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.522466 21215 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:06:56.523406 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.513813 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:56.424515 23479 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.495641 23494 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:56.496457 23485 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:56.562357 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:56.563017 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.552872 21215 pending_rounds.cc:85] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 3
I20260709 13:06:56.550417 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:56.566449 21215 pending_rounds.cc:107] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:06:56.553565 21214 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:06:56.567418 21215 raft_consensus.cc:2889] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:56.511078 23542 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:06:56.504026 23616 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:56.557562 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:56.569489 21140 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:56.555761 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.568512 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.574180 21214 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:56.555661 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.567598 21185 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:56.573607 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.577509 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:56.507858 23296 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:56.577548 21140 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:56.578359 21147 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:06:56.539944 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 334184us
I20260709 13:06:56.579783 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.581995 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:56.582633 21146 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:56.514096 23452 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:56.519765 23518 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:56.580924 21215 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:06:56.582643 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:56.584558 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.590746 21138 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:56.506484 23552 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:06:56.585311 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.584265 21141 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:56.579548 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:56.581647 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.564584 23650 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:56.562988 21069 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:06:56.584177 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.609059 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.592154 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:56.513296 23516 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:06:56.572894 23532 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:56.615127 21215 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:56.617774 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.620596 21217 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:56.622329 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.623131 21801 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:06:56.623991 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:56.624555 21801 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:56.590304 23452 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:56.563213 23568 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
W20260709 13:06:56.561743 23630 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.718s	user 0.011s	sys 0.000s
I20260709 13:06:56.616760 23648 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:56.655625 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:56.659890 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 80416us
I20260709 13:06:56.525849 23495 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.669685 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.614637 23549 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:56.590150 23518 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:56.627579 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.628011 23452 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:56.668609 23461 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:56.673540 21795 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:06:56.737789 21069 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:56.738147 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.666271 23526 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:56.668375 23630 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:56.754294 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 778) took 1943 ms. Trace:
I20260709 13:06:56.743528 21217 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:06:56.669762 23569 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:56.668282 23542 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.735047 21072 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:56.748960 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:56.757390 21036 leader_election.cc:336] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.540s (SENT)
I20260709 13:06:56.758534 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:56.667582 23373 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:06:56.563715 23654 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:56.736362 23296 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.765724 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:56.766906 23628 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.001s
W20260709 13:06:56.767688 23680 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.510s	user 0.002s	sys 0.000s
W20260709 13:06:56.768957 23684 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:06:56.734999 23552 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:56.784932 21801 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:56.868754 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:56.754743 21219 rpcz_store.cc:276] 0709 13:06:54.810515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.276884 (+466369us) service_pool.cc:224] Handling call
0709 13:06:55.277245 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.063704 (+786459us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:56.150630 (+ 86926us) log.cc:844] Serialized 14 byte log entry
0709 13:06:56.318384 (+167754us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.318432 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:56.318488 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.754066 (+435578us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b540034b7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:56.754118 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.754237 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54774,"prepare.queue_time_us":144060,"prepare.run_cpu_time_us":290,"prepare.run_wall_time_us":536,"raft.queue_time_us":59731,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":4318,"spinlock_wait_cycles":1808077568,"thread_start_us":190661,"threads_started":2,"wal-append.queue_time_us":498724}
W20260709 13:06:56.803505 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.824569 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.748076 23518 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:56.859184 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:56.880921 21072 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:56.873711 21795 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:56.672405 23499 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:56.875360 21798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.016s
W20260709 13:06:56.884755 21798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.000s	sys 0.017s
I20260709 13:06:56.748732 23675 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:56.760771 21793 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:06:56.819557 23616 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:56.889043 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.929131 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.930297 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:06:56.931022 21793 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:56.932838 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.933270 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.933416 21217 raft_consensus.cc:2376] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4. Re-sending same reply.
I20260709 13:06:56.935334 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.936007 21212 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:56.940384 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.940636 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.942076 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.892611 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:56.943804 21215 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:56.942765 21793 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:56.950243 21139 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:56.953500 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.960611 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.883772 23552 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:56.962194 21765 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.006s	sys 0.033s
W20260709 13:06:56.979831 21765 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.006s	sys 0.033s
W20260709 13:06:56.980357 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 542788us
W20260709 13:06:56.985277 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.773015 23542 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.774248 23569 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:56.862612 23650 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:56.761461 23461 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:56.971578 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 389423us
I20260709 13:06:56.973140 21801 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:57.031136 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:57.032094 21801 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:56.774559 23526 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:56.885236 23625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.007s	sys 0.012s
I20260709 13:06:56.965061 21212 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:06:56.978418 23451 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:56.971506 23679 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:06:56.898447 23668 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:57.014760 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:56.899770 23588 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:56.979831 23675 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:56.945525 23669 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:57.032799 21143 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.000s
I20260709 13:06:57.117117 21143 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:06:56.868054 23628 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:57.045099 21801 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:57.049181 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:57.128225 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79104.6us
I20260709 13:06:56.984615 23629 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:56.988092 23706 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:57.044696 23625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.008s	sys 0.013s
I20260709 13:06:57.132956 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:56.971041 23616 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:57.157068 23697 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:57.161064 23451 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.167649 23513 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.209033 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:57.210474 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:57.210827 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:57.211167 21212 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:06:57.195044 23693 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260709 13:06:57.228596 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.197799 23690 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:06:57.154577 23629 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:57.131594 23588 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:57.268538 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.301860 23678 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:57.302692 23494 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:57.315503 21799 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:57.327678 21212 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:57.394795 21212 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:57.395570 21801 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:57.395920 21212 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:57.371130 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.146412 23669 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:57.226850 23687 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:57.240144 23499 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:57.445731 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.178593 23552 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:57.208323 23451 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:57.222232 23513 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:57.419073 23516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.001s	sys 0.023s
I20260709 13:06:57.168489 23706 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:57.533986 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 757) took 1312 ms. Trace:
I20260709 13:06:57.534531 21066 rpcz_store.cc:276] 0709 13:06:56.221725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.222889 (+  1164us) service_pool.cc:224] Handling call
0709 13:06:56.223267 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.242738 (+ 19471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.242782 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:56.243339 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.525157 (+281818us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:56.525446 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.525535 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:57.533768 (+1008233us) raft_consensus.cc:1685] finished
0709 13:06:57.533789 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.533927 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15250,"raft.queue_time_us":56994,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":18131,"thread_start_us":281445,"threads_started":1,"wal-append.queue_time_us":645795}
W20260709 13:06:57.526718 23516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.002s	sys 0.024s
I20260709 13:06:57.205631 23650 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:57.543045 23579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.033s	sys 0.017s
I20260709 13:06:57.542971 23552 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:57.539333 23531 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:57.546708 23542 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:57.554809 21793 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:57.555629 23687 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:57.606472 23687 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:57.608582 23627 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.117s	user 0.010s	sys 0.000s
I20260709 13:06:57.608826 23687 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:57.296917 23675 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:57.533130 23683 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
W20260709 13:06:57.576673 23579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.987s	user 0.034s	sys 0.018s
W20260709 13:06:57.605957 23556 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.150s	user 0.011s	sys 0.000s
I20260709 13:06:57.574102 23569 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:57.586009 23461 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:57.609052 23627 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:57.650243 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.001s	sys 0.001s
W20260709 13:06:57.651202 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.002s	sys 0.001s
W20260709 13:06:57.651826 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 894475us
W20260709 13:06:57.652534 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.563s (SENT) [suppressed 36 similar messages]
W20260709 13:06:57.627097 23518 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.724s	user 0.005s	sys 0.005s
I20260709 13:06:57.624948 23628 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:57.624194 23706 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:57.675801 21796 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:57.601800 23531 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:57.683368 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.008s	sys 0.020s
W20260709 13:06:57.689041 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.012s	sys 0.020s
W20260709 13:06:57.689435 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 560392us
W20260709 13:06:57.676913 23708 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:06:57.691735 23669 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:57.695659 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:57.695037 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 792) took 1176 ms. Trace:
I20260709 13:06:57.699492 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:57.631214 23556 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:57.697247 21218 rpcz_store.cc:276] 0709 13:06:56.518563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.518687 (+   124us) service_pool.cc:224] Handling call
0709 13:06:56.561991 (+ 43304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.582681 (+ 20690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.582713 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:56.592206 (+  9493us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.889734 (+297528us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:56.889794 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.889899 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:57.694822 (+804923us) raft_consensus.cc:1685] finished
0709 13:06:57.694840 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.694976 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12275,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":11132,"thread_start_us":212381,"threads_started":1,"wal-append.queue_time_us":502565}
I20260709 13:06:57.709143 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:57.710143 21793 raft_consensus.cc:2393] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:06:57.715622 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:57.720525 23695 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:57.727087 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:57.727523 23707 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:06:57.727918 23655 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:57.728190 21144 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.728386 23655 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:57.729279 23655 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:57.730417 23655 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:57.731982 23513 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:57.732700 23627 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:57.732535 23628 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:57.749065 23652 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:57.749897 23652 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:57.751523 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 611) took 1290 ms. Trace:
I20260709 13:06:57.751894 23451 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:57.752161 21795 rpcz_store.cc:276] 0709 13:06:56.460774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.944442 (+483668us) service_pool.cc:224] Handling call
0709 13:06:56.944712 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.987818 (+ 43106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.987857 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:57.014013 (+ 26156us) log.cc:844] Serialized 26 byte log entry
0709 13:06:57.612723 (+598710us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:57.612785 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.612865 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:57.751155 (+138290us) raft_consensus.cc:1685] finished
0709 13:06:57.751187 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.751445 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42695,"raft.queue_time_us":46421,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":7541,"thread_start_us":493788,"threads_started":1,"wal-append.queue_time_us":715485}
I20260709 13:06:57.736323 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:06:57.754961 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:57.667428 23518 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:57.702231 23683 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:58.016667 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.813364 23451 raft_consensus.cc:993] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: : Instructing follower 46c1676121b64b7d9bcf2227cc639dd3 to start an election
I20260709 13:06:57.980026 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:57.793639 23706 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:06:58.026329 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.899085 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:58.151206 23452 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.378s	user 0.009s	sys 0.003s
W20260709 13:06:58.152171 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.756333 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:58.152284 21065 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:06:57.830340 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.166811 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.166882 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:58.167652 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.891870 21072 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:57.889127 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:57.780653 21215 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:57.779786 21144 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:06:57.735340 23629 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:57.945494 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:57.891140 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:58.161397 21185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.014s	sys 0.009s
W20260709 13:06:58.174188 21185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.014s	sys 0.009s
I20260709 13:06:58.173949 21068 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:57.707384 23687 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:57.890286 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:57.816259 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 112766us
W20260709 13:06:58.172851 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 416558us
W20260709 13:06:58.180646 21110 connection.cc:256] RPC call timeout handler was delayed by -1.2342446959999052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2141s for a response
W20260709 13:06:58.149484 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 394552us
I20260709 13:06:58.171490 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:58.181826 21219 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:58.175132 21214 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:58.175380 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:57.888504 23725 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
W20260709 13:06:58.103212 23670 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.597s	user 0.009s	sys 0.004s
W20260709 13:06:57.812616 23682 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:06:57.735692 23461 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.170610 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:06:57.810853 23694 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
W20260709 13:06:57.811389 23720 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 13:06:57.896713 23307 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.399s	user 0.006s	sys 0.004s
W20260709 13:06:57.893415 23701 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 13:06:57.735546 23588 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:57.981338 23444 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.483s	user 0.010s	sys 0.000s
I20260709 13:06:58.151611 23655 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:58.192571 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:57.749156 23728 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.158847 23479 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:58.220587 21140 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:58.180491 21221 raft_consensus.cc:2376] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7. Re-sending same reply.
I20260709 13:06:57.780503 23569 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:57.891860 23531 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:57.818130 23549 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:57.878103 23458 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:57.754283 23616 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:57.779687 23628 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:57.895584 23663 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:57.798884 23652 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:58.169839 23732 raft_consensus.cc:1081] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Signalling peer 46c1676121b64b7d9bcf2227cc639dd3 to start an election
I20260709 13:06:58.185056 23695 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.391489 23628 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.346514 23569 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:58.395370 23663 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:58.397369 23616 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:58.300964 23655 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:58.409493 23710 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.293s	user 0.000s	sys 0.010s
I20260709 13:06:58.398911 23719 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:58.410869 23719 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:06:58.411597 23655 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:58.412091 23719 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:58.413841 23719 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:58.279976 23670 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:58.421377 23700 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.850s	user 0.002s	sys 0.006s
I20260709 13:06:58.170352 23683 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:58.423926 23713 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.852s	user 0.000s	sys 0.008s
I20260709 13:06:58.384473 23731 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.424798 23731 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:58.426569 23737 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.609s	user 0.006s	sys 0.003s
I20260709 13:06:58.391901 23729 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:58.286952 23649 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:58.280861 23650 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 13:06:58.407689 23663 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:58.312479 23736 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.433203 23736 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:58.436684 23736 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:58.438499 23670 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:58.428057 23729 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:58.313871 23728 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:58.427124 23731 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:58.456192 23728 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:58.458745 23729 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:58.184064 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:06:58.495326 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.497635 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 313476us
I20260709 13:06:58.499600 21066 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:58.184417 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:58.508428 21068 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:58.193943 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:58.514318 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 755) took 2339 ms. Trace:
I20260709 13:06:58.515836 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:58.517210 21072 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
W20260709 13:06:58.193581 21110 connection.cc:256] RPC call timeout handler was delayed by -1.2171446959999912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
I20260709 13:06:58.518594 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:58.519088 21110 connection.cc:256] RPC call timeout handler was delayed by -1.1352446959999725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2251s for a response
I20260709 13:06:58.514875 21070 rpcz_store.cc:276] 0709 13:06:56.174751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.174916 (+   165us) service_pool.cc:224] Handling call
0709 13:06:56.175375 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.191852 (+ 16477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.191899 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:56.207763 (+ 15864us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.316121 (+108358us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:56.316160 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.316227 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:58.513658 (+2197431us) raft_consensus.cc:1685] finished
0709 13:06:58.513705 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:58.514083 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17094,"raft.queue_time_us":17881,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":11212,"thread_start_us":78345,"threads_started":1,"wal-append.queue_time_us":1441428}
W20260709 13:06:58.520223 21110 leader_election.cc:336] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 2.046s (SENT)
I20260709 13:06:58.521159 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:58.521116 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:58.522747 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:58.528159 21070 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:58.526396 21072 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
W20260709 13:06:58.528620 21185 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.748s (SENT)
I20260709 13:06:58.532881 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:58.533267 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:58.512825 21066 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
W20260709 13:06:58.541460 21185 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.749s (SENT)
I20260709 13:06:58.542172 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:58.534968 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:58.544353 21072 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:58.300338 21213 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:06:58.281379 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:58.548622 21792 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:06:58.549124 21214 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:58.547034 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:58.288399 21217 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:58.552000 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:58.552767 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:58.554358 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:58.280822 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:58.547524 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 610) took 2222 ms. Trace:
I20260709 13:06:58.556521 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:58.555572 21798 rpcz_store.cc:276] 0709 13:06:56.324976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.326099 (+  1123us) service_pool.cc:224] Handling call
0709 13:06:56.326449 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.338915 (+ 12466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.338962 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:56.339883 (+   921us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.963851 (+623968us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:56.963890 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.963954 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:58.420596 (+1456642us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:58.546656 (+126060us) raft_consensus.cc:1685] finished
0709 13:06:58.546699 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:58.547138 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9454,"raft.queue_time_us":103252,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":3864,"spinlock_wait_cycles":12927872,"thread_start_us":595367,"threads_started":1,"wal-append.queue_time_us":802478}
I20260709 13:06:58.552858 21221 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:06:58.557386 21219 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:06:58.558449 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:06:58.546993 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 372784us
I20260709 13:06:58.560318 21068 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:06:58.189944 23451 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:58.387749 23549 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:58.312961 23579 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.432514 23654 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:58.555116 21217 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 2.
W20260709 13:06:58.616204 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 434674us
I20260709 13:06:58.583668 23757 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:58.588624 21214 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:58.588191 23660 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:58.596784 23749 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:58.582223 23748 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:58.404915 23628 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:58.623780 23748 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:58.626766 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 249) took 1668 ms. Trace:
I20260709 13:06:58.606406 23734 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:58.627863 23734 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:58.629125 23569 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:58.422904 23695 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:58.607692 23655 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:06:58.286890 23307 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:58.607842 23735 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.430182 23649 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:58.173108 23452 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:58.621362 23451 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:58.622663 23444 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:58.629019 23749 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:58.284814 23461 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:58.618773 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 74475.7us
I20260709 13:06:58.729151 21070 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:06:58.776990 23656 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.629788 23729 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:58.676666 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.887106 23656 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:58.887656 23729 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:58.888247 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.630399 21072 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:06:58.778155 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.890405 23729 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:06:58.890710 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.891515 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.630541 21214 rpcz_store.cc:276] 0709 13:06:56.958538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.622496 (+1663958us) service_pool.cc:224] Handling call
0709 13:06:58.623000 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.626235 (+  3235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.626272 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:58.626371 (+    99us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:58.626455 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.626529 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:58.626667 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:58.781229 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:06:58.775557 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 180075us
I20260709 13:06:58.898000 23307 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:58.895965 21217 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:58.901229 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.902405 21066 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:58.786603 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:58.630390 21219 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:58.906733 21065 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
W20260709 13:06:58.687254 23576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.935s	user 0.011s	sys 0.023s
W20260709 13:06:58.909485 23576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.259s	user 0.016s	sys 0.028s
I20260709 13:06:58.909823 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 254) took 1194 ms. Trace:
I20260709 13:06:58.911479 23656 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:58.741425 23666 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.908282 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:58.635893 21068 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:58.912858 23649 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:58.909035 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:58.640185 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:58.933014 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.909075 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 246) took 1951 ms. Trace:
I20260709 13:06:58.859891 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:58.935953 21219 rpcz_store.cc:276] 0709 13:06:56.957694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.588384 (+1630690us) service_pool.cc:224] Handling call
0709 13:06:58.604068 (+ 15684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.908828 (+304760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.908999 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:58.630295 23728 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:58.898032 23695 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:58.942653 23695 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.943756 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:58.698403 23683 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:58.913221 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:06:58.783613 21213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 788) took 2269 ms (client timeout 1542 ms). Trace:
I20260709 13:06:58.784925 21792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 607) took 2460 ms. Trace:
I20260709 13:06:58.612908 23579 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:58.905723 23452 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:58.908082 23461 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:58.899151 21794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 264) took 2478 ms (client timeout 1905 ms). Trace:
I20260709 13:06:58.630627 23748 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:58.943598 23666 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:58.965482 23749 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:58.858165 23555 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:59.020109 23683 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:59.020886 23683 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.965543 23655 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.021476 23683 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:59.022228 21064 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:58.944854 23695 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:06:58.996474 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 332279us
W20260709 13:06:59.024966 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 165276us
I20260709 13:06:58.911608 21221 rpcz_store.cc:276] 0709 13:06:57.715206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.558070 (+842864us) service_pool.cc:224] Handling call
0709 13:06:58.786417 (+228347us) spinlock_profiling.cc:243] Waited 964 us on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:58.909743 (+123326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2232448}
I20260709 13:06:59.017225 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:58.778084 23756 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.694623 23734 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:06:59.022979 21213 rpcz_store.cc:269] 0709 13:06:56.513496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.513639 (+   143us) service_pool.cc:224] Handling call
0709 13:06:58.783071 (+2269432us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54001252d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:58.783333 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3967702272}
I20260709 13:06:59.023247 21792 rpcz_store.cc:276] 0709 13:06:56.324013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.334742 (+ 10729us) service_pool.cc:224] Handling call
0709 13:06:56.392218 (+ 57476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.549344 (+2157126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.784560 (+235216us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b54005e01d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:58.784857 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4946845568}
I20260709 13:06:58.964489 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:06:58.987730 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 347767us
I20260709 13:06:58.932149 23660 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:58.965404 23735 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.030742 23461 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:59.032612 23649 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:59.035022 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 417699us
I20260709 13:06:59.019714 23555 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:59.043730 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:59.048715 21070 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:58.965188 23451 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.071730 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:59.072659 21141 raft_consensus.cc:2368] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8.
I20260709 13:06:59.073316 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:59.077636 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.023453 21794 rpcz_store.cc:269] 0709 13:06:56.420557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.589451 (+168894us) service_pool.cc:224] Handling call
0709 13:06:58.780667 (+2191216us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7e9c 00007f7f3f3f7a5b 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:06:58.898985 (+118318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50065280}
I20260709 13:06:59.055466 23461 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.063588 23657 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.087157 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:59.090425 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50095us
I20260709 13:06:59.081029 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:59.067452 23649 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.098915 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.106199 21145 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:59.107755 23461 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.110859 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.110910 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.121030 21138 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:59.122462 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 244) took 1427 ms. Trace:
W20260709 13:06:59.113021 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46972.2us
W20260709 13:06:59.125205 21185 leader_election.cc:336] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 2.117s (SENT)
I20260709 13:06:59.125798 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:59.055506 23756 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.128032 23695 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.127390 23555 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:59.135856 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 94678.2us
I20260709 13:06:59.136513 23555 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:59.136806 23649 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.137202 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:59.138072 23761 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:59.138586 23761 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:59.138867 23479 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.827s	user 0.013s	sys 0.000s
I20260709 13:06:59.139027 23761 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:59.140460 23728 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:59.132787 23451 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.145535 21140 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:59.101470 23657 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.122872 21146 rpcz_store.cc:276] 0709 13:06:57.693312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.696351 (+  3039us) service_pool.cc:224] Handling call
0709 13:06:57.718147 (+ 21796us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:57.718533 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:57.718562 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:57.743006 (+ 24444us) log.cc:844] Serialized 26 byte log entry
0709 13:06:58.185990 (+442984us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:58.585025 (+399035us) log.cc:844] Serialized 14 byte log entry
0709 13:06:59.096430 (+511405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.101289 (+  4859us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.111934 (+ 10645us) raft_consensus.cc:1685] finished
0709 13:06:59.111950 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.120854 (+  8904us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":398786,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":42455,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":360,"thread_start_us":1167981,"threads_started":3,"wal-append.queue_time_us":519610}
I20260709 13:06:59.151255 21141 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:59.152822 23717 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.153519 23717 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.150046 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:59.158757 23763 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:59.158802 21216 raft_consensus.cc:2410] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:06:59.165477 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 121228us
W20260709 13:06:59.172227 23726 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.753s	user 0.004s	sys 0.000s
I20260709 13:06:59.151016 23655 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:59.157657 23666 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.156644 23579 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.187152 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:06:59.189375 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64220.6us
I20260709 13:06:59.192530 23767 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:59.192699 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.193094 23767 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.193519 23767 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:59.193634 21146 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:59.181602 23461 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:59.195439 23765 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:59.196218 23765 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.195106 21146 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:59.196763 23765 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.197872 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.200242 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.176126 23763 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:59.208768 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 262492us
I20260709 13:06:59.211501 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.215485 21141 pending_rounds.cc:85] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 5
I20260709 13:06:59.215723 23766 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:59.216471 23766 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.216903 23657 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.217329 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.217890 23495 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:59.220147 21146 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9.
W20260709 13:06:59.220454 21796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 263) took 2800 ms (client timeout 1616 ms). Trace:
I20260709 13:06:59.221240 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:06:59.215884 21141 pending_rounds.cc:107] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:06:59.221942 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:59.222240 21141 raft_consensus.cc:2889] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:59.224030 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 108440us
I20260709 13:06:59.222205 21146 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:59.225893 23734 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:59.222865 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.225272 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:59.220949 21796 rpcz_store.cc:269] 0709 13:06:56.420274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.462999 (+ 42725us) service_pool.cc:224] Handling call
0709 13:06:59.219992 (+2756993us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400070154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:59.220229 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2778610560}
I20260709 13:06:59.231927 21146 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:59.245776 23556 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.255769 23765 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.259687 21063 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:06:59.261332 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1741) took 1341 ms. Trace:
I20260709 13:06:59.260535 23763 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.261721 21063 rpcz_store.cc:276] 0709 13:06:57.919831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.979616 (+ 59785us) service_pool.cc:224] Handling call
0709 13:06:59.261127 (+1281511us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003a2fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:59.261255 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2480097536}
I20260709 13:06:59.263456 23756 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.264184 23452 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:59.264636 23452 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.264971 23452 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:59.269641 23628 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.000s
I20260709 13:06:59.259063 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 247) took 2301 ms. Trace:
I20260709 13:06:59.259713 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:06:59.273202 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1744) took 1273 ms. Trace:
W20260709 13:06:59.275533 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52865.1us
I20260709 13:06:59.219164 23766 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.283547 21065 rpcz_store.cc:276] 0709 13:06:57.999385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.166664 (+167279us) service_pool.cc:224] Handling call
0709 13:06:59.272888 (+1106224us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54000fa854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:59.273099 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1698236672}
I20260709 13:06:59.289467 23768 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:06:59.275962 23759 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:59.290562 23768 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.297605 21147 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:59.297716 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:06:59.277318 23458 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.687s	user 0.014s	sys 0.003s
I20260709 13:06:59.284536 23628 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.278162 21217 rpcz_store.cc:276] 0709 13:06:56.957992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.615558 (+1657566us) service_pool.cc:224] Handling call
0709 13:06:58.616138 (+   580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.904796 (+288658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.258638 (+353842us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400102f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:06:59.258927 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642942848}
I20260709 13:06:59.299181 21147 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:59.301309 23756 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:59.324640 23756 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:59.325721 23756 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.326119 23452 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:59.302683 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58829.1us
W20260709 13:06:59.320082 21139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2579) took 2756 ms (client timeout 2090 ms). Trace:
I20260709 13:06:59.321606 21144 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:59.322945 23373 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:59.324693 21147 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:06:59.318120 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.328604 21139 rpcz_store.cc:269] 0709 13:06:56.563759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.563880 (+   121us) service_pool.cc:224] Handling call
0709 13:06:59.319595 (+2755715us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400424ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:59.319863 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764913024}
W20260709 13:06:59.329305 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 192229us
I20260709 13:06:59.328423 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:06:59.303036 21797 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:06:59.302744 23458 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:59.311725 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:59.337011 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:59.271224 23526 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:06:59.342653 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.345427 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 49306.2us
I20260709 13:06:59.327798 23717 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.310810 23734 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.304139 23768 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.333240 23666 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.307567 23628 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.353298 21147 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:59.356854 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.357355 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:59.358060 21219 raft_consensus.cc:2376] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4. Re-sending same reply.
I20260709 13:06:59.358278 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.359558 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:59.359866 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:59.360651 21217 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:59.361264 23735 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.361459 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.362432 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:59.363018 21792 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:59.363425 21221 raft_consensus.cc:2410] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:06:59.364120 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:59.364253 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.365006 21221 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:59.364852 23373 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.365374 23499 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:59.365505 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:59.365788 23499 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:06:59.365844 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:59.366176 21139 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:59.366592 21221 raft_consensus.cc:2376] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7. Re-sending same reply.
I20260709 13:06:59.366802 21798 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:59.367626 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.367789 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:59.369102 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.369141 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.369743 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:59.358918 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.371783 21219 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:59.371946 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:59.353416 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:59.373636 21793 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 1 LEARNER]: Advancing to term 3
I20260709 13:06:59.373240 23666 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.375567 21792 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:59.370908 23760 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:59.359002 21214 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
W20260709 13:06:59.378765 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 67357.7us
I20260709 13:06:59.354945 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:59.380792 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:59.371186 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:06:59.385298 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44653.8us
I20260709 13:06:59.387081 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:06:59.349481 23660 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:06:59.392252 21221 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:59.393810 21217 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:06:59.353413 23616 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:59.406577 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.369282 23562 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.407878 23768 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.415669 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:59.423629 23660 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.372798 23734 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.425715 23452 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.409092 23552 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:59.427238 23373 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.426591 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.416561 23769 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:59.415743 23735 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:06:59.429265 21799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 705) took 3402 ms (client timeout 2137 ms). Trace:
I20260709 13:06:59.430610 23458 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:59.366931 23499 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:59.467301 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 72795.6us
I20260709 13:06:59.393404 23757 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:59.413836 23764 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:59.463688 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.409349 21218 raft_consensus.cc:2368] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
W20260709 13:06:59.466614 21797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 606) took 3143 ms (client timeout 2202 ms). Trace:
I20260709 13:06:59.470655 23769 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:06:59.469070 21799 rpcz_store.cc:269] 0709 13:06:56.026896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.027203 (+   307us) service_pool.cc:224] Handling call
0709 13:06:59.428950 (+3401747us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540046aad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:59.429089 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2960024960}
I20260709 13:06:59.475045 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.473618 23499 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:59.486726 23549 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.488113 23549 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:59.478266 21797 rpcz_store.cc:269] 0709 13:06:56.323711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.323844 (+   133us) service_pool.cc:224] Handling call
0709 13:06:59.417000 (+3093156us) spinlock_profiling.cc:243] Waited 2.97 s on lock 0x7b54005d5f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:59.466512 (+ 49512us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6826208128}
I20260709 13:06:59.481582 21219 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:59.492625 21221 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:59.470511 23735 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.496618 23761 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.484915 23764 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.485385 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.485749 23452 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:59.485913 23769 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:59.485997 23669 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:59.497295 23735 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:59.497997 21220 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:06:59.485328 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 54140.9us
I20260709 13:06:59.482470 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:06:59.454820 23663 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:59.497859 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 126008us
I20260709 13:06:59.504320 21214 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:06:59.453519 23628 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:06:59.461424 23616 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:06:59.514158 21111 connection.cc:256] RPC call timeout handler was delayed by -1.3881234579999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
I20260709 13:06:59.519856 21143 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:06:59.521358 23779 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.521695 21213 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:06:59.521890 23779 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.522486 23663 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:59.523983 21800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 602) took 3679 ms (client timeout 2002 ms). Trace:
I20260709 13:06:59.466816 23768 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:59.524507 21800 rpcz_store.cc:269] 0709 13:06:55.798763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.800312 (+  1549us) service_pool.cc:224] Handling call
0709 13:06:59.431921 (+3631609us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54005429d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:59.478414 (+ 46493us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080993024}
I20260709 13:06:59.532543 21801 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:06:59.535832 23777 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.539512 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.545540 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.548738 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.550065 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.559544 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.560565 21147 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:59.560654 23749 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:59.562282 21143 raft_consensus.cc:2410] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:06:59.564188 23761 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.569053 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.569942 21139 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
W20260709 13:06:59.581140 21216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 800) took 3060 ms (client timeout 2092 ms). Trace:
W20260709 13:06:59.585880 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.582325 21035 connection.cc:256] RPC call timeout handler was delayed by -1.3669382880000285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2137s for a response
W20260709 13:06:59.581696 21216 rpcz_store.cc:269] 0709 13:06:56.520673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.573372 (+ 52699us) service_pool.cc:224] Handling call
0709 13:06:59.580870 (+3007498us) spinlock_profiling.cc:243] Waited 2.58 s on lock 0x7b540034c6d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:06:59.581045 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5933100800}
I20260709 13:06:59.590627 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.592682 23556 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:59.605463 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:06:59.612336 23774 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:06:59.614322 21141 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:59.613780 23768 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.592576 23629 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.621454 23532 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:59.592518 23373 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:06:59.622131 23532 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:59.622519 23373 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:59.622954 23373 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:59.611698 23683 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.005s
I20260709 13:06:59.623903 23373 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.624559 23552 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:06:59.624441 23683 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.625057 23552 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:59.606263 21144 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:06:59.590515 23777 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.650295 23764 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.650663 23777 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:59.590272 23761 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:06:59.668691 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 341174us
W20260709 13:06:59.692191 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 158458us
W20260709 13:06:59.653786 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 115741us
I20260709 13:06:59.692529 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.631937 23654 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:06:59.663018 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.692847 23373 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:59.706856 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 167278us
I20260709 13:06:59.710274 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.711066 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.651108 23779 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.688712 23663 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.715433 21144 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:06:59.692134 23660 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:59.621532 23629 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:59.726552 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 163028us
I20260709 13:06:59.704278 23451 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:59.725999 21139 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:06:59.675911 23767 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.734627 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.742100 21147 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:59.743985 21145 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:06:59.744554 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:06:59.729348 23734 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:06:59.751432 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.714543 23761 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:06:59.753191 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 138853us
I20260709 13:06:59.759493 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:06:59.707894 23616 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:06:59.770119 23741 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.355s	user 0.005s	sys 0.000s
I20260709 13:06:59.771345 23660 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:59.730294 23654 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.742527 23769 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.795991 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:59.786793 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:59.795732 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:59.805899 21066 raft_consensus.cc:2410] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:59.818724 21064 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:06:59.822283 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 105046us
I20260709 13:06:59.824355 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:06:59.761356 23760 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:59.783557 23717 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:59.829366 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 84968.3us
I20260709 13:06:59.832306 23452 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:59.832907 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.834702 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:59.835500 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:59.837200 23696 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:59.848138 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.852839 23768 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.853252 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:06:59.853410 23768 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.853996 21066 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:06:59.854943 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:59.855789 23768 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:06:59.856734 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:06:59.858115 23666 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.865840 23767 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.859292 23555 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:59.888060 23555 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:06:59.889113 23555 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:06:59.890146 23555 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:59.861341 21068 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:06:59.861794 23579 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:59.863615 23761 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.852880 21064 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
W20260709 13:06:59.946018 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52967.3us
I20260709 13:06:59.948827 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:59.890362 23532 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:06:59.877097 23717 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:06:59.965586 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.966730 21064 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:59.967506 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:06:59.845316 23711 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 13:06:59.940030 23760 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:59.894228 23663 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.076s
W20260709 13:06:59.967511 21185 leader_election.cc:336] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 3.325s (SENT)
W20260709 13:06:59.970762 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 127915us
I20260709 13:06:59.917485 23654 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.975207 21070 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:06:59.984295 23532 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:06:59.973124 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:06:59.991868 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:00.003509 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52321.2us
W20260709 13:07:00.005429 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 110905us
I20260709 13:07:00.007469 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.008531 23517 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:00.008951 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.008971 23517 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.009724 23517 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:00.010696 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.013944 23532 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:00.015065 23761 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:00.015511 23761 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:00.015677 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:00.016404 23761 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.017506 23532 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:00.017748 23761 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:00.018010 23517 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.020503 23517 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:00.021183 23517 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:00.021520 23517 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.021869 23517 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.026273 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:00.031051 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 61613.1us
W20260709 13:07:00.036582 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 214115us
I20260709 13:07:00.036911 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:00.037161 21216 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:00.037753 21066 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:00.038296 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.039000 21216 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:00.039853 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.040784 21799 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:00.040927 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.043641 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.045297 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.033080 23756 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.054680 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:00.054998 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.055223 21797 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:00.056535 23686 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:00.057564 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.057936 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.058612 21795 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:00.058470 21216 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.059480 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.060819 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.061362 23760 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:00.062000 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:00.062561 21070 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.048440 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.063535 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.061458 21796 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:00.055265 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:00.048440 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.063548 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:00.078469 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.079339 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.972875 23735 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.033787 23654 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:00.083837 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.085075 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:00.085927 21800 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:00.099428 21793 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.000s
I20260709 13:07:00.100217 21793 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEARNER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:00.101104 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.101814 21068 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:00.103034 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.107554 21064 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:00.107982 23802 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:00.108400 23802 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:00.108665 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.108901 21213 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:00.109215 21214 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:07:00.110368 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.112756 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:00.113857 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:00.108779 23802 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.116606 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.117877 21795 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:00.118461 21797 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:07:00.076223 23686 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.082134 23761 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:00.054738 23756 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:00.126552 23552 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.003s
I20260709 13:07:00.118978 23696 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:00.119329 23793 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.125348 23562 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.120200 23666 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:00.125484 23797 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.120273 23768 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:00.146589 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.166683 21068 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9.
I20260709 13:07:00.126382 23761 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.172640 23761 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:00.173859 21800 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:00.174042 23517 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.156780 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.175437 23562 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.176469 23784 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:00.177021 23784 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.177440 23784 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:00.171514 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 314835us
I20260709 13:07:00.180902 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.182070 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:00.183727 23807 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:00.184239 23807 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:00.184705 23807 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:00.188191 23756 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:00.158360 23666 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.191784 23666 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:00.192782 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.194339 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:00.199385 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.161245 23793 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.200290 23803 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:00.200933 23803 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:00.202102 23793 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:00.202066 23803 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.099473 23711 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:00.143489 23686 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:00.145933 23552 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.117067 23717 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.209378 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.157013 21066 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:00.195106 21068 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:00.166306 23768 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:00.211555 23797 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.163779 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.215153 21216 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:00.122653 23654 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.218259 21220 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:00.216589 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:00.228845 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.242527 21065 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:00.238142 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:00.243489 21066 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:00.261046 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:00.259814 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.242152 23532 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:07:00.262478 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.263424 23756 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:00.263840 23756 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.263888 23552 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:00.264312 23756 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:00.264587 23552 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:00.265072 23552 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:00.265934 23552 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.266999 23686 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:00.267773 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.268435 21065 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:00.269333 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.269862 21065 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:00.269960 23769 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:00.270783 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.274441 23654 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:00.219274 23711 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:00.275978 23711 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:00.276666 23660 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.505s	user 0.004s	sys 0.004s
I20260709 13:07:00.276798 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.277135 23660 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:00.277269 23711 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.278123 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.278714 21214 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:00.279584 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.279860 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.280155 21214 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:00.281049 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.281379 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:00.281764 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.282284 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.283180 21216 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:00.282415 21220 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:00.284484 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:06:59.380168 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:00.285611 21213 raft_consensus.cc:2410] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:00.286942 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.288189 21187 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9.
I20260709 13:07:00.288547 21187 leader_election.cc:403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:00.288964 21066 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:00.289841 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.238674 23768 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:00.293722 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64275.5us
I20260709 13:07:00.294303 21800 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 2: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:00.295141 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.260283 23796 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:00.296741 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:00.298552 21185 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9.
I20260709 13:07:00.299885 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.300304 21070 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:00.302073 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.305269 21799 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:00.305459 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.305706 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:00.306772 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 93719.8us
I20260709 13:07:00.307633 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.310009 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.311266 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.312951 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:00.314895 21140 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:00.315312 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:00.281425 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.317252 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.291671 23555 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.321986 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:00.328553 21146 raft_consensus.cc:2376] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2. Re-sending same reply.
I20260709 13:07:00.328687 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:00.314370 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.328593 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.322707 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.321720 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.327868 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:00.321851 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:00.334758 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.334791 21139 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:00.304935 23686 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:07:00.336920 21799 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:00.323386 21035 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:00.340811 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.293445 21794 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:00.342911 21140 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:00.346906 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.340179 23798 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:00.350550 21216 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:00.353446 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.350596 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:00.354125 21144 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:00.301561 23793 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:07:00.353447 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:00.356889 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49421.4us
I20260709 13:07:00.357021 23660 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:00.357563 23660 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.357728 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:00.357945 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.357985 23660 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:00.358741 21067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.996s	user 0.012s	sys 0.015s
I20260709 13:07:00.357053 23808 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:07:00.359580 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.360395 21140 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:00.364187 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.365036 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.366416 21139 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:00.350387 21220 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:07:00.366390 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 55243.1us
W20260709 13:07:00.368649 21067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.006s	user 0.012s	sys 0.015s
I20260709 13:07:00.367220 21146 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:00.305743 23809 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:07:00.370610 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:00.340370 21793 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:00.351426 23654 raft_consensus.cc:2764] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:07:00.354933 23798 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.381552 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.388266 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:00.380940 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 105767us
I20260709 13:07:00.389309 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.370965 23549 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:00.400610 21220 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:00.368454 23805 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:00.353623 23555 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.373174 23815 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:00.406744 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.406458 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:00.406545 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.409955 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.395350 23798 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:00.425607 21145 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:00.404179 23805 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.419221 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.411542 23549 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:00.421039 23555 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:00.427984 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.432384 21146 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:00.432482 21141 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
W20260709 13:07:00.437459 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 84084.9us
I20260709 13:07:00.438179 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:00.447597 21765 leader_election.cc:336] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.749s (SENT)
I20260709 13:07:00.448351 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:00.449304 21145 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:00.449965 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 79443.4us
I20260709 13:07:00.411886 23815 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:00.403776 23813 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:00.428706 23779 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.425825 23734 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:00.433915 23805 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:00.451817 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.452082 23549 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.454298 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2405) took 3502 ms. Trace:
I20260709 13:07:00.457978 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.428208 23777 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.457356 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 758) took 4197 ms. Trace:
I20260709 13:07:00.460139 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:00.459971 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:00.461464 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 103853us
I20260709 13:07:00.489343 21146 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:00.403784 23809 consensus_queue.cc:1243] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:07:00.426628 23717 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:07:00.467761 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.471990 21066 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:00.490403 21066 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:00.473429 21144 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:00.447327 21035 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:00.463444 23762 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:00.492322 21141 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:07:00.492167 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53688.9us
I20260709 13:07:00.493043 21035 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:00.482688 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:00.414927 23761 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.494421 23555 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:00.478567 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.462412 23815 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:00.485924 21215 rpcz_store.cc:276] 0709 13:06:56.952001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.952153 (+   152us) service_pool.cc:224] Handling call
0709 13:06:57.018408 (+ 66255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.169695 (+1151287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.453812 (+2284117us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b540008fa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:00.454090 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752535424}
I20260709 13:07:00.553365 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.554328 21146 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:07:00.536558 21035 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:00.555042 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 72474.1us
I20260709 13:07:00.488207 21067 rpcz_store.cc:276] 0709 13:06:56.251984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.252136 (+   152us) service_pool.cc:224] Handling call
0709 13:06:56.252468 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.298227 (+ 45759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.298259 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:56.314891 (+ 16632us) log.cc:844] Serialized 26 byte log entry
0709 13:07:00.413365 (+4098474us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:00.427846 (+ 14481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.435733 (+  7887us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:00.435797 (+    64us) raft_consensus.cc:1685] finished
0709 13:07:00.435817 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:00.448985 (+ 13168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32427,"raft.queue_time_us":65836,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":14300,"thread_start_us":4006680,"threads_started":1,"wal-append.queue_time_us":873582,"wal-append.run_cpu_time_us":2811,"wal-append.run_wall_time_us":1517495}
W20260709 13:07:00.562408 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.553495 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:00.494551 23717 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:00.535374 23654 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:00.588531 21801 raft_consensus.cc:2368] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9.
I20260709 13:07:00.491129 23797 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:00.590691 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:00.594291 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:00.496232 23808 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:00.600791 23808 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.594566 21066 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.486203 23779 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.496438 23777 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.602362 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:00.599004 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.596064 23696 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:00.596735 23817 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:00.611824 21067 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:07:00.613184 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 60206.9us
I20260709 13:07:00.615846 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:00.582811 23555 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:00.607395 23816 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.210s	user 0.000s	sys 0.005s
I20260709 13:07:00.619563 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1941) took 1457 ms. Trace:
I20260709 13:07:00.608491 23808 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:00.621760 21142 rpcz_store.cc:276] 0709 13:06:59.158975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.159899 (+   924us) service_pool.cc:224] Handling call
0709 13:07:00.411935 (+1252036us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:07:00.616725 (+204790us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666752}
I20260709 13:07:00.627468 21066 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:00.628763 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.601761 23762 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:07:00.612946 23579 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:00.637502 21143 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:00.638870 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.614982 23818 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:00.643754 23828 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:00.606467 23802 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.624573 23696 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:00.650916 21147 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:00.638088 23451 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:00.642699 23817 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:00.637773 23654 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:00.660473 21064 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
W20260709 13:07:00.665349 23822 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:07:00.667824 23810 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
W20260709 13:07:00.668708 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.669426 23811 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.198s	user 0.000s	sys 0.000s
I20260709 13:07:00.619421 23825 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:07:00.672348 23825 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:00.672809 23825 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:00.673887 23825 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.653388 23818 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:00.676179 23818 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:00.678869 23784 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.501s	user 0.005s	sys 0.005s
I20260709 13:07:00.643764 23717 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.651610 23696 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:00.679407 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 63601.3us
I20260709 13:07:00.697464 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:00.700730 21111 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:00.691700 23761 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:00.704545 23513 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:00.705021 23513 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.705374 23513 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.708528 23777 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:00.709586 23802 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:00.710251 23771 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.376s	user 0.003s	sys 0.000s
I20260709 13:07:00.710459 23660 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.714330 21063 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:00.715664 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1935) took 1607 ms. Trace:
I20260709 13:07:00.715794 23686 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9.
I20260709 13:07:00.716091 21138 rpcz_store.cc:276] 0709 13:06:59.107907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.122767 (+ 14860us) service_pool.cc:224] Handling call
0709 13:06:59.123180 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:59.129723 (+  6543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:59.129766 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:59.150036 (+ 20270us) log.cc:844] Serialized 26 byte log entry
0709 13:06:59.192409 (+ 42373us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:59.192458 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.192535 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:00.709533 (+1516998us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:00.715363 (+  5830us) raft_consensus.cc:1685] finished
0709 13:07:00.715385 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:00.715603 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18502,"raft.queue_time_us":21137,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":10368,"spinlock_wait_cycles":8252288,"thread_start_us":16194,"threads_started":1,"wal-append.queue_time_us":65672}
I20260709 13:07:00.717025 23654 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.719609 23825 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:00.740062 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:00.740226 21145 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
W20260709 13:07:00.727416 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 133230us
I20260709 13:07:00.725538 21221 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:00.742682 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:00.745819 21066 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:07:00.746587 21066 pending_rounds.cc:85] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 3
I20260709 13:07:00.746980 21066 pending_rounds.cc:107] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:07:00.746758 21218 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:00.747445 21066 raft_consensus.cc:2889] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:00.744165 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:00.748775 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:00.750492 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.752666 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.753428 21221 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
W20260709 13:07:00.752856 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 216364us
I20260709 13:07:00.761160 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:00.762320 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.762955 21214 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:00.765293 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.765898 21220 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.759725 23761 raft_consensus.cc:2764] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:07:00.662586 23817 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:00.767998 23717 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:00.768092 23817 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:00.769321 23373 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:00.769766 23373 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:00.687333 23835 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:00.759974 23695 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.772938 23798 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.774580 23734 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:00.770675 23373 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:00.690923 23807 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.001s
I20260709 13:07:00.670967 23756 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.739542 23696 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:00.809742 21220 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:00.859022 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:00.859922 21220 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:00.776981 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:00.861299 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 320) took 1066 ms. Trace:
I20260709 13:07:00.861810 21072 rpcz_store.cc:276] 0709 13:06:59.795258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.776636 (+981378us) service_pool.cc:224] Handling call
0709 13:07:00.861231 (+ 84595us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:00.856853 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.868299 21214 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
W20260709 13:07:00.871644 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 168413us
I20260709 13:07:00.691890 23779 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:00.704547 23555 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:00.781020 23666 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.001s
I20260709 13:07:00.881000 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:00.775118 23734 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:00.874325 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:00.880568 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.759694 23532 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:00.775178 23798 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:00.873478 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 124806us
I20260709 13:07:00.880021 23451 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:00.893946 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:00.741636 23784 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.896907 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:00.687000 23815 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:00.898422 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 401999us
W20260709 13:07:00.899389 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 166719us
I20260709 13:07:00.881986 23764 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:00.911512 23764 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.911936 23764 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.881639 23759 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:00.780259 23805 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.917078 21215 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:00.918009 23805 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:00.901237 23761 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:00.919190 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.919531 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1738) took 3029 ms. Trace:
I20260709 13:07:00.919935 21071 rpcz_store.cc:276] 0709 13:06:57.889495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.897764 (+  8269us) service_pool.cc:224] Handling call
0709 13:06:57.898178 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.516646 (+618468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.516704 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:58.517466 (+   762us) log.cc:844] Serialized 26 byte log entry
0709 13:06:58.594780 (+ 77314us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:58.594834 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.919031 (+2324197us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540011c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:00.919077 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:00.919141 (+    64us) raft_consensus.cc:1685] finished
0709 13:07:00.919152 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:00.919368 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420677504,"thread_start_us":76803,"threads_started":1,"wal-append.queue_time_us":78738,"wal-append.run_cpu_time_us":2985,"wal-append.run_wall_time_us":1034864}
I20260709 13:07:00.920920 23832 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:00.921443 23832 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.921809 23832 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.923220 23660 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:00.901806 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 98988.2us
I20260709 13:07:00.901057 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:00.906442 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:00.906896 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:00.907327 23686 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:00.908818 23815 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:00.887567 23513 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.854187 23807 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:00.775501 23768 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.924606 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:00.896729 23373 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:00.893025 23532 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:00.936681 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:00.886929 23666 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:00.943861 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:00.944782 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:00.945961 21138 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:00.944500 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:00.915442 23549 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:00.923874 23735 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:00.955476 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:00.956828 21069 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:00.940672 23777 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:00.937829 23761 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:00.959447 21145 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:00.960013 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:00.945137 23532 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.942130 23452 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:00.963146 21213 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:00.964327 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:00.948896 23660 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:00.937743 23555 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:07:00.957845 23451 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:00.961078 23777 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.982851 23777 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:00.983162 23670 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:07:00.984229 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 701185us
I20260709 13:07:00.965209 23793 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:00.978551 23660 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:00.998239 23686 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:00.999045 23686 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:00.999591 23686 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.002319 23542 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.994779 21185 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:01.006523 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 52131.1us
I20260709 13:07:00.996919 23670 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:01.021466 23735 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:01.013895 23542 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.021911 23735 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:01.022850 23735 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.023586 23542 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:00.997323 23793 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:01.024189 23735 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:01.024621 23784 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:00.977481 23805 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:07:01.024813 23793 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.025633 23805 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:01.026007 23805 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:01.026090 23793 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:01.017545 23826 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:01.026878 23805 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.027568 23826 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.028234 23805 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:01.028520 23562 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:01.029574 23769 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:01.029946 23769 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:01.029856 23826 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.030664 23660 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.030788 23769 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.031700 21213 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:01.020529 23629 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:01.032896 23686 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:01.034405 21217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2429) took 1676 ms (client timeout 1780 ms). Trace:
W20260709 13:07:01.034826 21217 rpcz_store.cc:269] 0709 13:06:59.357488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.361208 (+  3720us) service_pool.cc:224] Handling call
0709 13:07:01.034225 (+1673017us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b54000bbdd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:01.034324 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72954624}
I20260709 13:07:01.036175 21138 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:01.037901 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.038488 23513 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.038905 23761 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:01.032251 23769 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:01.039309 23817 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.041862 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.011799 23654 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.049386 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 54734.3us
I20260709 13:07:01.049575 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.050120 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.050521 23798 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:01.055455 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.056103 21217 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:01.056291 23833 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.136s	user 0.002s	sys 0.000s
I20260709 13:07:01.059073 21068 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:01.059557 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.060415 23807 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:01.060648 23683 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.063889 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.064576 21065 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.065316 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:01.067346 23842 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.156s	user 0.002s	sys 0.000s
I20260709 13:07:01.068964 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.069391 23761 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:01.074772 23761 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:01.075896 23761 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.076813 23817 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:01.070057 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.070261 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2433) took 1711 ms. Trace:
I20260709 13:07:01.071620 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:01.080066 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 790) took 1095 ms. Trace:
I20260709 13:07:01.080257 23767 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:01.080766 23767 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:01.081163 23767 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.082149 23762 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:07:01.059041 21069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1752) took 2122 ms (client timeout 2112 ms). Trace:
I20260709 13:07:01.079660 21219 rpcz_store.cc:276] 0709 13:06:59.358745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.372391 (+ 13646us) service_pool.cc:224] Handling call
0709 13:06:59.372704 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.491522 (+118818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.069571 (+1578049us) spinlock_profiling.cc:243] Waited 99.9 ms on lock 0x7b5400102554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:01.069916 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229830016}
I20260709 13:07:01.050185 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:01.080490 21064 rpcz_store.cc:276] 0709 13:06:59.984712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.338153 (+353441us) service_pool.cc:224] Handling call
0709 13:07:00.351905 (+ 13752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:00.663109 (+311204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.079835 (+416726us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b540041c654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:01.080009 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709017856}
W20260709 13:07:01.083748 21069 rpcz_store.cc:269] 0709 13:06:58.936747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.279449 (+1342702us) service_pool.cc:224] Handling call
0709 13:07:01.058750 (+779301us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b54003c9e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:01.058937 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548032384}
I20260709 13:07:01.090653 21072 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
W20260709 13:07:01.091411 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.029923 23768 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:01.040670 23756 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:01.092761 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 80068.5us
W20260709 13:07:01.086628 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.039839 23532 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.046691 23695 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:01.046905 23734 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.094098 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.147538 21145 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:01.095536 23837 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:07:01.096447 23666 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:01.097276 23839 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 13:07:01.086666 23808 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.098769 23518 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.065583 23764 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.175688 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 787) took 1191 ms. Trace:
I20260709 13:07:01.092648 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.107016 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.107564 23817 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.021801 23670 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.110656 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.181736 21066 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:01.119705 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.124518 21217 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:07:01.125854 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.125805 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:01.186743 21795 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:01.130180 23696 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.139508 21065 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:01.194813 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.195603 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.142783 23832 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.197497 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.143870 23796 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.197981 21063 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
W20260709 13:07:01.144688 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.199271 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.146157 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 44982.9us
I20260709 13:07:01.131067 23579 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:01.202662 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.093694 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.166852 23808 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.203976 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.184929 23756 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.170171 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 793) took 1065 ms. Trace:
W20260709 13:07:01.172452 23823 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.554s	user 0.001s	sys 0.000s
I20260709 13:07:01.172925 23695 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.178897 21216 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:01.202308 23796 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:01.179916 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 87342.3us
I20260709 13:07:01.176152 21068 rpcz_store.cc:276] 0709 13:06:59.983864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.293985 (+310121us) service_pool.cc:224] Handling call
0709 13:07:01.058895 (+764910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.059670 (+   775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.175466 (+115796us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b54003c9e50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:01.175626 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1699037056}
I20260709 13:07:01.186268 21796 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.091347 23654 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.189255 23817 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.189847 23517 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.185390 23828 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:01.195801 21065 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:01.141873 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:01.196233 21072 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:01.144273 23532 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:01.201365 21797 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:07:01.156795 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 106666us
I20260709 13:07:01.204816 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.210004 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.213778 21145 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:01.167738 23734 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.214600 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.215888 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.251019 21138 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.251080 21063 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:01.215224 21070 rpcz_store.cc:276] 0709 13:07:00.105155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.353937 (+248782us) service_pool.cc:224] Handling call
0709 13:07:00.368925 (+ 14988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.400478 (+ 31553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.400512 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.410393 (+  9881us) log.cc:844] Serialized 26 byte log entry
0709 13:07:00.753398 (+343005us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:00.753443 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.753524 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:01.096560 (+343036us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:01.169799 (+ 73239us) raft_consensus.cc:1685] finished
0709 13:07:01.169831 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.170099 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2054,"raft.queue_time_us":21755,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2438,"spinlock_wait_cycles":12520320,"thread_start_us":329473,"threads_started":1,"wal-append.queue_time_us":461997}
I20260709 13:07:01.218060 21793 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:07:01.215186 23768 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:01.224521 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:01.225180 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:01.232581 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 90813.1us
I20260709 13:07:01.236068 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.253476 21067 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:01.253471 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 48723.4us
I20260709 13:07:01.236969 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.248309 21213 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:01.250048 21212 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.259433 21219 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:01.259533 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:01.260346 21214 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:01.261157 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.263346 21065 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:01.263646 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:01.264492 21218 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:01.269071 21220 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:07:01.269518 21064 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:01.270449 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:01.263203 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.272492 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.273026 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:01.271431 21221 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:01.274610 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.236527 23808 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:01.275470 21216 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:01.278501 21110 leader_election.cc:400] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9.
I20260709 13:07:01.278913 21110 leader_election.cc:403] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:07:01.223212 23756 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.281728 21215 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:01.283878 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.309242 21072 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:01.286422 23768 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:01.286612 23666 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:01.288595 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 283) took 1014 ms. Trace:
I20260709 13:07:01.291448 23630 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:01.293648 21070 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:01.294346 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.316847 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.295529 23757 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9.
I20260709 13:07:01.295459 23485 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:01.321300 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.321882 21801 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:01.296367 23735 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:01.297226 23818 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:01.298077 23518 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:01.227626 23695 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:01.304385 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.305117 21795 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:01.307330 21138 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:01.247262 23828 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:01.313611 21800 rpcz_store.cc:276] 0709 13:07:00.273821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.303436 (+ 29615us) service_pool.cc:224] Handling call
0709 13:07:00.303856 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.304327 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.304369 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.340501 (+ 36132us) log.cc:844] Serialized 26 byte log entry
0709 13:07:00.437018 (+ 96517us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:00.451891 (+ 14873us) log.cc:844] Serialized 14 byte log entry
0709 13:07:00.651185 (+199294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.653944 (+  2759us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:01.171756 (+517812us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:01.288245 (+116489us) raft_consensus.cc:1685] finished
0709 13:07:01.288365 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.288531 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48530,"prepare.queue_time_us":18294,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":65571,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":50628,"spinlock_wait_cycles":79591808,"thread_start_us":256866,"threads_started":2,"wal-append.queue_time_us":229678}
I20260709 13:07:01.325170 21063 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:01.317622 21217 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:01.281121 23836 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 13:07:01.318488 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:01.328444 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.329123 21218 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:01.319698 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.322948 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.330590 21064 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:07:01.280262 23808 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:01.334302 21068 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:01.325965 23630 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:01.336086 23817 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:01.336342 23630 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.337107 23695 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:01.337818 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 84592.5us
W20260709 13:07:01.338721 23841 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.418s	user 0.004s	sys 0.000s
I20260709 13:07:01.339953 21036 leader_election.cc:400] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:01.340221 23764 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:01.341626 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:01.341820 23756 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:01.342787 23847 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:07:01.342887 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:01.320470 23683 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:01.332887 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.346041 23797 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:01.371079 23797 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:01.348867 23813 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:01.349699 23808 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:01.372042 23797 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.373413 23797 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.374286 21796 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:01.374562 23660 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:01.375005 23660 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:01.352686 23696 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.375386 23660 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:01.354768 23768 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:01.382478 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.383237 23850 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:07:01.328115 23518 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.327780 23666 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:01.358283 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.336041 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.398523 21800 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.398886 21070 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:01.354396 21214 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:07:01.359338 21795 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:07:01.360484 21067 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:01.355670 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.398170 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.400514 21217 raft_consensus.cc:2376] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8. Re-sending same reply.
I20260709 13:07:01.345538 23654 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:01.365422 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:01.367225 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.400980 21218 raft_consensus.cc:2368] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8.
I20260709 13:07:01.367911 23828 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.357049 23832 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:01.355525 23796 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.348165 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.416507 21219 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.386212 21801 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:07:01.381899 21064 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:01.383553 23532 raft_consensus.cc:2764] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:07:01.430970 21218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:56816
I20260709 13:07:01.431792 21218 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:01.432238 21218 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.432279 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.433113 21138 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:01.437566 23808 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.439831 23808 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.440789 23579 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:07:01.398161 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65355.3us
I20260709 13:07:01.443029 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:01.444250 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.444554 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.402714 21185 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.412370 21212 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:07:01.418071 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 52756.6us
I20260709 13:07:01.418283 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.446328 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.421470 21800 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:01.346436 23767 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.346382 23777 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.423046 23654 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:01.423553 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.430063 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.449724 21219 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:01.452481 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2409) took 4496 ms. Trace:
I20260709 13:07:01.455895 21147 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:07:01.452419 21218 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.456961 23757 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:01.484725 23813 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:01.383064 23660 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.486378 23798 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:01.486842 23798 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.431844 23666 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.487185 23798 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:01.443779 23769 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:01.491700 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.491837 23769 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:01.492611 21064 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:01.493264 23769 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.487100 21799 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:01.445847 23702 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.495216 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:01.456179 21212 rpcz_store.cc:276] 0709 13:06:56.954838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.251451 (+296613us) service_pool.cc:224] Handling call
0709 13:06:57.327489 (+ 76038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.364032 (+2036543us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b180015d860. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42557646 00007f7f4249158e 00007f7f42512f9e 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4
0709 13:06:59.861858 (+497826us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b58007b3ed0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f424780aa 00007f7f424fa447 00007f7f42512fe7 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4
0709 13:07:01.249306 (+1387448us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b58007b41d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f424780aa 00007f7f424fa447 00007f7f42512fe7 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4
0709 13:07:01.445701 (+196395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.451052 (+  5351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6286817152}
I20260709 13:07:01.497622 21140 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:01.419060 23748 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:01.499943 21219 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:01.451499 23579 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.459918 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 57330us
I20260709 13:07:01.501483 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:01.510020 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46676.7us
I20260709 13:07:01.516461 23494 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:07:01.517186 23734 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:01.517632 23734 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:01.518014 23734 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:01.487344 23660 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:01.521032 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.500974 23796 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.519443 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.522459 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.484833 23732 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:01.531417 21218 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:01.526304 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.529687 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.532258 21064 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:01.532509 21220 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:01.501785 23748 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.462730 23513 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:01.518518 21794 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:01.537811 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:01.541201 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.461620 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.542004 21072 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:01.542502 21111 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.548858 21216 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:01.548810 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:01.551582 21216 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:01.553452 21214 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:01.550314 21035 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.554670 21035 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.555150 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:01.555577 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 94054.9us
I20260709 13:07:01.558532 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.559783 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.560071 23834 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:01.560187 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.561295 21147 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:07:01.561484 21145 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:01.561834 23826 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:01.562353 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.544763 23784 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:07:01.500448 23702 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.568679 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.565069 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.571154 21801 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:01.532092 23666 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:01.531845 23732 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:07:01.584499 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62122.8us
I20260709 13:07:01.537250 23748 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:01.589421 21185 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:01.590562 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.591444 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.546139 23695 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:01.519152 23579 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.553815 23756 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:01.593153 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:01.594007 21066 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.605446 21212 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:01.532788 23683 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:07:01.594835 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.609714 21065 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:01.595067 21220 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:01.610850 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.611902 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.587904 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.606528 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.614914 21143 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:07:01.591811 21217 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:01.604677 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.615872 21071 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:07:01.570549 23796 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:01.589427 23835 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:01.545189 23494 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:07:01.581889 23808 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:01.617583 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.637166 21142 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:01.616714 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:01.637166 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.621348 21066 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:01.622078 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.638085 21064 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:07:01.625015 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:01.538443 23813 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:07:01.592553 23732 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:01.616745 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.645599 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.646690 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.621055 23373 raft_consensus.cc:2764] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:07:01.632982 21036 connection.cc:256] RPC call timeout handler was delayed by -2.2777431130000423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
I20260709 13:07:01.633505 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.653378 21219 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:01.634413 21146 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:01.567451 23660 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:07:01.628165 23805 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.628329 23797 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:01.656360 21143 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:01.594326 23798 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.606952 23695 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.659389 21216 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:01.620483 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.660167 21141 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:01.640260 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.640908 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.607275 23666 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:01.553478 23757 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
W20260709 13:07:01.627784 23645 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:07:01.645640 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:01.553546 23451 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:07:01.648826 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.647861 23735 raft_consensus.cc:2764] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:07:01.651000 21072 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:01.592756 23826 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:07:01.621199 23518 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.657744 21139 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:01.666908 21142 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:07:01.635794 21147 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:01.655439 23756 raft_consensus.cc:2764] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:07:01.635597 23796 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:01.680567 21217 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:01.662880 23695 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:01.682693 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.683262 21144 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:01.665892 23485 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:01.686695 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.688161 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.689289 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:01.690177 21072 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:01.668210 23797 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:01.663488 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.691884 21146 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:07:01.665110 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:01.666250 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:01.650259 20970 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 163) took 11252 ms (client timeout 15000 ms). Trace:
I20260709 13:07:01.667654 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:01.672693 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:01.655767 23817 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:01.675918 21064 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:01.694406 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.703929 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:01.664256 23805 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.713122 23796 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.752450 23494 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:01.753051 23494 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.753435 23777 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.754134 23777 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:01.754591 23777 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.713657 23666 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.755311 23734 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.756139 23579 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:01.757315 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:01.757089 23797 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.757908 21796 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:01.757943 23734 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:01.715083 23853 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:07:01.714926 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:01.715740 23855 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 13:07:01.761078 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66738.7us
I20260709 13:07:01.728207 23798 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:01.735342 23817 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.735869 23660 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:01.737149 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.739449 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:01.798441 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.739718 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.740216 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:01.800575 21797 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:01.740921 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:01.742543 21141 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:01.712478 21143 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:01.801817 21795 raft_consensus.cc:2368] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5.
I20260709 13:07:01.721961 23818 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:01.766317 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:01.761909 23485 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:01.798489 21801 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:01.798869 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 59512.9us
I20260709 13:07:01.681032 23683 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:07:01.817902 21185 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.810681 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:01.820273 21795 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:01.821285 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:01.743837 23826 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:01.814404 21110 leader_election.cc:400] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5.
I20260709 13:07:01.825398 21110 leader_election.cc:403] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:01.825484 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 161988us
I20260709 13:07:01.797681 23735 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:01.760739 23494 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:01.834939 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 292500us
W20260709 13:07:01.842984 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8583431130000463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
I20260709 13:07:01.798069 23660 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:01.799240 23670 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:01.735430 23696 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:07:01.849213 21036 connection.cc:256] RPC call timeout handler was delayed by -1.6103014330000178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:07:01.814109 23579 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:01.805322 23702 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:01.805215 23818 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:01.834612 23826 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.854705 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.854784 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:01.855924 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.857089 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 237026us
I20260709 13:07:01.858489 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:01.852916 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4249014330000591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 13:07:01.836938 23735 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.839877 23798 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:07:01.877321 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4240014330000577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
I20260709 13:07:01.837678 23485 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:01.879189 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:01.693495 20970 rpcz_store.cc:269] 0709 13:06:50.347033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.347275 (+   242us) service_pool.cc:224] Handling call
0709 13:07:01.589584 (+11242309us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:56.591738 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:56.897813 (+306075us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:56.901451 (+  3638us) write_op.cc:435] Acquired schema lock
0709 13:06:56.917348 (+ 15897us) tablet.cc:662] Decoding operations
0709 13:06:57.300697 (+383349us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:57.300784 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:57.300817 (+    33us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:06:57.301693 (+   876us) tablet.cc:701] Row locks acquired
0709 13:06:57.301714 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:06:57.301867 (+   153us) write_op.cc:270] Start()
0709 13:06:57.301994 (+   127us) write_op.cc:276] Timestamp: P: 1783602417301814 usec, L: 0
0709 13:06:57.302016 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:06:57.621461 (+319445us) log.cc:844] Serialized 4014 byte log entry
0709 13:06:58.425505 (+804044us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:06:58.469282 (+ 43777us) op_driver.cc:464] REPLICATION: finished
0709 13:06:58.611502 (+142220us) write_op.cc:301] APPLY: starting
0709 13:06:58.611681 (+   179us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:58.612062 (+   381us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:58.612099 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:58.615221 (+  3122us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:58.615651 (+   430us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:06:58.615729 (+    78us) write_op.cc:312] APPLY: finished
0709 13:06:58.616318 (+   589us) log.cc:844] Serialized 77 byte log entry
0709 13:06:58.616563 (+   245us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:58.616941 (+   378us) write_op.cc:454] Released schema lock
0709 13:06:58.651941 (+ 35000us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:58.943783 (+291842us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:58.943825 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":199,"mutex_wait_us":10891,"reported_tablets":23,"spinlock_wait_cycles":2432,"child_traces":[["op",{"apply.queue_time_us":1070,"apply.run_cpu_time_us":5374,"apply.run_wall_time_us":332257,"num_ops":10,"prepare.queue_time_us":353209,"prepare.run_cpu_time_us":25567,"prepare.run_wall_time_us":1218014,"raft.queue_time_us":32903,"raft.run_cpu_time_us":1532,"raft.run_wall_time_us":148108,"replication_time_us":1167157,"spinlock_wait_cycles":4793216,"thread_start_us":484279,"threads_started":4,"wal-append.queue_time_us":186517,"wal-append.run_cpu_time_us":20881,"wal-append.run_wall_time_us":1812960}]]}
I20260709 13:07:01.894517 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.895434 21141 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:01.852555 23670 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.852891 23696 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:01.856364 23579 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.850267 23835 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:01.903337 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:01.946398 21144 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:01.949971 21146 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:07:01.944558 23696 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.912115 23767 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:01.915666 23734 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:01.851667 23630 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:07:01.917069 23660 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.920099 23732 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:01.921891 23654 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:01.924325 23518 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:01.886283 23735 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:01.932416 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.887151 23858 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:07:01.934455 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.934705 23817 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:01.885525 23757 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:01.935820 23805 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:01.876878 23826 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:01.901003 23670 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.882586 23494 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:07:01.898711 23663 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.008746 23663 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:01.952649 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 73520.3us
I20260709 13:07:02.011422 23696 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:02.012333 21111 leader_election.cc:400] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8.
I20260709 13:07:02.012665 21111 leader_election.cc:403] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:07:01.953965 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:02.013231 21146 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:01.969067 23518 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:02.015992 23518 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.915757 23818 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.017191 23518 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:02.018018 23818 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.958096 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:02.019644 21141 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:02.020622 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:02.021342 21141 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:02.022140 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:01.959209 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:01.951459 23808 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:01.961628 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58397.3us
I20260709 13:07:02.024171 23808 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:01.951439 23579 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:01.951745 23629 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5.
I20260709 13:07:01.885783 23451 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.072s
W20260709 13:07:01.969655 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 265799us
I20260709 13:07:01.974713 21800 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:01.956928 23756 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:01.962483 23798 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.003005 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:02.003556 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:02.041617 21147 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:02.003726 23861 raft_consensus.cc:2764] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:07:02.006024 21068 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:02.052270 21068 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:02.032256 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:01.997720 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:02.023192 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:02.037221 21146 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:02.025974 21072 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
W20260709 13:07:02.056502 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.057502 21141 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:02.057451 21138 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:02.040699 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:02.067011 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69398.8us
I20260709 13:07:02.070240 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:02.072105 21144 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:02.074429 21147 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:02.076864 21141 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:07:02.078590 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:02.094849 21142 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:02.095155 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:02.096649 21146 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:02.097772 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:02.098403 21146 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:02.011158 23663 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:02.101938 21146 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
W20260709 13:07:02.065666 23847 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:07:02.064675 23660 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:02.105280 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 73101.6us
I20260709 13:07:02.108920 21144 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:02.125735 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:02.095255 21142 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:02.109817 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.053714 23756 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:02.104074 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 109353us
I20260709 13:07:02.112018 21068 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.127274 21068 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:02.127592 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:02.121704 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:02.128798 21141 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:02.120606 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:02.070089 23373 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:02.094534 23764 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:02.135675 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:02.024585 23748 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:02.095875 23863 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:02.061363 23798 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:02.160198 21113 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:02.076354 23485 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:07:02.118784 23452 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8.
I20260709 13:07:02.124495 23797 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:02.165936 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 45402.4us
I20260709 13:07:02.175689 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:02.129670 23756 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:02.197567 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:02.136030 23373 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:02.213526 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 87853.3us
I20260709 13:07:02.111088 23702 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:07:02.214782 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:02.128605 23767 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:07:02.161645 23494 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:02.236147 21068 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:02.117233 23865 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.106s
I20260709 13:07:02.254277 21214 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:02.212787 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:02.222687 23373 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:02.229463 23401 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:02.110811 23835 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.263602 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:02.242120 23797 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:02.161450 23695 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.241743 23452 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:07:02.266891 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 69405.4us
W20260709 13:07:02.248878 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 88784.8us
I20260709 13:07:02.362370 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:02.363823 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:02.311031 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.367635 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.368019 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.242772 23876 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:02.269752 23863 raft_consensus.cc:2764] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:07:02.358937 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.309654 23451 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:02.349515 23532 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.345770 23805 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:07:02.371213 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 359004us
I20260709 13:07:02.327795 23732 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:02.377653 21068 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:02.379698 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:02.349493 21794 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:07:02.390406 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 215751us
I20260709 13:07:02.348415 23808 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.413009 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:02.414475 21185 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:02.418751 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:02.434406 21139 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:02.372445 23702 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:02.436225 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:02.378988 23452 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:07:02.386560 23494 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:02.443948 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:02.412914 23865 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:02.395700 23876 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:02.460199 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:02.403784 23532 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.467556 23485 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:02.468120 23485 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.468477 23485 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:02.469762 23532 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:02.470649 23748 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:02.471814 23881 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:02.473327 23881 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:07:02.480050 23885 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:07:02.449599 23494 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.458639 23868 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:02.462013 23805 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.486102 23863 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:02.420678 23808 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.445331 23702 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.510037 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:02.447885 23451 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:07:02.521896 23883 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 13:07:02.522488 23579 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.495857 23494 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:02.491341 23888 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:02.496605 23616 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:07:02.516269 23863 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:02.530059 23826 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.002s
I20260709 13:07:02.547657 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:02.519325 23798 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:02.568245 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:02.538584 23878 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:07:02.634949 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:02.635601 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 56501.1us
W20260709 13:07:02.635713 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67586.4us
W20260709 13:07:02.642150 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 223460us
I20260709 13:07:02.514724 23485 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.516651 23805 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:02.548182 23818 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.000s
I20260709 13:07:02.539496 23654 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:02.539049 23579 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:02.665078 21185 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:02.542183 23518 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.000s
I20260709 13:07:02.542595 23895 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:07:02.665946 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 245648us
I20260709 13:07:02.684002 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:02.536118 23886 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:02.624498 23532 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.688068 23532 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.667462 23645 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:07:02.536392 23702 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:02.569329 23757 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:02.691816 23757 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:02.692718 23757 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.693499 23903 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:02.694046 23757 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:02.694346 23903 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:02.673583 23642 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:02.674945 23485 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:02.695649 23903 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.696791 23532 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:02.697937 23517 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:02.698246 23887 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:02.699242 23887 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:02.700850 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:02.539326 23888 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.665789 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:02.547019 23494 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:02.670912 23670 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.652s	user 0.007s	sys 0.004s
I20260709 13:07:02.723992 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:02.592438 23401 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:02.724329 23670 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.724813 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:02.725224 23401 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.725654 21799 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:02.545136 23451 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:02.600231 23863 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:02.707075 21214 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:02.732088 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:02.709096 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:02.543488 23871 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.142s
I20260709 13:07:02.712110 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:02.568059 23896 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:07:02.721131 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:02.721541 23805 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:02.722114 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:02.716265 23485 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:02.723505 21215 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:02.670312 23663 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:02.703394 23695 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:02.716265 23858 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:02.739415 21216 raft_consensus.cc:2376] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9. Re-sending same reply.
I20260709 13:07:02.741190 21217 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:07:02.661615 23826 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:02.747601 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:02.730005 21794 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:02.735543 23735 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.726s	user 0.008s	sys 0.002s
I20260709 13:07:02.729750 23451 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:02.729187 23579 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:02.751515 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 270) took 1334 ms (client timeout 1579 ms). Trace:
I20260709 13:07:02.748304 23485 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.735117 23616 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.753635 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:02.780051 21212 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:02.752892 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 202688us
I20260709 13:07:02.758005 21215 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:02.548687 23876 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.752224 21145 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:02.765441 23735 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.771865 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:02.829954 21216 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:02.765858 21140 rpcz_store.cc:269] 0709 13:07:01.417500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.418012 (+   512us) service_pool.cc:224] Handling call
0709 13:07:02.751329 (+1333317us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b5400447ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:02.751462 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114333568}
I20260709 13:07:02.820883 23532 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:02.760108 23696 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.001s
I20260709 13:07:02.830253 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:02.820844 23579 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.764133 23642 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:02.763029 23858 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:02.834120 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:02.779449 23485 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:02.865818 21212 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:02.863669 21217 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:07:02.866500 21799 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:07:02.862409 21220 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
W20260709 13:07:02.831981 23882 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:07:02.873027 21801 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:02.828995 23808 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:02.880282 21795 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
W20260709 13:07:02.880748 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46696.2us
I20260709 13:07:02.840804 23663 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:02.881489 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:02.898262 21216 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:02.827466 23616 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.898833 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:02.887274 23373 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.579s	user 0.007s	sys 0.004s
I20260709 13:07:02.901255 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:02.892177 23871 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:02.864867 23889 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:07:02.864084 23532 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.892516 23695 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.910740 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:02.863794 23908 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:07:02.886426 23451 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.914947 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:02.923959 21218 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:02.909377 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:02.925042 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:02.897842 23808 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:02.928357 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:02.928941 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.928938 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:02.931257 21143 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:07:02.932761 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:02.906976 23663 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:07:02.936986 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.939777 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:02.941970 23451 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:02.943202 23532 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:02.943797 23906 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:07:02.945175 23579 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:02.946102 23826 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:02.947731 23900 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:07:02.908181 23748 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:02.951113 21218 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:07:02.954501 21138 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:07:02.895880 23904 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:07:02.901997 23654 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:07:02.920703 23702 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.957798 23818 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.958293 23818 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.958801 21219 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:02.936034 23808 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.959240 23808 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:02.959619 23702 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:02.960263 23818 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:02.901928 23835 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:07:02.902021 23696 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:07:02.927081 23695 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:02.962697 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:02.938251 23660 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.807s	user 0.007s	sys 0.006s
I20260709 13:07:03.030105 21221 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:02.985791 21765 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.579s (SENT) [suppressed 30 similar messages]
I20260709 13:07:02.907538 23373 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:02.956646 23748 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:02.968372 23494 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:03.026765 23654 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:03.094636 21214 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:03.066310 23732 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.068969 23826 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.069234 23735 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:03.079268 23748 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:02.969988 23908 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.119s
I20260709 13:07:03.076252 23793 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:03.078190 23901 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:02.925769 23858 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:03.120036 23905 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 13:07:03.120995 23913 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 13:07:03.129200 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 189730us
I20260709 13:07:03.131064 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:03.110791 21221 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:03.094342 23654 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:03.134433 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:03.105147 21794 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
W20260709 13:07:03.136173 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 150437us
I20260709 13:07:03.136463 21140 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:07:03.136874 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:03.107360 23793 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:03.132912 23868 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.619s	user 0.006s	sys 0.004s
W20260709 13:07:03.135521 23890 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:07:03.135270 23654 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:03.104635 23732 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:03.104846 23826 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:03.137725 23835 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:03.148887 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:03.140863 23660 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:03.129380 23901 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:03.132478 23696 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:07:03.137274 23683 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:07:03.143882 23868 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:03.227785 23868 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:03.228498 23868 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:03.156947 23917 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:03.232813 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:03.162446 23918 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:03.233484 21221 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:03.234700 23695 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:03.234855 21221 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:03.235756 23696 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:03.235903 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:03.236213 23696 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:03.236531 21221 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:07:03.236999 23696 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:03.237473 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:03.238201 23696 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:03.239533 23793 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:03.241194 23793 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:03.177192 23732 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:03.178062 23908 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:03.181025 23826 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:03.198750 23616 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:03.205091 23670 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:03.215912 23888 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:03.217185 21798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 626) took 2085 ms (client timeout 1542 ms). Trace:
I20260709 13:07:03.219316 23813 raft_consensus.cc:1027] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: attempt to promote peer b70fc04de37e4239a8336534f0c9c0fb: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:07:03.220616 23817 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:03.221648 23777 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:03.223464 23835 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:03.247180 21214 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:03.231204 23868 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:03.249366 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 88162.8us
I20260709 13:07:03.281622 21067 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:03.248560 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:03.395313 21798 rpcz_store.cc:269] 0709 13:07:01.132054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.214792 (+ 82738us) service_pool.cc:224] Handling call
0709 13:07:03.216861 (+2002069us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540047ead4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:03.217056 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2474639232}
I20260709 13:07:03.434425 21214 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:07:03.434909 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 186414us
I20260709 13:07:03.435568 21214 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:03.435832 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:03.286610 23884 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.959s	user 0.004s	sys 0.000s
I20260709 13:07:03.427172 21063 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:03.281298 23683 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.469578 21063 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:03.280931 23373 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:03.489293 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54947.2us
I20260709 13:07:04.102339 23826 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:04.105652 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.939867 21113 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.003s	sys 0.026s
W20260709 13:07:04.106731 23756 consensus_meta.cc:287] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.870s	user 0.008s	sys 0.008s
I20260709 13:07:03.416837 23660 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:04.106906 21113 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.003s	sys 0.030s
W20260709 13:07:04.122731 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 898353us
I20260709 13:07:03.549837 21798 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:07:03.466598 23670 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:03.410283 23663 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:04.132270 21216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.902s	user 0.022s	sys 0.018s
W20260709 13:07:04.139511 21216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.022s	sys 0.018s
I20260709 13:07:03.487629 23761 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:03.547188 23868 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:04.108047 23901 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:03.942454 23616 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:04.133920 21214 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.698s	user 0.010s	sys 0.000s
W20260709 13:07:04.120043 23808 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.149s	user 0.008s	sys 0.008s
I20260709 13:07:04.151615 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 846) took 1446 ms. Trace:
W20260709 13:07:03.567569 23916 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.399s	user 0.001s	sys 0.000s
I20260709 13:07:03.423965 23817 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:04.102984 23826 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:03.266849 23695 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:03.913077 23888 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:03.646260 23918 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:04.108197 23451 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.166s	user 0.009s	sys 0.005s
I20260709 13:07:04.133358 23756 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.152050 21218 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:07:04.152244 23901 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.487422 23769 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.658s
I20260709 13:07:03.534284 23813 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.613s
I20260709 13:07:04.140527 23670 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.549301 23532 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.603s
I20260709 13:07:04.156104 21214 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:04.158078 23835 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:03.595829 21794 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:04.145804 23761 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:04.152643 23748 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.041s	user 0.010s	sys 0.003s
I20260709 13:07:04.194139 21798 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:03.628214 23777 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:04.146796 23868 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:04.178851 23922 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.196043 23769 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:04.152892 21219 rpcz_store.cc:276] 0709 13:07:02.705488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.705650 (+   162us) service_pool.cc:224] Handling call
0709 13:07:04.151384 (+1445734us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b5400102554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:04.151549 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538284800}
I20260709 13:07:04.130223 23683 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.149785 23373 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:04.190162 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.007s	sys 0.018s
I20260709 13:07:04.184895 23670 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:04.154645 23808 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.156523 23616 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.138223 23660 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:04.189425 23876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.023s	sys 0.063s
I20260709 13:07:04.194447 23761 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.180490 23888 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:04.224258 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.007s	sys 0.019s
I20260709 13:07:04.168694 23695 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:04.223397 23863 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.484s	user 0.005s	sys 0.009s
W20260709 13:07:04.238807 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 790698us
W20260709 13:07:04.200565 23485 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.320s	user 0.014s	sys 0.000s
I20260709 13:07:04.182317 23918 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.196946 23924 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.189692 23901 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.196331 23835 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:07:04.237637 23876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.999s	user 0.023s	sys 0.063s
I20260709 13:07:04.198540 23748 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.221504 23769 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.232710 23858 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:04.214694 23777 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.264246 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 858) took 1552 ms. Trace:
I20260709 13:07:04.220094 23922 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.230012 23373 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.278384 21215 rpcz_store.cc:276] 0709 13:07:02.712177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.826260 (+114083us) service_pool.cc:224] Handling call
0709 13:07:02.838796 (+ 12536us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:02.876701 (+ 37905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.876731 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:02.886660 (+  9929us) log.cc:844] Serialized 52 byte log entry
0709 13:07:03.079254 (+192594us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:03.079327 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.079399 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.207961 (+128562us) spinlock_profiling.cc:243] Waited 900 us on lock 0x7b4800281958. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:07:04.245742 (+1037781us) raft_consensus.cc:1685] finished
0709 13:07:04.245769 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.264143 (+ 18374us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15678,"raft.run_cpu_time_us":7005,"raft.run_wall_time_us":327206,"spinlock_wait_cycles":2069888,"thread_start_us":169112,"threads_started":1,"wal-append.queue_time_us":230236}
I20260709 13:07:04.283846 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:04.290865 21220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 855) took 1582 ms (client timeout 1526 ms). Trace:
W20260709 13:07:04.283425 21070 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.036s	user 0.014s	sys 0.034s
W20260709 13:07:04.291587 21070 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.014s	sys 0.034s
I20260709 13:07:04.291895 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:04.291390 21220 rpcz_store.cc:269] 0709 13:07:02.708153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.708787 (+   634us) service_pool.cc:224] Handling call
0709 13:07:04.290722 (+1581935us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540015c054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:04.290809 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244712704}
W20260709 13:07:04.304937 21066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 813) took 2624 ms (client timeout 1629 ms). Trace:
I20260709 13:07:04.289875 21142 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.221755 23663 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.275733 23451 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.310480 21141 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:07:04.311546 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:04.281385 23876 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:04.231101 23817 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:04.305492 21066 rpcz_store.cc:269] 0709 13:07:01.664531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.664810 (+   279us) service_pool.cc:224] Handling call
0709 13:07:04.288366 (+2623556us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54003ca354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:04.288514 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2281144704}
I20260709 13:07:04.256719 23485 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.247229 23863 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.314160 21142 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.319937 23485 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.320303 23777 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.320542 23485 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.320838 23777 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.321291 23777 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.269235 23868 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.323778 23868 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:04.324963 23927 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:07:04.325744 21142 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
W20260709 13:07:04.326046 23793 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.084s	user 0.009s	sys 0.004s
W20260709 13:07:04.326342 23923 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:07:04.326390 21142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1982) took 2477 ms (client timeout 2126 ms). Trace:
W20260709 13:07:04.326894 21142 rpcz_store.cc:269] 0709 13:07:01.848949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.975457 (+126508us) service_pool.cc:224] Handling call
0709 13:07:04.289556 (+2314099us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400245ca0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4249a06f 00007f7f42512f60 00007f7f425147de 00007f7f4252027c 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2
0709 13:07:04.326310 (+ 36754us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619444608}
I20260709 13:07:04.327594 23817 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:04.326918 21221 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:07:04.328465 21221 pending_rounds.cc:85] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 5
I20260709 13:07:04.328692 21221 pending_rounds.cc:107] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:07:04.328914 21221 raft_consensus.cc:2889] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:04.328955 21138 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:07:04.330238 23876 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.331667 23918 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.331946 23769 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.333341 23876 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.333843 23695 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.334136 23748 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.334445 23695 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.334980 23695 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.337316 23695 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:04.338315 23696 consensus_meta.cc:287] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.100s	user 0.011s	sys 0.002s
I20260709 13:07:04.339182 21071 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:04.339926 23922 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.340201 23901 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.340366 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 343) took 1095 ms. Trace:
I20260709 13:07:04.340732 21071 rpcz_store.cc:276] 0709 13:07:03.245361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.246055 (+   694us) service_pool.cc:224] Handling call
0709 13:07:04.141488 (+895433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.339563 (+198075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.340117 (+   554us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54003bf954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:04.340284 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453720832}
I20260709 13:07:04.341480 23888 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.269892 23924 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.308590 23933 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.306388 23929 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.313060 23796 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.282845 23858 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:04.349784 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.350524 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 815) took 2416 ms. Trace:
I20260709 13:07:04.313601 23756 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.351229 23924 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:04.351428 23869 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.896s	user 0.006s	sys 0.028s
I20260709 13:07:04.352473 23808 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.352885 23918 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:04.348132 23936 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:07:04.353134 23926 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:04.361980 23826 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.358741 23933 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.355322 23683 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.395211 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 282) took 1161 ms. Trace:
I20260709 13:07:04.396667 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2454) took 1488 ms. Trace:
I20260709 13:07:04.353240 23373 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.360397 23777 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.352891 23895 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:04.354285 23871 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:04.359030 21065 rpcz_store.cc:276] 0709 13:07:01.934042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.934438 (+   396us) service_pool.cc:224] Handling call
0709 13:07:01.934842 (+   404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:02.015839 (+ 80997us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:07:02.015884 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:02.016659 (+   775us) log.cc:844] Serialized 52 byte log entry
0709 13:07:02.139952 (+123293us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:02.139996 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.342189 (+2202193us) spinlock_profiling.cc:243] Waited 80.4 ms on lock 0x7b540011b7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:04.342243 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:04.342285 (+    42us) raft_consensus.cc:1685] finished
0709 13:07:04.342300 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.350456 (+  8156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7796,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"spinlock_wait_cycles":184980352,"thread_start_us":122474,"threads_started":1,"wal-append.queue_time_us":664297,"wal-append.run_cpu_time_us":3467,"wal-append.run_wall_time_us":1394463}
I20260709 13:07:04.344705 21220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
W20260709 13:07:04.387372 23654 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.230s	user 0.009s	sys 0.004s
W20260709 13:07:04.367934 23874 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.912s	user 0.014s	sys 0.022s
I20260709 13:07:04.369282 23660 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.369172 23835 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.356052 23876 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.352937 23863 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.362191 23929 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.392601 23616 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.407033 23895 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:04.358347 23769 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:04.397078 21218 rpcz_store.cc:276] 0709 13:07:02.908699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.951911 (+ 43212us) service_pool.cc:224] Handling call
0709 13:07:04.151774 (+1199863us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:04.181152 (+ 29378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.396456 (+215304us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400102550. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:04.396608 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2758673280}
I20260709 13:07:04.344956 23930 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.321034 23485 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.397070 21221 rpcz_store.cc:276] 0709 13:07:03.233317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.239499 (+  6182us) service_pool.cc:224] Handling call
0709 13:07:03.239781 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.329566 (+1089785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.394798 (+ 65232us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54003fa7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:04.395123 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2499394432}
W20260709 13:07:04.402993 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.403926 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 857) took 1695 ms. Trace:
I20260709 13:07:04.427618 21217 rpcz_store.cc:276] 0709 13:07:02.708688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.754559 (+ 45871us) service_pool.cc:224] Handling call
0709 13:07:02.756963 (+  2404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.866984 (+110021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.403666 (+1536682us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b54004443d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:04.403867 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190071040}
I20260709 13:07:04.364359 23922 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.315196 23451 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.408004 21220 raft_consensus.cc:606] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:04.430275 21220 raft_consensus.cc:614] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:04.392825 23936 consensus_queue.cc:1243] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:04.363734 23796 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.407582 23871 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:04.422127 21064 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:04.418565 23876 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.419459 23933 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.412186 23654 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.446584 23895 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.448109 23895 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:04.426643 23769 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.452442 23769 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.453207 23796 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.456490 23654 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.465392 21143 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:04.435441 23873 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.455703 23858 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.464599 23929 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:04.510350 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2622) took 1601 ms (client timeout 1565 ms). Trace:
I20260709 13:07:04.513168 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:04.510962 21140 rpcz_store.cc:269] 0709 13:07:02.908996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.909152 (+   156us) service_pool.cc:224] Handling call
0709 13:07:04.510111 (+1600959us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b540030aa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:04.510259 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439124352}
I20260709 13:07:04.516148 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.516767 21800 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
W20260709 13:07:04.534175 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 242347us
I20260709 13:07:04.430012 23930 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.539335 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:04.538280 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.541649 23796 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:04.542671 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 250729us
I20260709 13:07:04.542418 23796 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.542932 23929 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.543428 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:04.477398 23513 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:07:04.545192 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.443490 23922 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.544672 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:04.551198 21801 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:07:04.535181 23938 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:07:04.543637 23826 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:04.515295 23654 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:04.554700 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 865) took 1716 ms. Trace:
I20260709 13:07:04.557029 21216 rpcz_store.cc:276] 0709 13:07:02.838518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.899028 (+ 60510us) service_pool.cc:224] Handling call
0709 13:07:02.909392 (+ 10364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.950122 (+ 40730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.950170 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:02.950704 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:07:03.230590 (+279886us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b1800393a00. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:07:04.140080 (+909490us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:04.516355 (+376275us) log.cc:844] Serialized 14 byte log entry
0709 13:07:04.546640 (+ 30285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.546739 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:04.546795 (+    56us) raft_consensus.cc:1685] finished
0709 13:07:04.554317 (+  7522us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.554508 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65181,"raft.queue_time_us":70545,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":202,"spinlock_wait_cycles":7444736,"thread_start_us":1274260,"threads_started":2,"wal-append.queue_time_us":1261539}
I20260709 13:07:04.565135 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.565948 21800 raft_consensus.cc:2368] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10.
I20260709 13:07:04.537983 23645 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.567973 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.551435 23513 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:04.536820 23616 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:07:04.473097 23873 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.617306 23949 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:04.618521 23939 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.616714 23942 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
W20260709 13:07:04.617009 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 47206.8us
I20260709 13:07:04.616880 23934 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:07:04.570732 23645 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:04.597337 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:04.621331 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 76743.9us
I20260709 13:07:04.596330 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.622964 21140 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:07:04.623580 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:04.593922 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:04.632879 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:04.633352 21142 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:04.634197 21184 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:04.634672 21065 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:04.634610 23513 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.635337 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:04.635787 23513 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:04.636031 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:04.636587 21072 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.636861 23793 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:04.553854 23817 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.638092 23817 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.626178 21141 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:07:04.639647 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:04.639678 23934 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:04.640432 21071 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:04.640820 23934 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:04.641249 23934 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.641505 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:04.642302 21069 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:04.642742 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.643546 21220 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 2.
I20260709 13:07:04.643592 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:04.645926 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.649042 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:04.649698 21141 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 2.
I20260709 13:07:04.641620 23645 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.652824 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.602788 23888 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.653445 21221 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:04.654381 23888 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.656039 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.656757 23888 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.658367 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.659090 21221 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:04.662210 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.641615 23934 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:04.665360 21072 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:07:04.667032 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:04.682577 21142 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.682746 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:04.667840 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:04.668080 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:04.687556 21140 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:04.669166 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:04.669441 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:04.671983 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.689188 21221 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:04.681689 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.672501 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:04.690887 21141 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
I20260709 13:07:04.689589 21216 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:04.704308 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71538.4us
I20260709 13:07:04.704839 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 344) took 1459 ms. Trace:
I20260709 13:07:04.554304 23373 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:04.706836 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.718137 21218 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:04.712435 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:04.712916 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:04.707741 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:04.727169 21141 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:04.723332 21221 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:04.606535 23863 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:04.729394 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:04.571797 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.615262 23695 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:04.729415 21219 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:07:04.730953 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:04.748769 23670 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.003s
I20260709 13:07:04.731734 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.776321 21218 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:04.562722 23777 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.615653 23901 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:04.616107 23796 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.550760 23868 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.154s
I20260709 13:07:04.624531 23945 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.603348 23769 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 13:07:04.739180 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:04.553629 23929 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.739904 21067 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:04.811929 21072 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:07:04.557868 23933 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.154s
I20260709 13:07:04.690716 23660 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.553681 23871 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.764084 23451 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.752689 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:04.837977 21141 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:04.714368 23485 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.711854 23926 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.753151 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.567240 23654 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.705981 23835 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.754868 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 347) took 1336 ms. Trace:
I20260709 13:07:04.854127 21068 rpcz_store.cc:276] 0709 13:07:03.418478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.418747 (+   269us) service_pool.cc:224] Handling call
0709 13:07:03.488591 (+ 69844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.235990 (+747399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:04.236030 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:04.236084 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:04.236135 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.754597 (+518462us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54000dd254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:04.754656 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.754794 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":846,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":4839,"spinlock_wait_cycles":1717976448}
I20260709 13:07:04.855686 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:04.856292 21068 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:04.760300 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.714671 23373 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.724009 23683 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:04.763355 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 345) took 1515 ms. Trace:
I20260709 13:07:04.568960 23826 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:04.764299 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 211425us
I20260709 13:07:04.705665 23935 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:04.765961 21142 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
I20260709 13:07:04.767172 21146 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:07:04.592048 23764 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.154s
I20260709 13:07:04.768160 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2609) took 3166 ms. Trace:
I20260709 13:07:04.768388 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:04.746392 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 176279us
I20260709 13:07:04.769487 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.728929 23696 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:04.759662 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.628120 23761 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.787014 21216 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:07:04.787869 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:04.750834 23924 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.810236 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.893272 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.893814 21793 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:04.893924 21796 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:04.748589 23887 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.554092 23936 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.158s
I20260709 13:07:04.881124 21145 rpcz_store.cc:276] 0709 13:07:01.587639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.587775 (+   136us) service_pool.cc:224] Handling call
0709 13:07:02.751970 (+1164195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:02.827894 (+ 75924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.753931 (+1926037us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400447ad0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:04.754107 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2638407040}
I20260709 13:07:04.797257 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:04.728717 23808 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:07:04.691779 23939 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.705201 23756 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:04.769709 23888 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.752329 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:04.947916 21069 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:04.820477 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:04.869165 23957 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:04.771728 23876 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:04.769157 23793 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:04.770748 23951 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:04.811975 23670 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:04.850630 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 258766us
I20260709 13:07:04.825698 23929 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:04.827270 23645 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:04.859388 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:04.934314 23683 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:04.865864 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.867878 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:04.848455 23947 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.895193 21146 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:04.810520 23695 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.897837 21110 leader_election.cc:400] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10.
I20260709 13:07:04.898705 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:04.899516 23941 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:07:04.900200 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:04.900738 23868 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:04.901214 23532 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:04.902622 23918 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.903980 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:04.906502 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:04.907982 21063 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.861378 23933 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:04.912539 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:04.917263 23934 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:04.918009 23953 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:07:04.918984 23946 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:07:04.919396 23952 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:04.868673 21064 rpcz_store.cc:276] 0709 13:07:03.247475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.247622 (+   147us) service_pool.cc:224] Handling call
0709 13:07:03.286642 (+ 39020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.440683 (+1154041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.753800 (+313117us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400215054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:04.763250 (+  9450us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2610930432}
I20260709 13:07:04.873754 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:04.925134 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:04.881963 21799 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:04.888046 21797 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:04.890188 21798 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:04.869664 21068 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:05.110795 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:05.111737 21068 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:04.810750 23777 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.872251 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:04.931885 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 134747us
W20260709 13:07:05.113049 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 206623us
W20260709 13:07:05.113731 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.815925 23901 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.868829 23373 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:04.838199 23660 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:04.935212 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 229775us
I20260709 13:07:04.933014 23887 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.937588 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:04.938839 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.949039 21219 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:04.838701 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 739) took 1560 ms. Trace:
I20260709 13:07:05.132686 21795 rpcz_store.cc:276] 0709 13:07:03.265578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.265879 (+   301us) service_pool.cc:224] Handling call
0709 13:07:04.108254 (+842375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:04.235356 (+127102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:04.235391 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:04.246601 (+ 11210us) log.cc:844] Serialized 26 byte log entry
0709 13:07:04.625108 (+378507us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:04.625156 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.625221 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:04.825982 (+200761us) raft_consensus.cc:1685] finished
0709 13:07:04.826003 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.826132 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":162501,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":12726,"thread_start_us":461302,"threads_started":2,"wal-append.queue_time_us":492780}
I20260709 13:07:04.890802 23858 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.821666 23945 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.889801 23761 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.960553 21144 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:04.861105 23485 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:07:04.938125 23865 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:04.931749 23943 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:04.862396 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:04.935026 23922 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:04.880954 23764 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:07:04.887593 23696 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:05.097361 23513 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:04.929741 23924 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.856992 23926 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:05.095481 21138 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:05.051585 21800 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:05.096366 21110 leader_election.cc:403] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:05.004987 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.096897 21796 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:05.153147 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:05.154196 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:05.153595 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:05.156891 21064 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:05.149030 21795 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:07:05.098865 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:05.060117 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 124341us
I20260709 13:07:05.095845 21063 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:05.104612 21143 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:04.976809 21141 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:05.108780 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:05.115423 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:04.934708 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:04.964203 23876 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.960716 23951 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:05.129043 21221 raft_consensus.cc:2368] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6.
I20260709 13:07:05.128929 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:05.165179 21218 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:05.129706 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:05.165989 21072 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:05.088738 23960 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:05.130122 21140 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:05.177239 21219 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:07:05.025804 23451 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.108707 23929 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.095330 23933 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:04.869603 23835 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:05.116241 23950 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:07:04.960870 23888 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.115408 23616 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:05.153159 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:05.162225 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 228897us
I20260709 13:07:05.107090 23868 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:04.972326 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:05.130736 23966 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.182760 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:05.025766 23683 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:05.119184 23918 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:04.960547 23957 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.973362 23645 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:05.147361 23793 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.108771 23947 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:05.186223 21218 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:05.150120 23959 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.655820 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:05.657214 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:05.191428 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:05.163308 23943 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.654968 21799 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:05.147220 23865 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.658221 21143 raft_consensus.cc:2410] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:07:05.599131 21219 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:07:05.614144 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.000s	sys 0.018s
W20260709 13:07:05.644381 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.030s	sys 0.101s
W20260709 13:07:05.708106 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.031s	sys 0.101s
I20260709 13:07:05.709295 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:05.711438 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:05.712571 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:05.236868 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:05.714277 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.659488 21793 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:05.662444 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:05.242285 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:05.304596 21141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:05.251199 23926 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:05.178786 23513 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.668577 21139 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
W20260709 13:07:05.737448 21138 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.002s
I20260709 13:07:05.737944 21138 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:05.671141 23835 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:05.255462 23670 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:05.741135 23670 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:07:05.482174 23965 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:07:05.742018 23670 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.742936 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 288) took 1066 ms. Trace:
I20260709 13:07:05.182173 23922 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.252667 23777 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:05.245106 23660 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:05.713984 23865 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:05.766681 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.013s	sys 0.015s
W20260709 13:07:05.768950 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.013s	sys 0.016s
I20260709 13:07:05.684758 23645 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:05.685892 23933 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.254216 23924 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:05.750540 23808 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.753571 23826 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:05.600566 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 286) took 2365 ms. Trace:
I20260709 13:07:05.760051 23942 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.710654 23451 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.655429 23959 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:05.708043 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.000s	sys 0.020s
I20260709 13:07:05.715615 21141 raft_consensus.cc:606] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:05.715860 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2464) took 1026 ms. Trace:
I20260709 13:07:05.658577 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:05.163375 23948 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:05.635532 23532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.010s
W20260709 13:07:05.720716 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58406.1us
I20260709 13:07:05.814508 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:05.164023 23873 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:05.739466 21138 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:05.472786 23961 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.547s
W20260709 13:07:05.741425 21072 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.000s
W20260709 13:07:05.223109 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 325448us
I20260709 13:07:05.713753 23929 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.541039 23952 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:05.674211 23868 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:05.175082 23960 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:05.743352 21212 rpcz_store.cc:276] 0709 13:07:04.676376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.676549 (+   173us) service_pool.cc:224] Handling call
0709 13:07:04.676935 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:04.704319 (+ 27384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:04.704355 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:04.725325 (+ 20970us) log.cc:844] Serialized 26 byte log entry
0709 13:07:04.940127 (+214802us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:04.940185 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.940282 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.742693 (+802411us) raft_consensus.cc:1685] finished
0709 13:07:05.742714 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.742876 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4218,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":16082,"thread_start_us":173244,"threads_started":1,"wal-append.queue_time_us":384275}
I20260709 13:07:05.655042 23616 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10.
I20260709 13:07:05.749164 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:05.842167 21064 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:05.842844 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 838) took 1058 ms. Trace:
I20260709 13:07:05.843225 21064 rpcz_store.cc:276] 0709 13:07:04.784387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.717996 (+933609us) service_pool.cc:224] Handling call
0709 13:07:05.842771 (+124775us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:05.749519 23934 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:05.751336 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1788) took 1037 ms. Trace:
I20260709 13:07:05.752228 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1996) took 1057 ms. Trace:
I20260709 13:07:05.644132 23485 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.152227 23858 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.678567 23918 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:05.249461 23696 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:05.656946 23901 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:05.246861 23945 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:05.258492 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:05.736126 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.871268 21142 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:05.221755 23936 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.534s
W20260709 13:07:05.770720 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 644024us
W20260709 13:07:05.774248 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.011s	sys 0.019s
I20260709 13:07:05.722681 23654 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:05.753263 23670 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:05.689270 23683 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:05.691481 23764 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:07:05.763904 21140 consensus_meta.cc:287] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.588s	user 0.009s	sys 0.003s
I20260709 13:07:05.881564 21140 raft_consensus.cc:2410] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:07:05.682281 23957 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:05.768965 23748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.017s	sys 0.007s
I20260709 13:07:05.801177 21213 rpcz_store.cc:276] 0709 13:07:03.235103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.578371 (+343268us) service_pool.cc:224] Handling call
0709 13:07:03.578793 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:03.584896 (+  6103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:03.584943 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:04.115681 (+530738us) log.cc:844] Serialized 26 byte log entry
0709 13:07:04.529842 (+414161us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:04.529904 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.529998 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.406104 (+876106us) raft_consensus.cc:1685] finished
0709 13:07:05.406132 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.600485 (+194353us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":146,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":225,"thread_start_us":372130,"threads_started":1,"wal-append.queue_time_us":580571}
W20260709 13:07:05.699954 21795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.000s	sys 0.011s
W20260709 13:07:05.780198 23937 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.004s	sys 0.017s
I20260709 13:07:05.747565 23939 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:05.808638 21219 rpcz_store.cc:276] 0709 13:07:04.689217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.183395 (+494178us) service_pool.cc:224] Handling call
0709 13:07:05.598732 (+415337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.707452 (+108720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.715792 (+  8340us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:05.808028 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 694278us
W20260709 13:07:05.813948 23532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.000s	sys 0.012s
W20260709 13:07:05.787799 23949 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.006s	sys 0.017s
I20260709 13:07:05.785919 23645 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.182310 23876 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:05.816934 23948 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:05.821208 21072 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:05.919443 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1801) took 1198 ms. Trace:
I20260709 13:07:05.919766 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:05.919873 21072 rpcz_store.cc:276] 0709 13:07:04.720959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.097054 (+376095us) service_pool.cc:224] Handling call
0709 13:07:05.919383 (+822329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:05.920421 21213 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:05.920827 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:05.758458 23922 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.877624 23901 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:05.909188 23937 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.005s	sys 0.017s
I20260709 13:07:05.824414 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1998) took 1100 ms. Trace:
W20260709 13:07:05.908980 23748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.018s	sys 0.007s
I20260709 13:07:05.839139 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:05.862287 23485 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.809185 23826 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:05.856223 23983 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:05.856467 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:05.861977 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:05.857460 21071 rpcz_store.cc:276] 0709 13:07:04.714088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.729960 (+ 15872us) service_pool.cc:224] Handling call
0709 13:07:04.730233 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.061673 (+331440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.061718 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.095722 (+ 34004us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.449428 (+353706us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:05.449477 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.449549 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.751101 (+301552us) raft_consensus.cc:1685] finished
0709 13:07:05.751123 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.751262 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107195,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10993,"thread_start_us":618281,"threads_started":2,"wal-append.queue_time_us":619445}
I20260709 13:07:05.857671 21139 rpcz_store.cc:276] 0709 13:07:04.695772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.949586 (+253814us) service_pool.cc:224] Handling call
0709 13:07:05.752005 (+802419us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:05.820604 23981 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.869042 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:05.956453 21801 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:05.825927 23873 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.804778 23808 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:05.816717 23978 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:07:05.874378 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.011s	sys 0.019s
I20260709 13:07:05.874512 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:05.876235 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1994) took 1204 ms. Trace:
I20260709 13:07:05.834746 23952 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:05.880254 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:05.864935 23971 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:07:05.897295 21111 leader_election.cc:336] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.526s (SENT)
I20260709 13:07:05.976822 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:05.907387 21795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.000s	sys 0.012s
I20260709 13:07:05.865135 23761 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:07:05.858023 23980 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:05.811170 23942 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.824288 23933 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:05.921573 21072 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:06.000895 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 840) took 1215 ms. Trace:
I20260709 13:07:06.001459 21072 rpcz_store.cc:276] 0709 13:07:04.784990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.920601 (+1135611us) service_pool.cc:224] Handling call
0709 13:07:06.000805 (+ 80204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:05.922587 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.002887 21141 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
W20260709 13:07:05.930753 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:05.829154 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.934260 21138 rpcz_store.cc:276] 0709 13:07:04.726150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.738774 (+1012624us) service_pool.cc:224] Handling call
0709 13:07:05.739226 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.817449 (+ 78223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.824354 (+  6905us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:05.807152 23769 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.053s	sys 0.124s
W20260709 13:07:05.947731 21063 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.789s	user 0.006s	sys 0.006s
I20260709 13:07:05.710263 23966 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:05.914800 23949 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.006s	sys 0.017s
I20260709 13:07:05.957820 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:05.968240 23978 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.883116 23868 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:05.961547 21215 raft_consensus.cc:2376] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8. Re-sending same reply.
I20260709 13:07:06.030660 21213 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
W20260709 13:07:05.969142 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 94714.8us
I20260709 13:07:05.969316 21142 rpcz_store.cc:276] 0709 13:07:04.671329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.178709 (+507380us) service_pool.cc:224] Handling call
0709 13:07:05.876168 (+697459us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:05.968606 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 1.09635e+06us
I20260709 13:07:06.033571 21767 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:05.972106 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49627.1us
I20260709 13:07:06.014132 23986 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:06.014858 23769 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.053s	sys 0.125s
I20260709 13:07:05.979641 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1796) took 1259 ms. Trace:
I20260709 13:07:05.882951 23987 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.124s
W20260709 13:07:05.988649 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 91407.7us
W20260709 13:07:06.094460 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.990173 21212 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:07:05.954383 23979 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:06.008598 21063 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:07:05.959362 23983 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:06.013317 23971 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:06.013626 23948 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:06.014864 21071 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:06.103426 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.103933 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 837) took 1319 ms. Trace:
I20260709 13:07:06.020658 23976 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.021413 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:06.040489 23683 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:07:06.023732 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:06.024679 23974 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.208s	user 0.003s	sys 0.000s
I20260709 13:07:06.025807 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:06.025983 23966 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:05.978857 23952 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:05.972015 23936 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:05.879001 23934 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.967253 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:05.913096 23887 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:06.038411 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:05.977144 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:05.951486 23645 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.959896 23901 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:04.705235 21070 rpcz_store.cc:276] 0709 13:07:03.245699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.246079 (+   380us) service_pool.cc:224] Handling call
0709 13:07:03.246441 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:03.246678 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:04.292223 (+1045545us) log.cc:844] Serialized 14 byte log entry
0709 13:07:04.704467 (+412244us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:04.704530 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:04.704581 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.704652 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.704784 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51602,"prepare.queue_time_us":1095761,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":48,"raft.queue_time_us":62939,"thread_start_us":1405147,"threads_started":2}
I20260709 13:07:06.010632 23943 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:05.968573 23826 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:05.973296 23873 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.095436 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2465) took 1405 ms. Trace:
I20260709 13:07:06.031561 23978 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:06.097306 21063 pending_rounds.cc:85] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 7
I20260709 13:07:06.096709 23748 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.051834 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:05.968961 23981 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:05.974310 23960 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.092056 21064 rpcz_store.cc:276] 0709 13:07:04.719824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.867058 (+1147234us) service_pool.cc:224] Handling call
0709 13:07:05.979554 (+112496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.340484 21069 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:06.341630 21141 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:06.341984 21063 pending_rounds.cc:107] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:07:06.342443 21063 raft_consensus.cc:2889] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:06.342592 21139 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:06.113857 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 295) took 1546 ms. Trace:
I20260709 13:07:06.119988 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:06.211719 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.341652 21069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1809) took 1449 ms (client timeout 1924 ms). Trace:
I20260709 13:07:06.117565 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:06.104293 21071 rpcz_store.cc:276] 0709 13:07:04.784042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.957584 (+1173542us) service_pool.cc:224] Handling call
0709 13:07:06.103873 (+146289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.343852 21800 rpcz_store.cc:276] 0709 13:07:04.567397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.567658 (+   261us) service_pool.cc:224] Handling call
0709 13:07:06.113593 (+1545935us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b5400662bd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:06.113773 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":996868480}
W20260709 13:07:06.279588 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.336619 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:06.221859 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:06.344964 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.344561 21068 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:06.345788 21067 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
W20260709 13:07:06.345222 21069 rpcz_store.cc:269] 0709 13:07:04.892497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.025516 (+1133019us) service_pool.cc:224] Handling call
0709 13:07:06.341572 (+316056us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.345446 21218 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:06.348500 21067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 829) took 1602 ms (client timeout 2091 ms). Trace:
W20260709 13:07:06.348961 21067 rpcz_store.cc:269] 0709 13:07:04.746180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.117231 (+1371051us) service_pool.cc:224] Handling call
0709 13:07:06.348424 (+231193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:06.213668 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.344141 21212 rpcz_store.cc:276] 0709 13:07:04.689597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.969750 (+1280153us) service_pool.cc:224] Handling call
0709 13:07:05.989788 (+ 20038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:06.095095 (+105307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.095374 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.104406 21142 raft_consensus.cc:2410] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:07:06.342594 21213 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:07:06.352515 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2466) took 1618 ms. Trace:
I20260709 13:07:06.343953 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 348) took 2917 ms. Trace:
W20260709 13:07:06.354452 21072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1802) took 1633 ms (client timeout 2042 ms). Trace:
W20260709 13:07:06.355077 21072 rpcz_store.cc:269] 0709 13:07:04.721182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.029058 (+1307876us) service_pool.cc:224] Handling call
0709 13:07:06.354360 (+325302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.355374 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:06.345409 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.356230 21070 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:06.356758 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 833) took 1574 ms. Trace:
I20260709 13:07:06.357070 21070 rpcz_store.cc:276] 0709 13:07:04.782809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.355089 (+1572280us) service_pool.cc:224] Handling call
0709 13:07:06.356706 (+  1617us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:06.357999 21068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 827) took 1625 ms (client timeout 2061 ms). Trace:
W20260709 13:07:06.358456 21068 rpcz_store.cc:269] 0709 13:07:04.733000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.021046 (+1288046us) service_pool.cc:224] Handling call
0709 13:07:06.357931 (+336885us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.359179 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:06.100023 23977 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:07:06.361183 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:06.361166 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:06.379041 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:06.364846 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:06.380681 23532 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:06.097157 23979 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:06.381213 23532 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:06.381888 23979 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.382252 23532 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.382944 23979 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:06.383566 23957 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:06.384954 23991 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:07:06.386190 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:06.386899 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:06.387559 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.387756 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1992) took 1717 ms. Trace:
I20260709 13:07:06.361760 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.387586 21212 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:06.385411 23929 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:06.031819 23947 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:06.353655 21063 rpcz_store.cc:276] 0709 13:07:03.426455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.426758 (+   303us) service_pool.cc:224] Handling call
0709 13:07:03.427058 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.907525 (+1480467us) spinlock_profiling.cc:243] Waited 83.2 ms on lock 0x7b54001eeea0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4249a06f 00007f7f42512f60 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62
0709 13:07:06.343041 (+1435516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.343262 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191256832}
W20260709 13:07:06.411728 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46943us
I20260709 13:07:06.404333 21142 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:06.099998 23983 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:06.411500 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:06.423015 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:06.352972 21213 rpcz_store.cc:276] 0709 13:07:04.733918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.031573 (+1297655us) service_pool.cc:224] Handling call
0709 13:07:06.342350 (+310777us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:06.352223 (+  9873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.352457 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.399806 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2001) took 1648 ms. Trace:
I20260709 13:07:06.388268 21143 rpcz_store.cc:276] 0709 13:07:04.670807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.718083 (+1047276us) service_pool.cc:224] Handling call
0709 13:07:05.749945 (+ 31862us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:05.975550 (+225605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.975581 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:05.983926 (+  8345us) log.cc:844] Serialized 52 byte log entry
0709 13:07:06.280724 (+296798us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:06.280778 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.280874 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.387559 (+106685us) raft_consensus.cc:1685] finished
0709 13:07:06.387577 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.387703 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37513,"raft.queue_time_us":125006,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":33379,"thread_start_us":486474,"threads_started":2,"wal-append.queue_time_us":365156}
I20260709 13:07:06.396777 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:06.424453 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2006) took 1611 ms. Trace:
I20260709 13:07:06.424612 21219 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:06.423230 21064 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
W20260709 13:07:06.358150 23813 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:07:06.293548 23761 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:06.311630 23696 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:06.097880 23980 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:06.424060 21138 rpcz_store.cc:276] 0709 13:07:04.751439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.398921 (+1647482us) service_pool.cc:224] Handling call
0709 13:07:06.399215 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.399435 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:06.399461 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.399522 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:06.399555 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.399608 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.399746 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.261926 23945 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:06.435447 21111 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.805s (SENT)
I20260709 13:07:06.439720 23826 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:06.353108 23934 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.389583 23971 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.424916 21142 rpcz_store.cc:276] 0709 13:07:04.812790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.401176 (+1588386us) service_pool.cc:224] Handling call
0709 13:07:06.403909 (+  2733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.422737 (+ 18828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.424395 (+  1658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.358310 23976 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.366566 23936 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:06.399515 23960 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:06.350036 23888 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:06.220997 23948 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:06.412330 23922 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:06.426842 21185 connection.cc:256] RPC call timeout handler was delayed by -1.1963250569999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
W20260709 13:07:06.441848 21064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1797) took 1721 ms (client timeout 2177 ms). Trace:
W20260709 13:07:06.502944 21064 rpcz_store.cc:269] 0709 13:07:04.720048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.389850 (+1669802us) service_pool.cc:224] Handling call
0709 13:07:06.441769 (+ 51919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.400300 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 1990) took 1730 ms. Trace:
I20260709 13:07:06.451748 23930 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:06.509517 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:06.438455 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:06.352975 23748 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.353956 23808 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:06.468050 21143 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:07:06.469091 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 435584us
I20260709 13:07:06.470901 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:06.377519 23998 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:06.483230 21142 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:06.481178 21218 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 3.
W20260709 13:07:06.465710 23895 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 2.017s	user 0.015s	sys 0.001s
I20260709 13:07:06.491932 23985 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:06.451807 23961 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:06.453394 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:06.508808 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:06.504206 21147 rpcz_store.cc:276] 0709 13:07:04.670317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.251997 (+581680us) service_pool.cc:224] Handling call
0709 13:07:05.252427 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.252870 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.252910 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.657340 (+404430us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.865175 (+207835us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:05.865226 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.865294 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.400116 (+534822us) raft_consensus.cc:1685] finished
0709 13:07:06.400135 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.400247 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":507572,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":10613,"thread_start_us":116352,"threads_started":1,"wal-append.queue_time_us":278781}
I20260709 13:07:06.527843 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2007) took 1689 ms. Trace:
I20260709 13:07:06.550467 21143 rpcz_store.cc:276] 0709 13:07:04.838849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.424324 (+1585475us) service_pool.cc:224] Handling call
0709 13:07:06.467634 (+ 43310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.527581 (+ 59947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.527787 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:06.532871 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.533056 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.536262 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 65458us
W20260709 13:07:06.551503 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.551275 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:06.466882 23761 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.478603 23696 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.553922 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:06.553421 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.506594 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 643) took 1682 ms. Trace:
I20260709 13:07:06.557575 21218 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:06.559520 21217 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:06.566327 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:06.490092 23826 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:06.566277 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.569203 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.558331 21063 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:07:06.540020 23922 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:06.552803 23865 consensus_meta.cc:287] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.003s
I20260709 13:07:06.506327 23485 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:06.535626 23876 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:06.564471 23793 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.688649 23793 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.593044 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:06.689970 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:06.549144 23961 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.556239 24002 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:06.594403 23922 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.595083 23980 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.525707 23948 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.548563 23998 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:06.597900 23748 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:06.566636 23373 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.427s	user 0.011s	sys 0.006s
I20260709 13:07:06.718626 23373 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.719213 23948 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:06.721309 23373 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:06.598861 23901 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:06.694890 23961 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:06.602646 23942 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:06.539829 23887 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:06.572846 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1807) took 1850 ms (client timeout 2176 ms). Trace:
W20260709 13:07:06.733119 21063 rpcz_store.cc:269] 0709 13:07:04.722305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.422706 (+1700401us) service_pool.cc:224] Handling call
0709 13:07:06.572791 (+150085us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:06.632258 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:06.734442 21212 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 5.
I20260709 13:07:06.631925 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:06.490801 23945 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.562358 23957 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:06.647498 21792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48444 (request call id 0) took 1764 ms. Trace:
I20260709 13:07:06.651666 23817 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:06.652045 23936 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.653604 23952 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.565338 21794 rpcz_store.cc:276] 0709 13:07:04.812427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.825032 (+ 12605us) service_pool.cc:224] Handling call
0709 13:07:04.825440 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.044954 (+219514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.051347 (+  6393us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.075434 (+ 24087us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.136604 (+ 61170us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:05.136650 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.136724 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.024019 (+887295us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:06.436201 (+412182us) raft_consensus.cc:1685] finished
0709 13:07:06.441405 (+  5204us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.494520 (+ 53115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":54860,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":115,"spinlock_wait_cycles":24074624,"thread_start_us":190321,"threads_started":2,"wal-append.queue_time_us":680441}
W20260709 13:07:06.657068 23994 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:07:06.657974 23997 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.217s	user 0.001s	sys 0.001s
I20260709 13:07:06.659157 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 750) took 1554 ms. Trace:
W20260709 13:07:06.661755 23756 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.714s	user 0.007s	sys 0.004s
I20260709 13:07:06.572621 21147 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:06.564430 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:06.573952 21142 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:06.593107 23995 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:07:06.669364 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 100974us
I20260709 13:07:06.675009 23976 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:06.675518 23670 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.000s
I20260709 13:07:06.758710 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:06.759016 23670 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.759411 21796 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:06.651089 23981 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:06.718629 23645 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:06.690550 21067 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:06.741012 23696 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:06.670928 23986 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.572s	user 0.004s	sys 0.016s
I20260709 13:07:06.693605 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:06.608181 23826 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.694511 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:06.696882 21218 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:06.779250 21799 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:06.779765 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:06.780814 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.781291 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:06.760989 23660 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:06.782043 23660 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.754283 23756 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:06.692013 23865 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.782581 23660 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:06.783581 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:06.602329 23975 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:07:06.739629 21143 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:06.738613 21792 rpcz_store.cc:276] 0709 13:07:04.870992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.153235 (+282243us) service_pool.cc:224] Handling call
0709 13:07:06.634643 (+1481408us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54005d5f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42522735 00007f7f4251a9fa 00007f7f474e24c9 00007f7f40a5ec54 00007f7f40a5ebe2 00007f7f40a5eb11 00007f7f40a5ea8c 00007f7f40a5dd42 00007f7f4085748c 00007f7f408568c6
0709 13:07:06.634931 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1641895680}
I20260709 13:07:06.747562 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.744629 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:06.821964 21147 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 3: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:06.716691 23998 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:06.742229 23887 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:06.752745 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.748358 21795 rpcz_store.cc:276] 0709 13:07:05.105254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.179129 (+ 73875us) service_pool.cc:224] Handling call
0709 13:07:05.179794 (+   665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.977909 (+798115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.977954 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.983218 (+  5264us) log.cc:844] Serialized 26 byte log entry
0709 13:07:06.409888 (+426670us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:06.409944 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.410040 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.658935 (+248895us) raft_consensus.cc:1685] finished
0709 13:07:06.658960 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.659096 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":560364,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"thread_start_us":1165479,"threads_started":2,"wal-append.queue_time_us":515733}
I20260709 13:07:06.755707 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:06.756536 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:06.856284 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:06.757944 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.667189 23983 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:06.761693 21801 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:06.741657 23957 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.690925 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 883) took 1925 ms. Trace:
I20260709 13:07:06.781531 23922 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:06.761150 23817 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:07:06.722970 21214 raft_consensus.cc:2368] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9.
W20260709 13:07:06.711463 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 157873us
I20260709 13:07:06.695243 24002 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.782089 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.784683 21146 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:06.691867 23873 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:06.782145 21218 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
W20260709 13:07:06.795346 21185 leader_election.cc:336] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.836s (SENT)
I20260709 13:07:06.822326 21797 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:06.832125 21067 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:07:06.752099 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 190952us
I20260709 13:07:06.765693 23995 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:06.855887 21063 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
W20260709 13:07:07.278436 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 483112us
W20260709 13:07:07.278848 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 836) took 2495 ms (client timeout 2370 ms). Trace:
I20260709 13:07:07.279510 21145 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
W20260709 13:07:07.279580 21063 rpcz_store.cc:269] 0709 13:07:04.783731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.743421 (+1959690us) service_pool.cc:224] Handling call
0709 13:07:07.278772 (+535351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.012738 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:07.280715 21216 rpcz_store.cc:276] 0709 13:07:04.765949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.218109 (+1452160us) service_pool.cc:224] Handling call
0709 13:07:06.278608 (+ 60499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.278893 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:07:06.409280 (+130387us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.687510 (+278230us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.687565 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:06.687620 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.687698 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.690864 (+  3166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":225748,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":612,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":407838,"threads_started":2,"wal-append.queue_time_us":283026}
I20260709 13:07:06.866627 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:07.283310 21212 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:07.267056 21794 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:06.866568 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:07.284869 21795 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:07.012970 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:07.292433 21147 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:06.857687 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:06.778434 23986 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:06.857869 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.320633 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.985951 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 290) took 1163 ms. Trace:
I20260709 13:07:06.912299 21064 raft_consensus.cc:2410] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:07:07.286072 21067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1794) took 2566 ms (client timeout 2309 ms). Trace:
W20260709 13:07:07.321532 21067 rpcz_store.cc:269] 0709 13:07:04.719315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.689699 (+1970384us) service_pool.cc:224] Handling call
0709 13:07:07.285996 (+596297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.289448 23957 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:07.307837 21798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 293) took 2743 ms (client timeout 3316 ms). Trace:
W20260709 13:07:07.322737 21798 rpcz_store.cc:269] 0709 13:07:04.564463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.564642 (+   179us) service_pool.cc:224] Handling call
0709 13:07:07.307633 (+2742991us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54003bbd54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:07.307772 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591690624}
W20260709 13:07:07.285229 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.313828 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:07.321254 21138 rpcz_store.cc:276] 0709 13:07:05.822253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.477032 (+654779us) service_pool.cc:224] Handling call
0709 13:07:06.482879 (+  5847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.483261 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:06.701329 (+218068us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.985491 (+284162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.985555 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:06.985605 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.985697 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.985872 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23407,"prepare.queue_time_us":72647,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":18225,"raft.queue_time_us":101077,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":92400,"thread_start_us":373916,"threads_started":2}
I20260709 13:07:07.324515 21212 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:07.331610 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:07.332316 21212 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:07.332906 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:06.912598 24017 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:07:07.340107 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:06.909706 24012 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:07:07.346511 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:07.335317 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:07.339917 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 843) took 2214 ms (client timeout 2134 ms). Trace:
W20260709 13:07:07.348060 21063 rpcz_store.cc:269] 0709 13:07:05.125649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.313805 (+2188156us) service_pool.cc:211] Skipping call since client already timed out
0709 13:07:07.339844 (+ 26039us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:07:07.015419 24015 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 13:07:07.340884 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:07.341645 21147 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:07.354938 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:07.355616 21147 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:07.356524 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:07.356853 21142 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:07:07.357846 21147 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:07:07.358482 21142 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:07:07.358928 21147 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:07.348369 21067 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:07:07.340778 21145 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:07.104316 23945 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:07.349985 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:07.353379 21221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 880) took 2620 ms (client timeout 1526 ms). Trace:
W20260709 13:07:07.368690 21221 rpcz_store.cc:269] 0709 13:07:04.733386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.768067 (+ 34681us) service_pool.cc:224] Handling call
0709 13:07:07.353122 (+2585055us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b540015c054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:07.353303 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":568641408}
I20260709 13:07:07.282716 23670 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:07.360378 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:07.348965 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:07.363693 21067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1795) took 2644 ms (client timeout 2897 ms). Trace:
W20260709 13:07:07.372270 21067 rpcz_store.cc:269] 0709 13:07:04.719584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.334988 (+2615404us) service_pool.cc:224] Handling call
0709 13:07:07.363616 (+ 28628us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.370939 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:07.373258 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:07.374145 21067 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:07.365286 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:07.303668 24002 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:07.315172 23873 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:07.383030 21067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 835) took 2599 ms (client timeout 2933 ms). Trace:
W20260709 13:07:07.383607 21067 rpcz_store.cc:269] 0709 13:07:04.783440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.373016 (+2589576us) service_pool.cc:224] Handling call
0709 13:07:07.382943 (+  9927us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.370662 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 280) took 2746 ms. Trace:
I20260709 13:07:07.384711 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:07.385593 21067 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
W20260709 13:07:07.353127 24022 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.443s	user 0.001s	sys 0.000s
I20260709 13:07:07.375150 21216 raft_consensus.cc:2410] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:07:07.381383 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 828) took 2639 ms (client timeout 2953 ms). Trace:
I20260709 13:07:07.384450 21144 rpcz_store.cc:276] 0709 13:07:04.624245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.624409 (+   164us) service_pool.cc:224] Handling call
0709 13:07:04.624679 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.138233 (+513554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.370368 (+2232135us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b540032c8d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:07.370588 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771299200}
W20260709 13:07:07.393707 21063 rpcz_store.cc:269] 0709 13:07:04.741660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.348779 (+2607119us) service_pool.cc:224] Handling call
0709 13:07:07.381300 (+ 32521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.394645 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2475) took 1071 ms. Trace:
I20260709 13:07:07.394882 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 886) took 2613 ms. Trace:
I20260709 13:07:07.395042 21216 rpcz_store.cc:276] 0709 13:07:06.323221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.346077 (+1022856us) service_pool.cc:224] Handling call
0709 13:07:07.394587 (+ 48510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.395246 21212 rpcz_store.cc:276] 0709 13:07:04.781363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.389378 (+2608015us) service_pool.cc:224] Handling call
0709 13:07:07.394375 (+  4997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.394568 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.394592 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.394632 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:07.394669 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.394739 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.394836 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:07.410696 21067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 354) took 2539 ms (client timeout 2876 ms). Trace:
W20260709 13:07:07.411299 21067 rpcz_store.cc:269] 0709 13:07:04.871041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.384403 (+2513362us) service_pool.cc:224] Handling call
0709 13:07:07.410615 (+ 26212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.412436 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:07.413503 21067 raft_consensus.cc:2410] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:07:07.387369 23865 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:07.422477 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:07.424603 21063 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
W20260709 13:07:06.902886 20954 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0d78b35ac92d402db8a5e3fe45be52b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b70fc04de37e4239a8336534f0c9c0fb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:07.398996 23952 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:07.395643 23873 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:07.405467 23995 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:07.433223 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1806) took 2705 ms (client timeout 3056 ms). Trace:
W20260709 13:07:07.433804 23933 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.407s	user 0.009s	sys 0.008s
W20260709 13:07:07.452446 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.466522 21214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 876) took 2784 ms (client timeout 2781 ms). Trace:
W20260709 13:07:07.466951 21214 rpcz_store.cc:269] 0709 13:07:04.682431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.682551 (+   120us) service_pool.cc:224] Handling call
0709 13:07:07.466310 (+2783759us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54002e5254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:07.466454 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4690409344}
W20260709 13:07:07.467849 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 341901us
W20260709 13:07:07.469223 21217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 881) took 2728 ms (client timeout 3366 ms). Trace:
W20260709 13:07:07.469614 21217 rpcz_store.cc:269] 0709 13:07:04.741250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.131449 (+390199us) service_pool.cc:224] Handling call
0709 13:07:07.469070 (+2337621us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540034c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:07.469169 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3225114880}
W20260709 13:07:07.475615 21067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1800) took 2754 ms (client timeout 3071 ms). Trace:
W20260709 13:07:07.476263 21067 rpcz_store.cc:269] 0709 13:07:04.720743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.412121 (+2691378us) service_pool.cc:224] Handling call
0709 13:07:07.475541 (+ 63420us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:07.462191 21063 rpcz_store.cc:269] 0709 13:07:04.722078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.402814 (+2680736us) service_pool.cc:224] Handling call
0709 13:07:07.426436 (+ 23622us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.480226 23887 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:07.481665 20954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 578929us
I20260709 13:07:07.463357 23871 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:07.482389 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:07.483618 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:07.456312 23951 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:07.589489 21063 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:07.492964 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 647) took 1132 ms. Trace:
I20260709 13:07:07.456830 23934 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:07.500414 21215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 885) took 2727 ms (client timeout 1805 ms). Trace:
W20260709 13:07:07.614665 21215 rpcz_store.cc:269] 0709 13:07:04.773395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.895661 (+122266us) service_pool.cc:224] Handling call
0709 13:07:07.500194 (+2604533us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54001caa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:07.500327 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1900063616}
I20260709 13:07:07.545754 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:07.616392 21067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 841) took 2501 ms (client timeout 3193 ms). Trace:
W20260709 13:07:07.616952 21067 rpcz_store.cc:269] 0709 13:07:05.115086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.477140 (+2362054us) service_pool.cc:224] Handling call
0709 13:07:07.616312 (+139172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.612115 21801 rpcz_store.cc:276] 0709 13:07:06.360404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.360856 (+   452us) service_pool.cc:224] Handling call
0709 13:07:07.492756 (+1131900us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b5400070154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:07.492901 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":858397952}
I20260709 13:07:07.549396 23813 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.431490 23826 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:07.573623 23660 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.791s	user 0.007s	sys 0.003s
I20260709 13:07:07.497431 21218 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:07.592715 21072 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
W20260709 13:07:07.631677 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 325262us
I20260709 13:07:07.619863 23951 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:07.547073 23978 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.187s	user 0.010s	sys 0.002s
I20260709 13:07:07.494331 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1792) took 2779 ms. Trace:
I20260709 13:07:07.620136 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1791) took 2905 ms. Trace:
I20260709 13:07:07.636358 21063 rpcz_store.cc:276] 0709 13:07:04.714902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.480929 (+2766027us) service_pool.cc:224] Handling call
0709 13:07:07.620076 (+139147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.636238 21070 rpcz_store.cc:276] 0709 13:07:04.715153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.493504 (+2778351us) service_pool.cc:224] Handling call
0709 13:07:07.493827 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.493993 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.494017 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.494057 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:07.494089 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.494140 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.494240 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.622123 21067 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:07.555076 23865 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:07.499125 23793 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:07.497045 23873 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:07.491847 23513 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:07.485169 23761 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:07.608469 23983 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:07.625308 23813 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:07.629873 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:07.663388 24023 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:07:07.628460 23660 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:07.631971 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 358) took 1072 ms. Trace:
I20260709 13:07:07.501631 21793 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:07.495848 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 293) took 1832 ms. Trace:
I20260709 13:07:07.500058 21797 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:07.640513 21070 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:07:07.635517 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:07.651377 21187 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.505s (SENT)
I20260709 13:07:07.641208 23978 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:07.684454 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1804) took 2962 ms. Trace:
I20260709 13:07:07.634202 23986 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.692139 21217 rpcz_store.cc:276] 0709 13:07:05.663904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.494974 (+1831070us) service_pool.cc:224] Handling call
0709 13:07:07.495357 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.495544 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.495570 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.495609 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:07.495648 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.495698 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.495795 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.693101 21070 pending_rounds.cc:85] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 4
I20260709 13:07:07.714265 21070 pending_rounds.cc:107] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:07:07.714577 21070 raft_consensus.cc:2889] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:07.715444 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1805) took 2993 ms. Trace:
I20260709 13:07:07.715893 21070 rpcz_store.cc:276] 0709 13:07:04.721858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.637703 (+2915845us) service_pool.cc:224] Handling call
0709 13:07:07.638000 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.715158 (+ 77158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.715378 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.717142 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:07.718716 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:07.693499 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58076.3us
I20260709 13:07:07.693722 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:07.705462 21067 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:07:07.794659 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 143362us
W20260709 13:07:07.743192 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.687983 24020 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:07:07.686575 24021 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:07:07.686986 23961 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.944s	user 0.005s	sys 0.007s
I20260709 13:07:07.787721 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:07.800700 21217 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:07.691850 21072 rpcz_store.cc:276] 0709 13:07:06.560329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.483341 (+923012us) service_pool.cc:224] Handling call
0709 13:07:07.631912 (+148571us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.703531 21063 rpcz_store.cc:276] 0709 13:07:04.721642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.637297 (+2915655us) service_pool.cc:224] Handling call
0709 13:07:07.683810 (+ 46513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.684038 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.684076 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.684131 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:07.684176 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.684246 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.684382 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:07.792014 21220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 868) took 3147 ms (client timeout 2348 ms). Trace:
W20260709 13:07:07.802000 21220 rpcz_store.cc:269] 0709 13:07:04.644894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.645696 (+   802us) service_pool.cc:224] Handling call
0709 13:07:07.777100 (+3131404us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54004439d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:07.791919 (+ 14819us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3022787328}
I20260709 13:07:07.694563 23980 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:07.692780 23660 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:07.689661 21215 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:07.794595 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1799) took 3074 ms. Trace:
I20260709 13:07:07.738708 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:07.701002 23978 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:07.662159 23865 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:07.810576 23871 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.815534 23957 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:07.816663 21066 raft_consensus.cc:2410] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:07:07.795892 23895 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:07.796593 23761 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.818825 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:07.826215 21063 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:07.803998 23922 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.744272 23986 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:07.805683 23980 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.823844 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:07.827316 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.831787 21217 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:07:07.826977 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 821) took 3104 ms. Trace:
I20260709 13:07:07.829802 21212 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:07.822359 23868 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.833843 21072 rpcz_store.cc:276] 0709 13:07:04.722547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.819903 (+3097356us) service_pool.cc:224] Handling call
0709 13:07:07.826414 (+  6511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.826636 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.826660 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.826696 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:07.826726 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.826800 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.826915 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.807749 23961 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:07.808727 21067 rpcz_store.cc:276] 0709 13:07:04.720527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.621694 (+2901167us) service_pool.cc:224] Handling call
0709 13:07:07.622012 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.794207 (+172195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.794518 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.843950 21067 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:07.844655 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 823) took 3121 ms. Trace:
I20260709 13:07:07.845034 21067 rpcz_store.cc:276] 0709 13:07:04.722992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.843366 (+3120374us) service_pool.cc:224] Handling call
0709 13:07:07.843722 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.844390 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.844592 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.845762 21063 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:07.846562 21067 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:07.847110 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 825) took 3120 ms. Trace:
I20260709 13:07:07.847811 21067 rpcz_store.cc:276] 0709 13:07:04.726478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.845893 (+3119415us) service_pool.cc:224] Handling call
0709 13:07:07.846227 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.846900 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.847063 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.848341 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:07.849273 21217 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:07.849514 21067 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:07.808259 24002 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:07.841615 23868 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:07.828871 23660 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.842164 23959 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:07.847005 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:07.860215 21765 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.020s	sys 0.071s
I20260709 13:07:07.855923 23969 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:07.856302 24002 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:07.856912 23695 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:07.857607 21138 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:07.832965 23980 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:07.850382 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 831) took 3076 ms. Trace:
I20260709 13:07:07.865907 21067 rpcz_store.cc:276] 0709 13:07:04.773750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.848931 (+3075181us) service_pool.cc:224] Handling call
0709 13:07:07.849276 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.850095 (+   819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.850321 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.867493 21067 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:07:07.821913 23871 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:07.829996 23813 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:07.860808 21765 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.020s	sys 0.072s
I20260709 13:07:07.858964 23513 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.831612 23922 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:07.808346 21187 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.795s (SENT)
I20260709 13:07:07.847088 21063 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:07.880934 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 826) took 3154 ms. Trace:
I20260709 13:07:07.881373 21063 rpcz_store.cc:276] 0709 13:07:04.726707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.846568 (+3119861us) service_pool.cc:224] Handling call
0709 13:07:07.846858 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.880604 (+ 33746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.880863 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.882516 21067 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:07:07.878134 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 806159us
I20260709 13:07:07.883347 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 834) took 3100 ms. Trace:
I20260709 13:07:07.883666 21067 rpcz_store.cc:276] 0709 13:07:04.783103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.866951 (+3083848us) service_pool.cc:224] Handling call
0709 13:07:07.867353 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.883090 (+ 15737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.883268 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.884970 21067 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:07:07.877135 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 286993us
I20260709 13:07:07.886195 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:07.860968 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:07.886945 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 78632.9us
I20260709 13:07:07.887167 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:07.886181 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:07.890825 21187 leader_election.cc:336] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.836s (SENT)
I20260709 13:07:07.903721 21217 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:07.833423 23761 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:07.906263 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:07.943300 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:07.890904 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1989) took 3220 ms (client timeout 1836 ms). Trace:
I20260709 13:07:07.918540 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1811) took 2808 ms. Trace:
W20260709 13:07:07.945521 21140 rpcz_store.cc:269] 0709 13:07:04.670060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.712163 (+ 42103us) service_pool.cc:224] Handling call
0709 13:07:07.881271 (+3169108us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54002b4dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:07.890801 (+  9530us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":944381696}
I20260709 13:07:07.890625 21071 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:07.920668 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:07.945683 21067 rpcz_store.cc:276] 0709 13:07:05.110391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.884344 (+2773953us) service_pool.cc:224] Handling call
0709 13:07:07.884605 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.885281 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.918433 (+ 33152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.865097 23987 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:07:07.858822 23936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.013s	sys 0.043s
I20260709 13:07:07.862191 23969 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:07.893985 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.949805 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59001.6us
W20260709 13:07:07.953640 21187 leader_election.cc:336] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.882s (SENT)
I20260709 13:07:07.948539 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1808) took 3097 ms. Trace:
W20260709 13:07:07.955273 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.940958 23985 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:07.940058 23951 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:07.942366 23955 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:07.955350 21071 rpcz_store.cc:276] 0709 13:07:04.850996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.880305 (+3029309us) service_pool.cc:224] Handling call
0709 13:07:07.880717 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.947355 (+ 66638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.948485 (+  1130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:07.940321 24028 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:07.954584 23936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.014s	sys 0.044s
I20260709 13:07:07.965588 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:07.968581 21217 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:07.969655 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:07.920506 23513 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:07.951562 23695 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:07.951684 23895 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:07.863073 23660 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:07.941303 23959 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:07:07.966483 23955 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.971190 21217 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:07.957435 23987 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:07.973830 21140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:07.961541 23951 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.970095 23858 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:07.985047 23871 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:07.970527 23985 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:07.973332 23695 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.996950 23695 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:07.979749 21140 raft_consensus.cc:606] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:08.000315 21793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 648) took 1639 ms (client timeout 2138 ms). Trace:
I20260709 13:07:07.954612 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:07.991164 23871 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:07:07.993074 21066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1786) took 3279 ms (client timeout 2027 ms). Trace:
W20260709 13:07:08.009630 21066 rpcz_store.cc:269] 0709 13:07:04.713509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.730737 (+ 17228us) service_pool.cc:224] Handling call
0709 13:07:07.992808 (+3262071us) spinlock_profiling.cc:243] Waited 3.04 s on lock 0x7b5400214b54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:07.992984 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6994031872}
W20260709 13:07:08.010161 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.000881 21793 rpcz_store.cc:269] 0709 13:07:06.360680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.360934 (+   254us) service_pool.cc:224] Handling call
0709 13:07:08.000084 (+1639150us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54006612d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:08.000240 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2564161152}
W20260709 13:07:07.993542 23998 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.088s	user 0.009s	sys 0.001s
I20260709 13:07:07.993713 23616 raft_consensus.cc:993] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:07.980218 23951 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:07.994762 23961 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:07.996013 23922 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:07.983754 23959 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:07.977662 23813 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:07:08.008448 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 54845.5us
I20260709 13:07:08.011003 21066 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
W20260709 13:07:08.054679 21187 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.936s (SENT) [suppressed 39 similar messages]
I20260709 13:07:08.012230 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:08.015692 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:08.055572 21219 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:07.982837 23955 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:08.035532 21113 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.805s (SENT)
I20260709 13:07:07.989494 23865 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:08.081475 23761 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:07.970257 24028 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.047632 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:08.042641 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.011914 23895 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:08.075516 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:08.077431 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:08.080333 23660 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:07.989087 23957 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:08.083499 23969 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.086544 24028 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:08.088943 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:08.121779 21220 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:08.089877 21146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 1999) took 3356 ms (client timeout 2200 ms). Trace:
I20260709 13:07:08.083477 24027 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:08.122258 21146 rpcz_store.cc:269] 0709 13:07:04.734539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.108536 (+373997us) service_pool.cc:224] Handling call
0709 13:07:08.089658 (+2981122us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54002959d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:08.089803 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3729432448}
I20260709 13:07:08.012089 23986 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:08.091954 23987 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.147445 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 888) took 3033 ms. Trace:
I20260709 13:07:08.147902 21216 rpcz_store.cc:276] 0709 13:07:05.114501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.399840 (+2285339us) service_pool.cc:224] Handling call
0709 13:07:07.407372 (+  7532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.407875 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:07:07.407917 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:07.414363 (+  6446us) log.cc:844] Serialized 26 byte log entry
0709 13:07:07.862084 (+447721us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:07.862127 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.862197 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.147193 (+284996us) raft_consensus.cc:1685] finished
0709 13:07:08.147216 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.147379 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":190060,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":4435,"thread_start_us":390939,"threads_started":1,"wal-append.queue_time_us":690843}
I20260709 13:07:08.093516 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1789) took 3379 ms. Trace:
I20260709 13:07:08.093258 23985 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.096781 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2651) took 1351 ms. Trace:
I20260709 13:07:08.009442 23871 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:08.097328 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:08.156087 21217 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:07:08.097641 21064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 842) took 2977 ms (client timeout 1805 ms). Trace:
W20260709 13:07:08.097905 21187 leader_election.cc:336] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 2.027s (SENT)
I20260709 13:07:08.099670 21069 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:08.095019 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:08.157099 21064 rpcz_store.cc:269] 0709 13:07:05.120554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.503912 (+1383358us) service_pool.cc:224] Handling call
0709 13:07:08.097430 (+1593518us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54000dc354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:08.097556 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832417536}
I20260709 13:07:08.091981 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 889) took 2977 ms. Trace:
I20260709 13:07:08.158005 21212 rpcz_store.cc:276] 0709 13:07:05.114804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.405941 (+2291137us) service_pool.cc:224] Handling call
0709 13:07:07.418433 (+ 12492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.834469 (+416036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.091693 (+257224us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54002ee0d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:08.091909 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945619328}
I20260709 13:07:08.053944 24040 raft_consensus.cc:1081] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:08.054299 23813 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:08.121042 21219 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:08.011415 23936 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:07:08.124213 24027 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:08.153844 21138 rpcz_store.cc:276] 0709 13:07:06.745503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.354677 (+609174us) service_pool.cc:224] Handling call
0709 13:07:08.096560 (+741883us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b5400258ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:08.096717 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1107873536}
I20260709 13:07:08.181042 24039 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.185699 24039 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:08.120957 23865 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:08.168226 21212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:07:08.192798 21212 raft_consensus.cc:606] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:08.193351 21212 raft_consensus.cc:614] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:08.172108 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 77200.4us
I20260709 13:07:08.150113 21065 rpcz_store.cc:276] 0709 13:07:04.714347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.715525 (+  1178us) service_pool.cc:224] Handling call
0709 13:07:04.716601 (+  1076us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:05.669882 (+953281us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:05.788399 (+118517us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.045318 (+256919us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:06.045950 (+   632us) log.cc:844] Serialized 52 byte log entry
0709 13:07:06.046220 (+   270us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:06.046262 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.093101 (+2046839us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b540001aed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:08.093152 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.093201 (+    49us) raft_consensus.cc:1685] finished
0709 13:07:08.093291 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.093460 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":192122,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":204,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":194,"spinlock_wait_cycles":2191210880,"thread_start_us":310692,"threads_started":2,"wal-append.queue_time_us":598811,"wal-append.run_cpu_time_us":3598,"wal-append.run_wall_time_us":1411621}
I20260709 13:07:08.194340 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:08.194777 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.195607 21111 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:08.181310 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.034148 23998 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.069562 23959 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:08.189011 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 91129.8us
I20260709 13:07:08.196729 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:08.214581 21187 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 2.076s (SENT)
I20260709 13:07:08.171406 23451 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:08.191038 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:08.216989 21220 raft_consensus.cc:2368] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9.
I20260709 13:07:08.217092 21212 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:07:08.218107 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2468) took 2118 ms. Trace:
I20260709 13:07:08.191666 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:08.188252 23986 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:08.198450 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:08.222468 21216 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:07:08.223304 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:08.199855 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.200062 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 884) took 3431 ms. Trace:
W20260709 13:07:08.202875 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 167371us
I20260709 13:07:08.182798 23813 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:08.225792 21796 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:08.197577 21068 raft_consensus.cc:2410] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:08.218080 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:08.218643 21212 rpcz_store.cc:276] 0709 13:07:06.099410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.213624 (+2114214us) service_pool.cc:224] Handling call
0709 13:07:08.216867 (+  3243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.217861 (+   994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.218051 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:08.221120 21213 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 3: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:08.227397 21065 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:08.220973 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.196019 23969 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:08.226019 21216 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:08.228859 21069 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:08.222731 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:08.241825 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:08.229478 24038 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:07:08.221741 23986 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:08.215358 23865 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:07:08.233590 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 1.71716e+06us
I20260709 13:07:08.224126 21221 rpcz_store.cc:276] 0709 13:07:04.768927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.375093 (+2606166us) service_pool.cc:224] Handling call
0709 13:07:07.375408 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.375670 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:07:07.834997 (+459327us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.199726 (+364729us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:08.199775 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:08.199825 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.199895 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.200010 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":633262,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":11705,"raft.queue_time_us":14091,"raft.run_wall_time_us":4091,"thread_start_us":823341,"threads_started":2}
I20260709 13:07:08.213128 23616 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:08.238435 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.265002 24042 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:08.272572 24042 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:08.273811 24042 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:08.278293 23695 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:08.279816 24027 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:08.280792 23695 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:08.281035 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 755) took 1713 ms. Trace:
I20260709 13:07:08.239105 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.236912 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:08.247107 21111 leader_election.cc:400] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9.
I20260709 13:07:08.282584 21111 leader_election.cc:403] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:07:08.265425 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:08.283072 21068 raft_consensus.cc:2410] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:08.283355 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:08.266355 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:08.220744 23978 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:08.267519 21065 raft_consensus.cc:2376] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6. Re-sending same reply.
I20260709 13:07:08.267791 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.281587 21796 rpcz_store.cc:276] 0709 13:07:06.567133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.760249 (+193116us) service_pool.cc:224] Handling call
0709 13:07:08.280800 (+1520551us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400633dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:08.280955 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3315335424}
W20260709 13:07:08.290611 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 326636us
I20260709 13:07:08.291007 21069 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:07:08.291435 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:08.290383 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.292335 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:08.293100 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:08.294088 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.292495 21065 raft_consensus.cc:2376] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8. Re-sending same reply.
I20260709 13:07:08.294996 21068 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:08.295583 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.293449 21212 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:08.306066 21138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:08.308279 21212 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:07:08.296543 21065 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:08.308619 21138 raft_consensus.cc:606] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:08.281944 23865 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.271673 24038 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:08.309617 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.310431 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.267071 23616 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.322474 21068 raft_consensus.cc:2410] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:07:08.289598 23952 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:08.266808 23986 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:08.295979 23858 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:08.329872 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:08.337503 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 112987us
I20260709 13:07:08.337817 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.289552 23978 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.298250 23983 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:08.307142 23695 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:08.321132 23451 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:07:08.349952 21113 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.716s (SENT)
I20260709 13:07:08.350615 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:08.313153 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.311228 24048 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:08.351398 23939 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:08.351398 24048 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:08.351981 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.352291 24048 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:08.353425 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.354333 21069 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:08.355502 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.356870 21069 raft_consensus.cc:2410] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:07:08.357841 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.331216 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:08.265133 23980 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.362902 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:07:08.321316 23945 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.000s	sys 0.019s
I20260709 13:07:08.321010 23865 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:08.331449 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.294117 23960 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:08.313890 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:08.363900 24048 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:08.365319 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51534.9us
I20260709 13:07:08.367376 24048 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:08.372352 23865 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:08.367733 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:08.422662 21143 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:07:08.367388 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.368534 23939 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:08.369993 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:08.364037 23945 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.000s	sys 0.020s
I20260709 13:07:08.371013 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.372100 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.372396 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.373509 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.492697 21144 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:08.493772 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:08.374539 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.374864 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:08.294529 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:08.495113 21145 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:08.495530 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 201065us
I20260709 13:07:08.496537 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:08.496790 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:08.321708 24038 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:08.492411 21141 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:08.497892 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:08.499599 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 82505.6us
I20260709 13:07:08.494800 21142 raft_consensus.cc:2410] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:07:08.376386 21065 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:08.270479 23955 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:08.376866 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.500259 21187 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 2.177s (SENT)
I20260709 13:07:08.330289 23616 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:08.339232 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.537089 21068 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 3.
I20260709 13:07:08.537626 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.537915 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.384555 21147 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:07:08.347657 23978 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:08.348546 23695 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.543685 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.399597 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:08.560325 21138 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:08.376340 23451 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.403831 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:08.406047 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:08.409091 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.564800 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.565531 21144 raft_consensus.cc:2368] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:07:08.566390 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.410203 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 297) took 1922 ms. Trace:
I20260709 13:07:08.375583 24039 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:08.419152 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 300) took 1871 ms. Trace:
W20260709 13:07:08.489609 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.491639 21139 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:08.570834 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:08.569728 21215 rpcz_store.cc:276] 0709 13:07:06.547558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.276051 (+1728493us) service_pool.cc:224] Handling call
0709 13:07:08.276376 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.276577 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:08.362408 (+ 85831us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.418624 (+ 56216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:08.418741 (+   117us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:08.418802 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.418898 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.419053 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":50074,"thread_start_us":141222,"threads_started":2}
I20260709 13:07:08.571642 21795 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:08.571837 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:08.532346 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:08.572661 21799 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:08.572921 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.567812 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:08.538405 21065 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:08.542201 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44393.6us
I20260709 13:07:08.546519 21069 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:08.574229 21141 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:08.539394 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.574738 21147 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:08.574900 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.560643 21793 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:08.575592 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.567724 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.568387 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.567770 21216 rpcz_store.cc:276] 0709 13:07:06.487931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.242769 (+1754838us) service_pool.cc:224] Handling call
0709 13:07:08.243129 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.243404 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:08.330999 (+ 87595us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.407305 (+ 76306us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:08.407362 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:08.407407 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.407465 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.410053 (+  2588us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":661,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":4429,"thread_start_us":162755,"threads_started":2}
I20260709 13:07:08.572715 21215 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:08.576591 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:08.401497 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:08.576932 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.576980 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.492396 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:08.577782 21065 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:08.578608 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.538384 21068 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:08.569744 23451 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:08.580026 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.580608 21068 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:07:08.581578 21068 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:08.582613 21068 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
W20260709 13:07:08.584502 24050 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:07:08.587550 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 357) took 2028 ms. Trace:
I20260709 13:07:08.587762 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:08.588855 24045 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.245s	user 0.000s	sys 0.000s
I20260709 13:07:08.587952 21071 rpcz_store.cc:276] 0709 13:07:06.559530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.963389 (+1403859us) service_pool.cc:224] Handling call
0709 13:07:07.963748 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.147783 (+184035us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.147842 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.159893 (+ 12051us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.319532 (+159639us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:08.319585 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.319677 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.587351 (+267674us) raft_consensus.cc:1685] finished
0709 13:07:08.587371 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.587493 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21629,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5308,"thread_start_us":297201,"threads_started":2,"wal-append.queue_time_us":207271}
I20260709 13:07:08.593075 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.593999 21069 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:07:08.594699 24038 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:08.569839 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.575044 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:08.595948 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 194524us
W20260709 13:07:08.599226 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.605203 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:08.414016 23959 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:08.603457 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:08.609905 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 355) took 3738 ms. Trace:
I20260709 13:07:08.604111 21068 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:07:08.610299 21063 rpcz_store.cc:276] 0709 13:07:04.871454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.882177 (+3010723us) service_pool.cc:224] Handling call
0709 13:07:07.882507 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.882846 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.882873 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:07.886565 (+  3692us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.163339 (+276774us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:08.163418 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.163518 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.609701 (+446183us) raft_consensus.cc:1685] finished
0709 13:07:08.609720 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.609847 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":47748,"thread_start_us":209474,"threads_started":1,"wal-append.queue_time_us":348016}
I20260709 13:07:08.494740 23939 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:08.620574 21145 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:08.598170 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.577178 21147 raft_consensus.cc:2376] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10. Re-sending same reply.
I20260709 13:07:08.627585 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.628520 21063 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:07:08.623101 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:08.558622 23960 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:08.562564 23660 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:08.558322 23761 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:08.562961 23695 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:08.625109 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.638443 21035 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:08.542660 24049 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:08.596107 24048 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:08.570323 23998 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:08.660085 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:08.622799 24041 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.396s	user 0.001s	sys 0.000s
I20260709 13:07:08.668036 21793 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:08.595754 24038 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:08.595117 23951 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.526s	user 0.004s	sys 0.004s
I20260709 13:07:08.667987 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:08.661031 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:08.670037 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:08.670773 21216 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:08.669055 21215 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:07:08.613384 24039 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:08.623803 23939 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:08.632265 23922 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:08.673260 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:08.679065 21141 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:08.630770 24040 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.635691 24028 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:08.677733 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.662585 23959 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:08.639410 23960 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.689544 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.665853 24048 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.695600 21138 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:08.664232 24049 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:08.675805 24043 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.381s	user 0.000s	sys 0.001s
W20260709 13:07:08.736056 21113 leader_election.cc:336] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.731s (SENT)
I20260709 13:07:08.656031 23451 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.680804 23922 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.742121 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.745296 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:08.745594 21068 raft_consensus.cc:2410] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:08.695420 23959 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.677243 24039 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.725513 24028 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.703428 23960 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:08.723747 24048 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:08.724413 24049 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:08.757761 23451 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:08.666648 23761 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.765769 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.766425 21068 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:08.697193 24040 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.676108 23951 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.788116 23951 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:08.792520 24072 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:08.793108 24072 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.793501 24072 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:08.794462 21187 leader_election.cc:336] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 2.309s (SENT)
I20260709 13:07:08.794932 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:08.795362 23616 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:08.752859 23959 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:08.798072 21142 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:07:08.759742 24039 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:08.783967 24052 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
I20260709 13:07:08.784746 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 298) took 2296 ms. Trace:
W20260709 13:07:08.765882 24044 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.422s	user 0.000s	sys 0.001s
I20260709 13:07:08.785248 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 296) took 2896 ms. Trace:
I20260709 13:07:08.749461 23922 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:08.796030 24026 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:07:08.770635 24019 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.151s	user 0.006s	sys 0.006s
I20260709 13:07:08.803681 23660 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:08.774269 23858 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9.
I20260709 13:07:08.816090 23939 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.762562 24028 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:08.763108 23756 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.783872 23865 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:08.821553 23813 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.583s	user 0.005s	sys 0.005s
I20260709 13:07:08.775182 23978 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.888139 23939 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:08.811235 24072 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.911572 24028 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.918563 24072 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:08.920958 24048 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:08.922616 24048 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:08.748513 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 110132us
I20260709 13:07:08.954294 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:08.977869 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.980052 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.979945 21066 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:08.981324 21063 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:08.982376 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.982227 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.984488 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.986322 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:08.799031 21141 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:09.018251 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:09.019605 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 299) took 2379 ms. Trace:
I20260709 13:07:09.020116 21221 rpcz_store.cc:276] 0709 13:07:06.503011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.266278 (+1763267us) service_pool.cc:224] Handling call
0709 13:07:08.266613 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.284383 (+ 17770us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.284421 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.284962 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.412689 (+127727us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:08.412732 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.412802 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.881245 (+468443us) raft_consensus.cc:1685] finished
0709 13:07:08.881266 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.881399 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17336,"raft.queue_time_us":22562,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":11898,"thread_start_us":99736,"threads_started":1,"wal-append.queue_time_us":445506}
I20260709 13:07:09.024791 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 904) took 1547 ms. Trace:
I20260709 13:07:09.028566 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 365) took 1232 ms. Trace:
I20260709 13:07:09.031728 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.033524 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.034960 21069 raft_consensus.cc:2410] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:09.002908 23660 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.029179 21070 rpcz_store.cc:276] 0709 13:07:07.796358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.085399 (+289041us) service_pool.cc:224] Handling call
0709 13:07:08.085712 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.086148 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.086187 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.086733 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.296735 (+210002us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:08.408258 (+111523us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.408446 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.409475 (+  1029us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.987149 (+577674us) raft_consensus.cc:1685] finished
0709 13:07:08.987182 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.028360 (+ 41178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56958,"prepare.queue_time_us":271210,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":13090,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":129898,"thread_start_us":304460,"threads_started":2,"wal-append.queue_time_us":289127}
I20260709 13:07:08.996122 21146 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:07:08.996241 21140 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:08.997987 21139 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:08.999364 21143 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:08.883761 23756 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.043967 23955 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.045504 23756 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.001760 23761 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:09.003618 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.048182 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.049739 21070 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:07:08.873147 21219 rpcz_store.cc:276] 0709 13:07:05.889029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.171809 (+2282780us) service_pool.cc:224] Handling call
0709 13:07:08.172127 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.220239 (+ 48112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.220285 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.235962 (+ 15677us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.399215 (+163253us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:08.695582 (+296367us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.695863 (+   281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.695937 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.785074 (+ 89137us) raft_consensus.cc:1685] finished
0709 13:07:08.785091 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.785201 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49328,"prepare.queue_time_us":328338,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":4203,"raft.queue_time_us":42921,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":37004,"spinlock_wait_cycles":24724736,"thread_start_us":383987,"threads_started":2,"wal-append.queue_time_us":398167}
I20260709 13:07:09.004164 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:09.051085 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.051472 21799 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:09.051721 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:09.005823 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:09.052248 21070 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:09.052708 21795 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:09.053700 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:09.057588 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.058488 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:09.058693 21141 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:08.987586 21063 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:08.885023 23813 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.059698 23978 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:09.060756 21064 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:09.060966 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.062096 23813 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.062526 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:09.040058 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:09.063313 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.064280 21064 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
W20260709 13:07:09.064316 21218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2470) took 2872 ms (client timeout 2688 ms). Trace:
I20260709 13:07:09.064543 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:09.064913 21218 rpcz_store.cc:269] 0709 13:07:06.192142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.315367 (+1123225us) service_pool.cc:224] Handling call
0709 13:07:09.063794 (+1748427us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54002eefd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:09.064068 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287687040}
I20260709 13:07:09.065495 21221 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:09.065407 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.066579 21064 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.055111 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:09.008591 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 289) took 3801 ms. Trace:
I20260709 13:07:09.009495 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:09.011808 23945 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:09.014277 23616 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.075762 24048 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.077106 24048 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:08.824352 21213 rpcz_store.cc:276] 0709 13:07:06.488299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.245824 (+1757525us) service_pool.cc:224] Handling call
0709 13:07:08.270876 (+ 25052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.289816 (+ 18940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.289876 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:08.290371 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.406857 (+116486us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:08.406901 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.406970 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.784499 (+377529us) raft_consensus.cc:1685] finished
0709 13:07:08.784523 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.784682 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18489,"raft.queue_time_us":23560,"spinlock_wait_cycles":20582144,"thread_start_us":87960,"threads_started":1,"wal-append.queue_time_us":381776}
I20260709 13:07:09.025380 21212 rpcz_store.cc:276] 0709 13:07:07.477019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.321249 (+844230us) service_pool.cc:224] Handling call
0709 13:07:08.330365 (+  9116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.330579 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:07:08.553037 (+222458us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.023836 (+470799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:09.023999 (+   163us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:09.024208 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.024354 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.024599 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6685,"prepare.queue_time_us":368456,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":74808,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":3054,"thread_start_us":679571,"threads_started":3,"wal-append.queue_time_us":364111}
W20260709 13:07:09.007968 23969 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.713s	user 0.010s	sys 0.002s
I20260709 13:07:09.074429 24049 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.081161 23945 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:09.066838 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:09.075883 21794 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:09.066896 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:09.068974 21219 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:09.083015 23945 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.086270 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:08.915470 24028 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:09.089772 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.091064 21064 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.067909 21214 rpcz_store.cc:276] 0709 13:07:05.207316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.467799 (+2260483us) service_pool.cc:224] Handling call
0709 13:07:07.468137 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:07.947944 (+479807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.947978 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:07.951698 (+  3720us) log.cc:844] Serialized 52 byte log entry
0709 13:07:08.126266 (+174568us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:08.171026 (+ 44760us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.191104 (+ 20078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.191747 (+   643us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.797433 (+605686us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b180034c540. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:09.007708 (+210275us) raft_consensus.cc:1685] finished
0709 13:07:09.007857 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.008275 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":54796,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":2757,"raft.queue_time_us":176617,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":26506,"spinlock_wait_cycles":5231360,"thread_start_us":636528,"threads_started":2,"wal-append.queue_time_us":337310}
I20260709 13:07:09.092923 21221 raft_consensus.cc:2410] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:07:09.116770 21066 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:09.148010 21795 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
W20260709 13:07:09.153930 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.156237 24049 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:09.159974 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1450813800000788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
I20260709 13:07:09.116767 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:09.172344 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0820813800000906s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 13:07:09.175022 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0712813800000731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
I20260709 13:07:09.174464 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.177505 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.177925 21064 raft_consensus.cc:2368] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8.
I20260709 13:07:09.179032 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:09.180802 23695 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.003s
I20260709 13:07:09.181270 23695 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:09.180819 21064 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:09.183807 21138 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:09.177554 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:09.184924 23695 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:09.187862 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.190505 24048 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.191228 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:09.193513 23922 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.192147 24026 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:07:09.194902 24048 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:09.204375 24048 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.203982 23960 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.204819 24048 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:09.206506 23960 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.193967 23887 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.207789 23887 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.208843 21035 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:09.210369 23887 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:09.192190 23871 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.202081 23959 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.211716 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.215477 21141 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:09.216153 23871 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:09.216830 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.217593 21141 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:09.210889 23936 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:09.219743 23936 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:09.221590 23936 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.227918 21069 raft_consensus.cc:2368] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:07:09.228487 24089 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:09.230051 23959 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.227950 21796 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:07:09.231406 21796 pending_rounds.cc:85] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 6
I20260709 13:07:09.231768 21796 pending_rounds.cc:107] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:07:09.232172 21796 raft_consensus.cc:2889] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:09.213070 21063 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:09.236166 23922 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:09.236990 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.218796 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:09.238436 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:09.243695 24039 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.253870 23513 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.254654 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:09.255481 21795 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:09.255570 23995 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:09.258108 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.259621 21070 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.260782 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.261459 23922 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:09.262091 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.244279 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.263015 24039 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.264485 24048 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.266758 24048 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.267501 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:09.268242 21143 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:09.269114 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 79592.3us
I20260709 13:07:09.237053 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.272696 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.273484 21147 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:09.274433 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.276033 21066 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:09.276799 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.277616 21139 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 3.
I20260709 13:07:09.278687 24090 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:09.280964 24090 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:07:09.283942 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56273.7us
I20260709 13:07:09.285174 24027 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:09.253284 21141 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:09.278753 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.263018 21140 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 3: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:07:09.266697 23986 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.919s	user 0.019s	sys 0.001s
I20260709 13:07:09.294881 21795 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:09.295781 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:09.305585 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:09.315085 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 890) took 3565 ms. Trace:
I20260709 13:07:09.315651 21217 rpcz_store.cc:276] 0709 13:07:05.718088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.168877 (+2450789us) service_pool.cc:224] Handling call
0709 13:07:08.169241 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.187736 (+ 18495us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:08.329796 (+142060us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.376874 (+ 47078us) log.cc:844] Serialized 14 byte log entry
0709 13:07:08.377045 (+   171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:08.377093 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:08.377144 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.798935 (+421791us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b1800262a80. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:09.202539 (+403604us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b4800281958. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:07:09.218541 (+ 16002us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b1800262a80. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:09.281664 (+ 63123us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b1800262a80. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3fd2a6 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419
0709 13:07:09.282919 (+  1255us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b54004434d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:09.282962 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.283188 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":847588,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":71,"raft.queue_time_us":163,"raft.run_cpu_time_us":9641,"raft.run_wall_time_us":831496,"spinlock_wait_cycles":120366464,"thread_start_us":176095,"threads_started":2,"wal-append.queue_time_us":270394}
I20260709 13:07:09.318764 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.319588 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.320355 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.320369 21140 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:09.321713 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.321884 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.323585 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.324347 21140 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:09.325946 21140 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:09.324427 21147 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:09.327545 21147 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:07:09.328845 24075 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.417s	user 0.000s	sys 0.001s
W20260709 13:07:09.330435 24063 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.654s	user 0.003s	sys 0.000s
I20260709 13:07:09.335647 23868 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.332310 23695 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:07:09.336596 23695 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:09.336539 23978 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:09.337144 23695 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:09.307134 23986 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.339335 21144 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:09.340405 23986 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:09.341578 21144 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:09.347479 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 48854.3us
I20260709 13:07:09.347936 23871 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:09.348807 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:09.349246 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78073.5us
I20260709 13:07:09.350705 21139 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:09.338348 23695 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.352929 23695 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:09.333556 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.312539 23995 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:07:09.356165 21065 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.347535 23980 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:09.343080 21144 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:07:09.331581 21140 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:07:09.358132 24040 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:09.359932 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:09.346942 21143 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:09.359678 23945 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:09.369737 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:09.376101 21141 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:09.391677 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:09.383714 23945 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:09.410816 21184 connection.cc:256] RPC call timeout handler was delayed by -1.0995673759999818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
W20260709 13:07:09.412221 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 77134.6us
I20260709 13:07:09.415114 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:09.402961 23936 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:09.420954 21185 leader_election.cc:400] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:07:09.424588 21185 leader_election.cc:403] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:09.439774 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51685.8us
W20260709 13:07:09.441586 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 90383.2us
I20260709 13:07:09.439188 23980 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.443490 24027 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:09.443779 23980 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.445564 23995 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:07:09.447081 23980 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:09.448870 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.449894 21066 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.449903 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.450419 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:09.451457 24046 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.222s	user 0.009s	sys 0.000s
I20260709 13:07:09.451712 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.453109 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.453279 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:09.453904 21070 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.455193 24027 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:09.455240 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 822) took 4732 ms. Trace:
I20260709 13:07:09.455701 24027 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.456200 24027 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:09.455829 21072 rpcz_store.cc:276] 0709 13:07:04.722762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.836932 (+3114170us) service_pool.cc:224] Handling call
0709 13:07:07.912914 (+ 75982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.931611 (+ 18697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.931654 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:07.932274 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.097106 (+164832us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:08.097144 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.097223 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:08.146888 (+ 49665us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b4800080358. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:07:09.426583 (+1279695us) spinlock_profiling.cc:243] Waited 6.2 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:09.454740 (+ 28157us) raft_consensus.cc:1685] finished
0709 13:07:09.454768 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.454994 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29777,"raft.queue_time_us":60141,"raft.run_cpu_time_us":2623,"raft.run_wall_time_us":171560,"spinlock_wait_cycles":19483264,"thread_start_us":139748,"threads_started":1,"wal-append.queue_time_us":243995}
I20260709 13:07:09.458667 21215 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:07:09.451179 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.462529 23513 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:07:09.469724 24092 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:09.470577 21147 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:07:09.473788 24091 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:07:09.474100 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48444 (request call id 2) took 1493 ms. Trace:
I20260709 13:07:09.474503 21801 rpcz_store.cc:276] 0709 13:07:07.980581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.980832 (+   251us) service_pool.cc:224] Handling call
0709 13:07:09.473582 (+1492750us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54005d5f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42522735 00007f7f4251a9fa 00007f7f474e24c9 00007f7f40a5ec54 00007f7f40a5ebe2 00007f7f40a5eb11 00007f7f40a5ea8c 00007f7f40a5dd42 00007f7f4085748c 00007f7f408568c6
0709 13:07:09.473909 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425329536}
I20260709 13:07:09.475836 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:09.441344 23987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.161s	user 0.006s	sys 0.030s
I20260709 13:07:09.463199 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:09.485321 21218 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:09.490590 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:09.491251 21140 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:09.492447 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:09.493753 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:09.494786 23895 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:09.495702 23373 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:09.496162 23373 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.496562 23373 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:09.497457 24047 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.148s	user 0.000s	sys 0.002s
I20260709 13:07:09.499073 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 356) took 2939 ms. Trace:
W20260709 13:07:09.486829 23987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.254s	user 0.007s	sys 0.031s
I20260709 13:07:09.507467 24026 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:07:09.502627 24094 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:07:09.507948 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:09.508255 24026 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:09.489095 23945 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.509037 24040 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:09.509547 24040 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.504617 23995 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:07:09.509557 24026 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.510366 24040 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:09.510681 23936 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.511241 23936 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.513628 23936 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.499497 21140 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:09.501516 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:09.519071 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:09.500972 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:09.509030 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:09.499511 21067 rpcz_store.cc:276] 0709 13:07:06.559184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.948657 (+1389473us) service_pool.cc:224] Handling call
0709 13:07:07.949009 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.950558 (+  1549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.950599 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:07.951102 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.098477 (+147375us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:08.098512 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.098573 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.496520 (+1397947us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:09.498529 (+  2009us) raft_consensus.cc:1685] finished
0709 13:07:09.498547 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.498853 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24976,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":1639,"spinlock_wait_cycles":4386560,"thread_start_us":140927,"threads_started":1,"wal-append.queue_time_us":367674}
I20260709 13:07:09.526913 21141 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:09.526891 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:09.535465 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 66801.1us
W20260709 13:07:09.559162 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56145.3us
I20260709 13:07:09.535714 23895 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:09.532234 23936 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:09.570052 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:09.573267 21220 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:09.581137 21218 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:09.582096 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:09.576857 24106 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:09.587006 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 103086us
I20260709 13:07:09.593763 21214 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:09.600083 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.605016 23373 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.587144 23616 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:07:09.609458 23986 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:09.610723 23616 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:09.611104 23616 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:09.611851 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 102944us
I20260709 13:07:09.612830 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.613190 23616 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.611888 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:09.608702 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:09.614799 24102 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:07:09.637035 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:09.643460 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.645215 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2032) took 1011 ms. Trace:
I20260709 13:07:09.646996 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:09.585604 24103 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:07:09.660313 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:09.660435 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:09.671161 21187 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 2.897s (SENT)
I20260709 13:07:09.671633 23616 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:09.674135 24027 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.675567 21215 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:09.675724 21221 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:09.658339 21142 rpcz_store.cc:276] 0709 13:07:08.580073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.587498 (+  7425us) service_pool.cc:224] Handling call
0709 13:07:09.545671 (+958173us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54002b48d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:09.591690 (+ 46019us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481121024}
I20260709 13:07:09.676266 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:09.680848 21185 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:09.681926 21219 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:09.663107 24048 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:09.677937 23868 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.685230 21218 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
W20260709 13:07:09.686278 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55897.7us
I20260709 13:07:09.688989 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:09.692287 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.692768 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.693117 21069 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:09.690346 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46060.5us
I20260709 13:07:09.686450 24040 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.695411 21217 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:07:09.698825 21067 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:09.702633 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:09.700331 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 91837.6us
I20260709 13:07:09.680296 24089 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:07:09.704877 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:09.705576 21072 raft_consensus.cc:2368] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6.
I20260709 13:07:09.704855 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.710624 21212 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:09.715334 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.716823 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.714223 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:09.718983 21069 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:09.718451 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 47794.3us
I20260709 13:07:09.718756 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:09.699906 23887 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:09.732753 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.738008 21067 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:09.744549 21143 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:09.746390 21068 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:07:09.731256 24081 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:09.717900 23868 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.748203 21213 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:09.751763 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.749352 21185 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:09.755164 21071 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.757419 23868 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:09.739828 24040 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:09.762882 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65925.1us
I20260709 13:07:09.772493 21795 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:09.773765 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:09.774516 21800 raft_consensus.cc:2368] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8.
I20260709 13:07:09.775710 23373 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:09.776448 21068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39510
I20260709 13:07:09.777281 21068 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:07:09.777719 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 87620.9us
I20260709 13:07:09.779176 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:09.776757 21212 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:09.779690 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:09.780077 21793 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:09.781088 21111 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:09.783205 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:09.783948 24090 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:09.784888 23978 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:09.787096 24040 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:09.787726 24040 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.788250 24040 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:09.788460 21799 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:07:09.789631 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64648.9us
I20260709 13:07:09.779731 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:09.777720 21068 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:09.791036 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:09.794744 21221 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:09.799355 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:09.800742 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:09.801024 24090 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:09.802073 24090 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:09.802184 23995 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:09.803650 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:09.799571 24081 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:09.825687 21213 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:09.804946 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.806931 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.828256 21064 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.807675 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.830735 21072 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
I20260709 13:07:09.809141 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.831944 21066 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.833134 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.834599 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.808384 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.811717 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:09.814333 21216 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:09.816310 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:09.818555 21071 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:07:09.818615 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.836392 21216 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:09.821583 21185 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:09.806324 23513 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:09.806114 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.845871 21065 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:09.829201 21214 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:07:09.820312 20967 catalog_manager.cc:5697] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:09.837199 21071 pending_rounds.cc:85] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 3
I20260709 13:07:09.838399 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:09.847093 21071 pending_rounds.cc:107] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:07:09.847920 21071 raft_consensus.cc:2889] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:09.850332 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 79860.6us
I20260709 13:07:09.827365 24089 raft_consensus.cc:2764] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:07:09.854517 21069 raft_consensus.cc:2410] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:09.858335 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.859799 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.860931 21066 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:09.862295 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.864089 21067 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:09.857892 23868 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:09.866961 21068 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:09.839212 24088 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:09.837741 24027 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:09.802999 24090 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:09.892310 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.889999 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.893998 21069 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.889912 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.895517 21068 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:09.895818 21795 raft_consensus.cc:2376] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9. Re-sending same reply.
I20260709 13:07:09.895951 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.845250 23513 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:09.896807 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 58518.1us
I20260709 13:07:09.826982 23960 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:07:09.852769 23980 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:09.864953 23887 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:07:09.896930 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 113793us
W20260709 13:07:09.904625 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 101010us
I20260709 13:07:09.903779 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.905515 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:09.906793 21071 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:09.907341 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 91196.8us
I20260709 13:07:09.908005 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.908646 21063 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:09.908205 21184 leader_election.cc:400] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8.
I20260709 13:07:09.910530 21184 leader_election.cc:403] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:09.911618 21066 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:09.914716 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:09.916821 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 137205us
I20260709 13:07:09.880417 23995 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:07:09.896831 23978 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:09.902129 23373 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:09.920290 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:09.924826 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.926203 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:09.927040 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:09.905428 23980 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:09.913079 24109 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:09.929395 21069 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:09.931273 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:09.929404 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:09.916884 20967 catalog_manager.cc:5697] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 2 to 5, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 5 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:09.936054 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:09.938915 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:09.936131 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:09.939700 21067 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:09.940858 21795 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:09.944487 20954 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0d78b35ac92d402db8a5e3fe45be52b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b70fc04de37e4239a8336534f0c9c0fb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:07:09.945461 20954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46873.4us
I20260709 13:07:09.930871 23887 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:07:09.950240 24090 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:09.925868 23373 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.931241 23980 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:09.951555 23373 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:09.945537 23995 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:09.951022 24090 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:09.962298 21068 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:09.965437 21068 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:09.931669 24109 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.968422 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:09.966498 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 44687.1us
I20260709 13:07:09.966945 21795 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:09.922899 24027 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 13:07:09.967561 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 43638.2us
I20260709 13:07:09.938273 24103 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:09.971418 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56064us
I20260709 13:07:09.938426 24106 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:09.960197 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:09.984686 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:10.011863 21063 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:09.953428 24088 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:09.964313 23868 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:10.017691 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:10.020416 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 93442.1us
I20260709 13:07:09.965235 24090 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:09.951110 24040 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:09.972821 24109 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:10.034298 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.045506 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:10.045754 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 73180.6us
I20260709 13:07:10.065712 21796 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:10.033756 24026 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:10.065104 21069 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:10.014058 24106 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.070808 21068 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.019968 23995 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:10.087978 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.964681 23943 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.195s
I20260709 13:07:10.040290 24090 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:09.941018 23513 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.102705 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:10.098088 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 173358us
I20260709 13:07:10.037899 23978 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.082509 24081 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:10.080610 23868 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:10.119867 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 86302.9us
I20260709 13:07:10.087729 24106 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:10.146819 24114 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:10.019550 23761 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:07:10.134600 23995 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:10.135807 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 117319us
I20260709 13:07:10.105515 24027 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:10.137671 24040 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:10.149928 23616 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:10.151831 20967 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 2 to 3, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 3 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:10.153079 21068 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:10.159811 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:10.164908 24089 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.173481 21067 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:07:10.180823 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 171966us
I20260709 13:07:10.225978 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.233938 20967 catalog_manager.cc:5697] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 2 to 6, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:10.241357 21067 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:07:10.179690 23943 consensus_queue.cc:1243] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:10.248221 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:10.240669 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 137811us
I20260709 13:07:10.240489 23887 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8.
I20260709 13:07:10.239990 24040 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:10.243731 23660 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:10.277248 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 57073.7us
W20260709 13:07:10.280826 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58227.8us
I20260709 13:07:10.253535 23943 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:07:10.294057 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 97782.3us
W20260709 13:07:10.299593 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 139940us
I20260709 13:07:10.312475 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:10.279723 24110 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:07:10.239126 24089 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.318514 21068 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:10.329285 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:10.329955 21215 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:10.343092 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 333603us
W20260709 13:07:10.357832 24116 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:07:10.322312 24026 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:10.358568 21216 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:10.359589 24111 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:10.370805 20967 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:10.376166 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:10.341580 23887 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:10.369681 24136 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:07:10.384217 21187 leader_election.cc:400] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8.
I20260709 13:07:10.369446 24026 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:10.392660 24103 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:10.393298 24103 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:10.399662 24115 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:07:10.401690 23616 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.402338 23616 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.404634 24089 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:10.405534 24125 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:07:10.385746 24117 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 13:07:10.405972 20967 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:10.414865 20967 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 4 to 5, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 5 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:10.430768 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 48514.5us
W20260709 13:07:10.436058 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50973.8us
I20260709 13:07:10.446059 21215 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:10.436854 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:10.323012 23868 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:10.436936 20967 catalog_manager.cc:5697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:10.456969 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:10.458427 24140 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:10.465770 24136 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.466392 24136 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.468775 24136 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:10.481371 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.481658 23616 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:10.482535 23616 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:10.488567 23980 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.001s
I20260709 13:07:10.489223 23980 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:10.458958 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:10.498921 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:10.499895 21214 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:10.438822 24114 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:10.483028 23616 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:10.508215 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 162741us
I20260709 13:07:10.486011 24090 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:10.515398 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 180858us
I20260709 13:07:10.517071 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.518119 24136 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:10.519205 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:10.528126 24121 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:10.528878 24121 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:10.530139 24121 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:10.510051 23616 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:10.539348 21221 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:10.519596 23761 raft_consensus.cc:2764] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:07:10.554667 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 95776.8us
W20260709 13:07:10.555694 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.555989 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:10.566008 24147 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.538501 23978 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:07:10.572569 23373 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.613s	user 0.013s	sys 0.000s
W20260709 13:07:10.584896 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 217821us
W20260709 13:07:10.580193 24134 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:07:10.590281 24114 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.576227 23980 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:10.593227 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:10.596545 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:10.579778 23978 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:10.602778 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:10.584494 24151 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.053s	user 0.003s	sys 0.002s
I20260709 13:07:10.601238 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:10.577438 23995 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:07:10.605450 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.612840 23980 raft_consensus.cc:2764] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:07:10.616096 24109 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.529s	user 0.013s	sys 0.000s
I20260709 13:07:10.616707 24109 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:10.622927 24127 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:07:10.624658 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:10.625268 21072 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:10.587406 23373 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.629554 23373 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:10.580468 24147 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:10.632520 24147 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:10.632707 24088 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.609s	user 0.009s	sys 0.001s
I20260709 13:07:10.633167 24088 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:10.633333 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:10.634564 21147 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:10.634336 23978 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.635550 21113 leader_election.cc:400] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6.
I20260709 13:07:10.635814 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:10.636832 21144 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:10.635864 21113 leader_election.cc:403] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:10.639667 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 62564.5us
I20260709 13:07:10.658440 24109 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:10.662359 24133 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:07:10.654950 21142 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
W20260709 13:07:10.674211 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 104274us
I20260709 13:07:10.640430 24146 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:10.678578 21144 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:10.681149 21072 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:10.684468 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.687273 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:10.688042 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52653.4us
I20260709 13:07:10.680418 24114 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:07:10.695662 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:10.684122 24147 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:10.699535 24129 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:10.737692 24129 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.738121 24129 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:10.699863 24146 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:10.715108 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.703219 24103 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:10.707609 24106 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.001s
I20260709 13:07:10.707897 23887 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:10.709185 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:10.711760 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:10.715422 23985 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:10.751925 21142 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:10.715610 23980 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:10.717859 24114 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6.
I20260709 13:07:10.715466 23987 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:07:10.758270 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:10.698037 24090 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:10.736474 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 55705.4us
I20260709 13:07:10.686815 24081 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:07:10.743170 24027 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.001s
I20260709 13:07:10.760682 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:10.755466 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 147756us
I20260709 13:07:10.763089 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:10.769636 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:10.773252 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58209.6us
I20260709 13:07:10.767554 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.774430 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.754429 24106 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:10.777603 24154 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:07:10.777364 23952 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:07:10.758675 23955 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:10.783687 23978 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:10.784145 23978 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:10.784543 23978 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:10.784742 23616 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.785997 21139 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:07:10.787425 24090 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:10.739609 24146 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.787832 24090 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.789621 21141 raft_consensus.cc:2410] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:07:10.797808 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.788861 23887 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:10.817286 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:10.792148 24106 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:10.798956 24090 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:10.815024 23952 raft_consensus.cc:2764] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:07:10.823522 23887 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:10.824568 23887 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.826082 23513 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:10.819988 21187 leader_election.cc:400] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:07:10.828469 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.829980 24088 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:10.830229 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:10.820529 23373 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:10.831530 24089 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:10.831898 23373 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:10.832605 24103 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:10.833247 23373 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.834117 24112 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:10.834600 24112 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:10.840370 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:10.845242 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:10.807322 23616 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:10.793707 24027 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:07:10.849591 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55902.8us
I20260709 13:07:10.815122 24140 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:10.854526 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:10.870777 21218 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:10.871129 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.861418 21142 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:10.881021 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.854140 24103 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:10.855324 24147 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:10.886646 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.892977 21798 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:07:10.887338 24161 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:07:10.891515 24103 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:10.897960 24103 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:10.904397 21218 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
W20260709 13:07:10.907728 24040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.012s	sys 0.011s
I20260709 13:07:10.915894 21147 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:10.850474 23943 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 13:07:10.916100 24040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.013s	sys 0.012s
I20260709 13:07:10.920619 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:10.921591 24132 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.545s	user 0.003s	sys 0.000s
I20260709 13:07:10.923182 24040 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:10.924065 21216 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:10.879703 24112 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:10.933513 24090 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:10.936014 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 118382us
I20260709 13:07:10.937196 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.936172 24090 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:10.940678 24081 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.941741 24090 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:10.941377 24081 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.942855 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:10.943686 21139 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:10.944849 24081 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:10.947132 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:10.947968 21147 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:10.949025 24090 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:10.951545 24090 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.952800 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.965626 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:10.924342 24040 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:10.971560 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.972273 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:10.972561 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.972774 24088 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:10.972978 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:10.988619 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:10.989259 21142 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:10.973626 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:10.973773 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.974408 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:10.974797 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:10.975181 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:10.981707 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.982895 21139 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:10.985415 21214 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:10.990932 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.992923 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:10.995498 21797 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:10.994877 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.996253 21792 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:10.994405 23952 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:10.962509 24129 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:10.997762 21800 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:10.997644 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:11.000690 21141 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:11.000838 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.997418 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:10.997680 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:11.005102 21142 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:07:11.007081 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 310) took 1333 ms. Trace:
I20260709 13:07:10.987901 23985 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:10.997418 23952 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:11.020699 23952 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:11.019869 21216 rpcz_store.cc:276] 0709 13:07:09.673467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.685045 (+ 11578us) service_pool.cc:224] Handling call
0709 13:07:09.695566 (+ 10521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.814237 (+118671us) spinlock_profiling.cc:243] Waited 63.4 ms on lock 0x7b54002c04a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42493dc0 00007f7f4251e1c0 00007f7f4251f231 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c
0709 13:07:10.358013 (+543776us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54002c04a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4249a06f 00007f7f42512f60 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62
0709 13:07:10.923594 (+565581us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b58007c43d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f424780aa 00007f7f424fa447 00007f7f42512fe7 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4
0709 13:07:11.006623 (+ 83029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:11.006665 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:11.006718 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:11.006769 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.006874 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.007014 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138,"spinlock_wait_cycles":1747538944}
I20260709 13:07:11.019086 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:10.997188 24090 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:11.021730 21794 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:11.009868 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:11.023123 21140 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:10.994936 23955 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:07:11.010944 21220 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:11.028401 23952 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.027480 21796 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:11.031108 23952 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:11.032694 21140 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:11.034178 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:11.034201 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:11.035374 23895 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:07:11.036454 24106 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:11.041970 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:11.043545 23978 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:11.042835 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:11.049472 21800 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:11.050509 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:11.078480 21146 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:11.088274 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:11.049461 24106 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:11.093441 24145 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.004s	sys 0.004s
W20260709 13:07:11.101397 24145 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.005s	sys 0.004s
I20260709 13:07:11.105464 21141 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:11.103919 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:11.101756 23868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.003s	sys 0.012s
W20260709 13:07:11.110747 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 173712us
I20260709 13:07:11.121848 21216 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:07:11.128554 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:11.118602 23868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.004s	sys 0.013s
I20260709 13:07:11.120550 23978 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:11.133123 23985 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:11.136014 24107 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:11.136416 24107 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:11.137282 24107 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.138592 24107 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:11.129674 24167 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:11.140632 24109 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.141119 24109 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:11.141608 23955 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:11.130131 24157 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:11.144745 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:11.145718 21216 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:11.174722 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:11.111099 24129 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:11.178304 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 869) took 1450 ms. Trace:
I20260709 13:07:11.178503 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1854) took 1068 ms. Trace:
I20260709 13:07:11.181397 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 376) took 1253 ms. Trace:
I20260709 13:07:11.181892 21066 rpcz_store.cc:276] 0709 13:07:09.928024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.928168 (+   144us) service_pool.cc:224] Handling call
0709 13:07:09.935916 (+  7748us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.951677 (+ 15761us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:10.065506 (+113829us) log.cc:844] Serialized 14 byte log entry
0709 13:07:10.661681 (+596175us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:11.181035 (+519354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:11.181089 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:11.181139 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.181200 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.181344 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":921701,"prepare.queue_time_us":137732,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"spinlock_wait_cycles":49514880,"thread_start_us":271609,"threads_started":2,"wal-append.queue_time_us":331378}
I20260709 13:07:11.185963 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:11.186463 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:11.186836 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 108738us
W20260709 13:07:11.187017 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.189908 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:11.190680 21068 raft_consensus.cc:2410] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:11.186609 21212 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:11.143271 24109 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:11.195573 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:11.194353 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 378) took 1178 ms. Trace:
I20260709 13:07:11.181597 21069 rpcz_store.cc:276] 0709 13:07:10.110185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.110393 (+   208us) service_pool.cc:224] Handling call
0709 13:07:10.153857 (+ 43464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.154336 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.154376 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:10.787776 (+633400us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b2c000014c8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:07:10.788241 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:07:10.917300 (+129059us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:10.917333 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.917379 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.144672 (+227293us) raft_consensus.cc:1685] finished
0709 13:07:11.144702 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.178418 (+ 33716us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13906,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6925,"spinlock_wait_cycles":32471936,"thread_start_us":128633,"threads_started":1,"wal-append.queue_time_us":249150}
I20260709 13:07:11.144176 23955 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:07:11.199049 21071 rpcz_store.cc:276] 0709 13:07:10.015947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.016305 (+   358us) service_pool.cc:224] Handling call
0709 13:07:10.039636 (+ 23331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.039833 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:10.246827 (+206994us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.193994 (+947167us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:11.194055 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:11.194104 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.194173 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.194294 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":943955,"prepare.queue_time_us":93851,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":149382,"threads_started":1}
I20260709 13:07:11.211069 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:11.211777 21068 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:11.212738 24109 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:11.212821 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:11.213277 24109 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:11.213516 21068 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:11.213714 24109 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:11.213768 21216 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:11.215162 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:11.228752 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:11.230669 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.231520 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:11.232003 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.232369 21069 raft_consensus.cc:2410] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:07:11.233223 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1855) took 1122 ms. Trace:
I20260709 13:07:11.233371 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:11.230515 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:11.235127 21071 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:11.234411 21066 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:11.233760 21063 rpcz_store.cc:276] 0709 13:07:10.110590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.123087 (+ 12497us) service_pool.cc:224] Handling call
0709 13:07:10.161544 (+ 38457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.162001 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.162040 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:10.356198 (+194158us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b4800080358. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:07:10.521686 (+165488us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b2c000014c8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:07:10.544821 (+ 23135us) log.cc:844] Serialized 26 byte log entry
0709 13:07:10.652711 (+107890us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:10.653443 (+   732us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.232742 (+579299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.232868 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.232926 (+    58us) raft_consensus.cc:1685] finished
0709 13:07:11.232949 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.233137 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1427178,"prepare.queue_time_us":211,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":6270,"raft.run_cpu_time_us":2873,"raft.run_wall_time_us":184145,"spinlock_wait_cycles":23381376,"thread_start_us":79145,"threads_started":1,"wal-append.queue_time_us":99664}
I20260709 13:07:11.238444 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:11.207698 23871 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:11.268939 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:11.269294 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:11.270704 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:11.271348 21071 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:11.271203 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:11.269971 21063 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
W20260709 13:07:11.279069 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 63988us
I20260709 13:07:11.280874 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:11.243829 23985 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:11.264768 23868 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:11.265829 23986 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:07:11.290536 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 110034us
W20260709 13:07:11.292117 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 754118us
I20260709 13:07:11.296103 21068 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:11.198523 24106 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.297597 21068 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:11.283445 23995 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:11.288995 24040 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:11.303562 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:11.304312 21068 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:07:11.306190 24180 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:07:11.292748 24150 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.289342 23985 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:11.285807 23871 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.301355 24088 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:11.325102 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:11.326758 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1857) took 1215 ms. Trace:
I20260709 13:07:11.321920 23986 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:07:11.331522 24179 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 13:07:11.332392 24176 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:07:11.334198 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 375) took 1434 ms. Trace:
I20260709 13:07:11.323596 24088 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:11.313472 23868 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:07:11.343753 21146 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:11.347759 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:11.346443 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:11.337597 21065 rpcz_store.cc:276] 0709 13:07:09.899668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.899951 (+   283us) service_pool.cc:224] Handling call
0709 13:07:09.910268 (+ 10317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.915423 (+  5155us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:10.064990 (+149567us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.333879 (+1268889us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:11.333932 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:11.333979 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.334037 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.334145 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":903006,"prepare.queue_time_us":153537,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":4082,"raft.queue_time_us":920442,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"thread_start_us":440097,"threads_started":3,"wal-append.queue_time_us":368619}
W20260709 13:07:11.354272 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52525.1us
I20260709 13:07:11.354604 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:11.344471 23986 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:11.353682 24129 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:11.320649 24150 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:11.368487 24150 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:11.368919 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:11.362310 21185 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:11.371101 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 932) took 1082 ms. Trace:
I20260709 13:07:11.371872 21221 rpcz_store.cc:276] 0709 13:07:10.289046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.293515 (+  4469us) service_pool.cc:224] Handling call
0709 13:07:10.311697 (+ 18182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.541236 (+229539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.370872 (+829636us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540015cf54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:11.371041 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522781952}
I20260709 13:07:11.373139 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:11.374293 23986 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:11.346163 24088 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:11.361724 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:11.364167 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:11.361785 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:11.400906 21795 raft_consensus.cc:2393] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:11.403262 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:11.405408 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 43688.3us
I20260709 13:07:11.371533 23761 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:11.407181 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:11.361304 24129 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:11.416085 21146 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:11.407691 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1859) took 1296 ms. Trace:
I20260709 13:07:11.416610 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:11.417377 21072 rpcz_store.cc:276] 0709 13:07:10.111767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.696380 (+584613us) service_pool.cc:224] Handling call
0709 13:07:10.715655 (+ 19275us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:11.323146 (+607491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:11.323181 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:11.332385 (+  9204us) log.cc:844] Serialized 52 byte log entry
0709 13:07:11.407276 (+ 74891us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:11.407382 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.407451 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.407494 (+    43us) raft_consensus.cc:1685] finished
0709 13:07:11.407536 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.407647 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":519781,"raft.queue_time_us":562379,"spinlock_wait_cycles":7417216,"thread_start_us":151791,"threads_started":2,"wal-append.queue_time_us":68928}
I20260709 13:07:11.422530 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:11.432137 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:11.434069 21138 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:11.431519 24107 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:11.464711 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 896) took 1527 ms. Trace:
I20260709 13:07:11.410483 24040 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:11.410672 23761 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:11.435441 24190 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:07:11.475409 24194 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:07:11.500089 21071 raft_consensus.cc:2410] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:07:11.491007 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 118613us
I20260709 13:07:11.515256 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:11.516562 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62757.3us
I20260709 13:07:11.517710 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:11.465176 21064 rpcz_store.cc:276] 0709 13:07:09.937452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.937589 (+   137us) service_pool.cc:224] Handling call
0709 13:07:09.964378 (+ 26789us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.008959 (+ 44581us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:10.026887 (+ 17928us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.338453 (+1311566us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.343109 (+  4656us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.343347 (+   238us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:11.464049 (+120702us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.464403 (+   354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.464475 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.464515 (+    40us) raft_consensus.cc:1685] finished
0709 13:07:11.464525 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.464655 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":973179,"prepare.queue_time_us":162704,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":3474,"raft.queue_time_us":958701,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":17376,"thread_start_us":443767,"threads_started":3,"wal-append.queue_time_us":322378}
I20260709 13:07:11.452126 23969 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:11.448846 24186 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:11.520498 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:11.419639 24114 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:07:11.470127 24040 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.375196 23985 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.449016 24189 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:11.490327 23868 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.532436 21801 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:11.559574 21072 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:11.521162 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:11.522603 23895 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:07:11.522779 24192 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:11.450830 24141 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:07:11.436951 24129 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:11.553591 21796 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:11.535930 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:11.555686 23969 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:11.574307 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 287099us
I20260709 13:07:11.575330 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:11.555011 24040 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:11.559031 24143 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:11.591010 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:11.583554 23660 raft_consensus.cc:2764] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:07:11.597414 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 118547us
I20260709 13:07:11.569679 23868 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:11.600232 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:11.593343 24140 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:11.586346 23871 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:11.587342 24026 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.560345 23761 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.573686 24192 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:11.607457 24140 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.615967 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:11.606717 24193 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:11.609634 23922 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:11.609334 24197 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:07:11.678447 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62638.8us
W20260709 13:07:11.678114 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 157692us
I20260709 13:07:11.682926 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:11.671375 23761 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:11.673657 24168 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:07:11.696657 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.673926 24186 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:11.695490 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 91463us
I20260709 13:07:11.709596 21221 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:11.676159 24140 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:11.708372 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:11.670804 24107 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:11.677210 23922 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:11.690191 23660 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:11.600835 24143 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.718173 21217 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:11.613061 24026 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.717765 23922 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:11.609905 23969 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.725992 21213 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:07:11.790549 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56710.6us
I20260709 13:07:11.692462 24189 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:11.824235 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:11.824687 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 129755us
I20260709 13:07:11.825984 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:11.715420 24197 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.718021 23660 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:11.726341 24208 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:11.782527 24213 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:11.827514 24143 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:11.832836 23960 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:11.809270 24214 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:07:11.843852 23660 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:11.839735 24198 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:07:11.856056 24200 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 13:07:11.879166 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:11.883881 24203 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.213s	user 0.001s	sys 0.001s
W20260709 13:07:11.882551 24205 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:07:11.858585 23936 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:11.907259 23936 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.911005 23936 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:11.881261 23868 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:11.916657 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67040.2us
I20260709 13:07:11.847489 24107 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.850441 24186 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.851498 24208 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.891789 24198 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:11.848523 24197 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:11.933514 24208 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:11.940232 23660 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:11.941488 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2056) took 1242 ms. Trace:
I20260709 13:07:11.945554 21144 rpcz_store.cc:276] 0709 13:07:10.698662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.698817 (+   155us) service_pool.cc:224] Handling call
0709 13:07:10.748489 (+ 49672us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:10.799890 (+ 51401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.799941 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:10.816257 (+ 16316us) log.cc:844] Serialized 52 byte log entry
0709 13:07:11.600544 (+784287us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b18000e64a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:07:11.922141 (+321597us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:11.922181 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.922542 (+   361us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.941161 (+ 18619us) raft_consensus.cc:1685] finished
0709 13:07:11.941182 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.941406 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25317,"raft.queue_time_us":39289,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":12428,"spinlock_wait_cycles":28987136,"thread_start_us":321391,"threads_started":1,"wal-append.queue_time_us":1116362}
I20260709 13:07:11.947499 24192 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:11.180380 21070 rpcz_store.cc:276] 0709 13:07:09.727925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.728043 (+   118us) service_pool.cc:224] Handling call
0709 13:07:09.744131 (+ 16088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.753293 (+  9162us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:09.914094 (+160801us) log.cc:844] Serialized 14 byte log entry
0709 13:07:10.216533 (+302439us) log.cc:844] Serialized 14 byte log entry
0709 13:07:10.239601 (+ 23068us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.168547 (+928946us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b1800200380. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287 00007f7f42537208
0709 13:07:11.168734 (+   187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:11.168818 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:11.168890 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.168983 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.178230 (+  9247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28438,"prepare.queue_time_us":180040,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":3336,"raft.queue_time_us":1005233,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":13950,"spinlock_wait_cycles":24087552,"thread_start_us":325037,"threads_started":2,"wal-append.queue_time_us":341577}
I20260709 13:07:11.942881 23660 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:11.327162 21067 rpcz_store.cc:276] 0709 13:07:10.111247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.272828 (+161581us) service_pool.cc:224] Handling call
0709 13:07:10.287630 (+ 14802us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.313655 (+ 26025us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:10.337628 (+ 23973us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.326229 (+988601us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.326449 (+   220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:11.326496 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:11.326531 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.326599 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.326706 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":657899,"prepare.queue_time_us":23929,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":68,"raft.queue_time_us":675654,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":10645,"thread_start_us":222108,"threads_started":2,"wal-append.queue_time_us":194520}
I20260709 13:07:11.956774 21113 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:11.966781 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:11.976541 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47187.5us
I20260709 13:07:11.966215 21767 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:11.989858 24197 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:11.991436 24219 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:11.992053 24219 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:11.955191 24088 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.000s
I20260709 13:07:11.893744 24189 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:11.991151 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.006351 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:12.008168 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 70158.9us
W20260709 13:07:12.014263 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 719386us
I20260709 13:07:12.015692 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:11.979668 23985 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:12.019793 21215 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:12.020565 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:12.026451 21801 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:11.994287 24219 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:12.009305 23936 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:12.009436 24130 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:12.031989 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:12.029412 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:12.009058 24026 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:12.039503 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:11.996951 24088 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.039708 21797 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:11.981164 24208 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:07:12.039086 23936 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.047930 23695 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:12.048619 23936 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:12.018893 23868 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:07:12.050017 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:12.051107 24040 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.051541 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:12.052891 24130 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:12.053284 24040 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:12.053578 24130 raft_consensus.cc:2764] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:07:12.053803 24198 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.056154 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:12.057042 23695 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:12.040221 24109 consensus_meta.cc:287] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.826s	user 0.015s	sys 0.000s
I20260709 13:07:12.057476 23695 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:12.057928 23695 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:12.057770 24109 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.060859 24109 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:12.062662 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:12.064013 21216 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:12.064957 24088 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:12.083125 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.085072 24088 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:12.085650 24088 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.088644 23943 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:12.089205 23943 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:12.090910 24088 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:12.091068 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:12.091897 21066 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:12.031904 24223 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.019443 23660 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:07:12.100704 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:12.108372 21069 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:12.105645 21071 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:12.106380 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:12.114440 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.123454 21070 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:12.034560 24226 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:07:12.128202 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:12.138902 21066 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
W20260709 13:07:12.110975 23955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.005s	sys 0.009s
W20260709 13:07:12.171379 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 115294us
I20260709 13:07:12.090314 23943 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:12.170218 23955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.006s	sys 0.009s
I20260709 13:07:12.142714 24027 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:12.110749 24140 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.105825 23985 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:07:12.197356 21794 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:12.103622 24223 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.188881 24027 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:12.217306 23922 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.228240 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:12.228415 20954 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0d78b35ac92d402db8a5e3fe45be52b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:07:12.224529 21146 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:07:12.229166 21072 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:12.229581 23922 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:12.230218 21144 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:12.225265 21147 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:07:12.225519 21138 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:07:12.231820 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:12.221094 24027 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:12.227423 21143 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:12.237099 24239 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:12.221331 23695 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.247509 21212 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:12.259094 21218 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:07:12.259548 21217 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:07:12.260941 24109 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:12.261988 24040 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.262522 24040 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.264829 24040 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:12.265388 24040 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:12.268112 24157 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.268467 24140 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:12.268718 24157 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.269531 24112 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:12.271648 24130 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:12.272557 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:12.273144 21219 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:12.356846 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 128677us
I20260709 13:07:12.358140 21215 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:07:12.360358 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:12.245445 23695 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:12.366330 23373 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:12.385483 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 153834us
I20260709 13:07:12.386447 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:12.388393 21219 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:12.389808 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:12.390427 21063 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:12.390556 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:12.385710 23373 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.399755 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:12.393747 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 115605us
I20260709 13:07:12.402830 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:12.241185 23995 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:12.403877 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 152506us
I20260709 13:07:12.405500 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:12.242033 24026 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:12.373590 24157 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:12.383841 24223 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:12.413528 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:12.415341 23695 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:12.417873 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:12.418546 21144 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:12.427911 21138 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:07:12.430137 24213 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.000s
I20260709 13:07:12.430591 24213 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:12.431813 21220 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:12.432650 21214 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:12.398722 24185 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:07:12.434877 23952 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:12.435328 23952 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:12.436194 23952 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.436499 24204 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:12.437330 23952 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:12.439662 24204 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.440284 24204 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.443642 24204 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:12.448222 24150 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:12.448992 24150 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:12.449179 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:12.451465 24026 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:12.395681 24111 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:12.509721 21146 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
W20260709 13:07:12.501974 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.247993 24129 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.161s
I20260709 13:07:12.239249 24239 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.166s
I20260709 13:07:12.408751 24109 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:12.506486 21063 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:12.508168 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:12.511518 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:12.536424 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.545414 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.513738 24226 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.553761 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:12.556838 24081 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:12.559656 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:12.507848 24157 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:12.563445 21800 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:07:12.409369 23995 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.543485 24204 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.544078 24130 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:12.555105 23695 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:07:12.565237 23761 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.865s	user 0.008s	sys 0.005s
W20260709 13:07:12.579548 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 169353us
I20260709 13:07:12.589368 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:12.589461 23373 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:12.590309 21142 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:12.595656 21213 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:12.569819 24109 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:12.580668 24130 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.606245 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:12.543459 24111 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.546772 24247 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:07:12.615437 21142 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:12.609449 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:12.609846 24103 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.169s	user 0.021s	sys 0.029s
I20260709 13:07:12.622248 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:12.624655 21801 raft_consensus.cc:2376] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6. Re-sending same reply.
W20260709 13:07:12.631948 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 208439us
I20260709 13:07:12.633526 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:12.583190 24027 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.638041 21142 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:12.634795 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:12.625337 24103 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.189s	user 0.021s	sys 0.030s
I20260709 13:07:12.613938 24130 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:12.576742 24226 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:12.585767 24204 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.616845 23987 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.615471 24109 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:12.611949 23761 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:12.660797 23995 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:12.634429 23985 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 13:07:12.664951 24245 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:07:12.671911 21795 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:07:12.704685 21795 pending_rounds.cc:85] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 6
I20260709 13:07:12.705133 21795 pending_rounds.cc:107] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:07:12.705461 21795 raft_consensus.cc:2889] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:12.675400 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:12.676975 24157 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:12.680831 24129 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:12.682349 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 588520us
I20260709 13:07:12.682806 23695 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:12.683734 24247 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:12.684118 24253 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:12.687224 21063 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:12.722378 21063 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:12.688289 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46068.2us
I20260709 13:07:12.690604 21065 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
W20260709 13:07:12.701640 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 92229.9us
I20260709 13:07:12.698721 23660 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:07:12.710285 24027 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:12.711704 24026 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:12.701082 23987 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.707351 21800 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:12.734944 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:12.740063 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:12.746685 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:12.742642 21063 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.748677 21063 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:12.751065 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:12.703814 23761 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.751924 21216 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:12.753526 23761 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:12.755992 21218 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:12.756891 23987 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:12.743607 23995 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:12.758745 23995 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.695250 24194 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:07:12.759091 23995 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:12.778422 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:12.740037 23695 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:12.722904 24129 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:12.786262 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:12.900524 21063 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:12.813297 21067 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:12.955816 21067 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:12.956218 21799 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:12.886059 21142 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:07:12.965116 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.966583 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 287769us
I20260709 13:07:12.967762 21111 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:12.968058 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:12.971185 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 247384us
I20260709 13:07:12.977293 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:12.978372 21147 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:12.783346 23660 consensus_queue.cc:1243] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:07:12.917544 24221 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:07:12.992991 21185 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:12.943639 24112 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.844367 23955 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.970870 21067 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.014067 21067 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:12.998847 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.825176 24204 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:13.003577 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:12.842563 24143 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:12.777735 24206 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.214s
I20260709 13:07:12.969609 24239 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.969831 24209 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.052582 24209 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:13.055001 24209 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:13.055892 24239 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:13.056396 24143 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:13.057147 24250 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
W20260709 13:07:13.058876 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.778326 24254 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.257s
I20260709 13:07:12.993948 24157 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:13.062170 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:13.067087 24258 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:07:12.994364 24088 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:13.090147 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.090837 21142 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:13.099933 21064 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:13.100389 21064 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:13.101229 21064 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.102190 21064 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:13.082468 23695 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:13.058964 23980 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:07:13.045029 23955 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.106925 21067 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:07:13.107353 21067 pending_rounds.cc:85] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 7
I20260709 13:07:13.107717 21067 pending_rounds.cc:107] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:07:13.107952 21067 raft_consensus.cc:2889] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:13.014323 24081 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:07:13.079360 24157 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:13.126735 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 64624.5us
I20260709 13:07:13.127802 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:13.114540 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:13.118371 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 125517us
W20260709 13:07:13.124331 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:13.123515 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 131543us
I20260709 13:07:13.138188 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:13.138867 21068 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:13.139742 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:13.140843 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:13.141630 21066 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:13.139755 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:13.146236 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.083732 24109 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:13.149260 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:13.044587 24112 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:13.147755 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:13.157243 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:13.158268 21063 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:13.091437 24088 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:13.109385 24107 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.107893 23955 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:13.167511 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.172729 21069 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:13.173244 21142 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:13.169698 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:13.166805 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:13.135998 24026 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:13.135509 24157 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:13.178326 21063 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:13.148588 24226 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:13.154066 24239 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:13.115139 24130 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:13.160218 24223 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:07:13.237567 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 330) took 1198 ms. Trace:
I20260709 13:07:13.238306 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 785) took 1235 ms. Trace:
I20260709 13:07:13.188320 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.243028 21212 rpcz_store.cc:276] 0709 13:07:12.040556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.062275 (+ 21719us) service_pool.cc:224] Handling call
0709 13:07:13.237405 (+1175130us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54002ebb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:13.237511 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423026560}
I20260709 13:07:13.244705 21142 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:13.231860 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:13.245072 21065 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:13.243546 21794 rpcz_store.cc:276] 0709 13:07:12.002926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.003054 (+   128us) service_pool.cc:224] Handling call
0709 13:07:12.006292 (+  3238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.197863 (+191571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.238042 (+1040179us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400701a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:13.238213 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438970240}
I20260709 13:07:13.168694 24088 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:13.217191 21066 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:13.208345 21063 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:07:13.258765 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.176328 24107 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.258209 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.260970 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:13.168601 24261 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:13.263739 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:13.265815 21066 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:13.266475 21069 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:13.193311 21068 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:13.263146 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:13.268076 21068 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:13.270284 21217 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:07:13.231998 24266 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:07:13.231392 24262 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:07:13.229035 24112 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:13.228220 24260 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:07:13.295126 21063 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:07:13.285161 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.300650 21063 pending_rounds.cc:85] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 2
I20260709 13:07:13.233714 24027 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:13.296074 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:13.296592 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 168888us
I20260709 13:07:13.297458 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:13.302306 21065 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 4: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:13.297389 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.688s (SENT) [suppressed 42 similar messages]
I20260709 13:07:13.265043 24261 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:13.300946 21063 pending_rounds.cc:107] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:07:13.306386 21063 raft_consensus.cc:2889] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:13.214982 24143 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:13.309401 21142 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:13.310675 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.311797 21142 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:07:13.319181 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.188392 23980 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:07:13.343044 21068 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:13.346879 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 199194us
I20260709 13:07:13.347503 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.349542 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.350411 21068 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:13.352069 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:13.352173 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:13.352684 21063 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:13.352980 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:13.299463 21066 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:13.355851 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.356565 21142 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:13.366858 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:13.375370 23995 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.000s
I20260709 13:07:13.387552 24282 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:13.387434 23995 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.346858 24027 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:13.388866 24194 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:13.389345 24194 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.389721 24194 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:13.389452 24027 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:13.390007 23995 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:13.395889 21064 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:13.359139 24129 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:13.412509 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2068) took 1002 ms. Trace:
I20260709 13:07:13.413131 21143 rpcz_store.cc:276] 0709 13:07:12.410457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.410573 (+   116us) service_pool.cc:224] Handling call
0709 13:07:12.415745 (+  5172us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:12.632932 (+217187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.632978 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:12.639315 (+  6337us) log.cc:844] Serialized 52 byte log entry
0709 13:07:12.767732 (+128417us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:12.767784 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.767881 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:13.412198 (+644317us) raft_consensus.cc:1685] finished
0709 13:07:13.412229 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:13.412426 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86013,"raft.queue_time_us":171920,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":14696,"thread_start_us":216534,"threads_started":2,"wal-append.queue_time_us":362304}
I20260709 13:07:13.394346 24274 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:13.417565 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:13.418562 21064 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:13.364775 24223 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:07:13.424922 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.428650 21068 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:07:13.429190 21068 pending_rounds.cc:85] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 3
I20260709 13:07:13.429556 21068 pending_rounds.cc:107] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:07:13.429538 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:13.430219 21068 raft_consensus.cc:2889] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:13.439574 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:13.440835 21142 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:13.398448 24275 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:07:13.443998 24275 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:13.444442 24275 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:13.445411 24275 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.446492 24275 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:13.455440 24264 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:13.456130 24264 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.456583 24264 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:13.461318 24279 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.462040 24279 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:13.464367 24277 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:07:13.421821 24274 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:13.483645 21068 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:07:13.485086 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:13.444499 24272 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:07:13.495553 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.487529 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:13.491880 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 138955us
I20260709 13:07:13.497984 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:13.498584 21792 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: []
W20260709 13:07:13.500447 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 83485.6us
I20260709 13:07:13.504666 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:13.506515 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.497239 21142 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:13.507215 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 209828us
W20260709 13:07:13.508706 21036 connection.cc:256] RPC call timeout handler was delayed by -2.7787223669998866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 13:07:13.509087 21036 connection.cc:256] RPC call timeout handler was delayed by -2.7202223669999057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 13:07:13.509147 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.510808 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.511683 21142 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:13.540311 21143 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:13.540802 21143 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:13.541294 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.489468 24274 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:13.431339 24261 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:13.572445 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.696270 24210 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:07:13.566391 21036 connection.cc:256] RPC call timeout handler was delayed by -2.5852223669999148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260709 13:07:13.704170 21036 connection.cc:256] RPC call timeout handler was delayed by -2.5555223669999805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
W20260709 13:07:13.704648 21036 connection.cc:256] RPC call timeout handler was delayed by -2.4232223669999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
W20260709 13:07:13.705159 21036 connection.cc:256] RPC call timeout handler was delayed by -2.3413223669999752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
W20260709 13:07:13.705715 21036 connection.cc:256] RPC call timeout handler was delayed by -2.0425223669999468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2338s for a response
I20260709 13:07:13.507198 23980 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:13.688627 21213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 339) took 1239 ms (client timeout 1617 ms). Trace:
I20260709 13:07:13.556320 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:13.702666 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 406634us
I20260709 13:07:13.712360 21141 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
W20260709 13:07:13.706278 21036 connection.cc:256] RPC call timeout handler was delayed by -2.0182223669999075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23650000000000004s for a response
W20260709 13:07:13.713706 21036 connection.cc:256] RPC call timeout handler was delayed by -1.9939223669999819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2392s for a response
I20260709 13:07:13.555711 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.514007 24130 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:13.711203 21213 rpcz_store.cc:269] 0709 13:07:12.448812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.448930 (+   118us) service_pool.cc:224] Handling call
0709 13:07:13.688439 (+1239509us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54001257d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:13.688565 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335187712}
W20260709 13:07:13.718590 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8526223669999808s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2549s for a response
W20260709 13:07:13.722636 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3981223669999281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3054s for a response
I20260709 13:07:13.720562 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.566934 21143 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:07:13.721390 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:13.723171 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2478223669999124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32210000000000005s for a response
I20260709 13:07:13.725879 21141 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:13.671599 24107 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:13.576200 24282 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:13.731717 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.731771 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.738377 21141 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:13.710853 23980 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:13.703828 24210 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:13.755935 24210 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.757542 24210 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:13.670528 24112 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.742544 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:13.772017 21141 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:07:13.744014 24283 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:07:13.576077 24293 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.164s
W20260709 13:07:13.749297 24157 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.532s	user 0.005s	sys 0.004s
I20260709 13:07:13.740177 24107 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:07:13.749908 23995 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:13.773751 21067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.012s	sys 0.019s
W20260709 13:07:13.778728 21067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.012s	sys 0.019s
I20260709 13:07:13.777518 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:13.796558 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:13.786675 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.779187 24088 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.768836 23980 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:13.779215 23995 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:13.822103 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.756335 24274 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.822259 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:13.826253 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.826486 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:13.833009 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56588.9us
I20260709 13:07:13.835508 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.836385 21141 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:13.837296 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:13.837648 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.774844 23960 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:13.858139 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.860122 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:13.861045 21141 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:13.863359 24304 raft_consensus.cc:2764] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:07:13.741051 24282 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:13.872354 23960 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:13.872049 24302 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:13.904927 20954 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0d78b35ac92d402db8a5e3fe45be52b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:07:13.906867 24303 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.907140 23980 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.908838 21070 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:07:13.910545 21141 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:13.910729 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:13.910818 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:13.802834 24157 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:13.954804 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 331) took 1077 ms. Trace:
I20260709 13:07:13.778064 24112 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:13.968168 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 57434.2us
I20260709 13:07:13.858530 24264 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:13.917464 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:13.976365 21068 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:13.919886 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:13.922961 21065 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:07:13.842856 24279 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:13.955062 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 117870us
W20260709 13:07:13.936816 24290 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:07:13.960963 21141 rpcz_store.cc:276] 0709 13:07:12.877016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.885817 (+1008801us) service_pool.cc:224] Handling call
0709 13:07:13.900882 (+ 15065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.954502 (+ 53620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.954732 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:13.913638 21071 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:13.996587 21140 raft_consensus.cc:2368] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:13.956007 24293 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:13.988533 24194 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.598s	user 0.007s	sys 0.006s
I20260709 13:07:14.009171 21792 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:13.984627 24107 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:13.926580 24302 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.010385 24150 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.030619 24274 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:14.031013 24150 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:14.032582 23960 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:14.034303 23995 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:13.937919 24303 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:14.036036 24306 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:07:13.972026 21142 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
W20260709 13:07:14.090184 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 231196us
I20260709 13:07:14.090720 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:14.037967 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 963) took 1075 ms. Trace:
I20260709 13:07:14.093750 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:14.098197 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:14.096354 24157 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:14.098815 21141 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:14.089167 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.100069 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:14.101665 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:14.095898 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 126624us
I20260709 13:07:14.099138 24286 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:14.102340 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:14.108011 24129 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:14.097404 21212 rpcz_store.cc:276] 0709 13:07:12.962179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.256426 (+294247us) service_pool.cc:224] Handling call
0709 13:07:13.294871 (+ 38445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.295215 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.295242 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:13.300429 (+  5187us) log.cc:844] Serialized 26 byte log entry
0709 13:07:13.697063 (+396634us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:13.697107 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.697174 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.037735 (+340561us) raft_consensus.cc:1685] finished
0709 13:07:14.037758 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.037893 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6342,"thread_start_us":327036,"threads_started":1,"wal-append.queue_time_us":384462}
I20260709 13:07:14.110047 24274 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:14.111589 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:14.113754 21068 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:14.113786 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:14.097625 24253 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:14.040078 24111 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:14.102388 24157 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:14.097852 24296 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:14.091758 24112 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:14.037781 24284 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.118789 21141 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:14.037228 24303 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:14.036818 24302 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:14.032610 24264 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:14.112334 23960 raft_consensus.cc:2764] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:07:14.124668 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:14.133263 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:14.024731 24107 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.134095 21141 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:14.023973 24194 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.038021 24293 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:14.145999 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.147174 24253 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:14.150676 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:14.151649 21144 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:14.152109 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:14.153939 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:14.156517 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 677) took 1585 ms. Trace:
I20260709 13:07:14.157227 21795 rpcz_store.cc:276] 0709 13:07:12.571068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.571202 (+   134us) service_pool.cc:224] Handling call
0709 13:07:12.571572 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.718349 (+146777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.156128 (+1437779us) spinlock_profiling.cc:243] Waited 99.9 ms on lock 0x7b5400662bd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:14.156433 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229659008}
I20260709 13:07:14.128751 24286 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:07:14.162953 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:14.170260 21068 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:14.134337 24112 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.171897 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:14.139230 24107 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:14.174072 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 789) took 1477 ms. Trace:
I20260709 13:07:14.174360 24310 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:14.175179 24264 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:14.176407 21794 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:14.177675 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:14.141180 23980 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:14.128896 24296 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.174508 21793 rpcz_store.cc:276] 0709 13:07:12.696460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.696635 (+   175us) service_pool.cc:224] Handling call
0709 13:07:12.713488 (+ 16853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:12.757211 (+ 43723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.757240 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:12.757734 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:07:12.975657 (+217923us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:12.975701 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.170176 (+1194475us) spinlock_profiling.cc:243] Waited 43.3 ms on lock 0x7b5400633dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:14.170261 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.170332 (+    71us) raft_consensus.cc:1685] finished
0709 13:07:14.170355 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.174005 (+  3650us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":232,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"spinlock_wait_cycles":99675904,"thread_start_us":217534,"threads_started":1,"wal-append.queue_time_us":342524,"wal-append.run_cpu_time_us":1589,"wal-append.run_wall_time_us":1001602}
I20260709 13:07:14.194128 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:14.168833 23960 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:14.191857 24310 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:14.197363 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:14.200392 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.205171 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 99289.6us
I20260709 13:07:14.135200 24284 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.173599 24286 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:07:14.194003 24264 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:14.197590 24253 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:14.184907 21144 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:14.194325 24193 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:14.211369 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:14.193614 24318 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.197975 24310 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:14.223716 24310 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:14.224239 24313 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:14.181651 24112 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.215024 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:14.216884 23980 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 13:07:14.228157 24314 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:07:14.208798 24274 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:14.229202 24274 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.230280 24309 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:14.230988 24296 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:14.230855 24309 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.231989 24274 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:14.217684 24150 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:14.218816 24261 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.197679 24157 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:07:14.233502 24261 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.220490 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:14.235806 21144 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:14.222627 21141 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:14.242525 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:14.252270 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51548.6us
I20260709 13:07:14.253124 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:14.254243 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:14.215767 24193 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:14.254983 21069 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:14.256343 21767 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:14.262609 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.273531 24298 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:14.274335 23980 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:14.274626 24253 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:14.274904 23980 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:14.258270 24313 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:14.275794 23980 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:14.249907 24317 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:14.277299 24210 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:14.275830 24284 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:14.278270 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.279317 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.280434 24264 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.280977 24210 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:14.284457 24210 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.285115 24112 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:14.286746 24293 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:14.247591 24302 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:14.286275 23980 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:14.257620 24242 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.742s	user 0.007s	sys 0.009s
I20260709 13:07:14.288872 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:14.290414 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.260461 24129 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:14.294931 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:14.302616 21066 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:14.309293 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:14.309790 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2077) took 1746 ms. Trace:
I20260709 13:07:14.310081 21066 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:14.310096 24112 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.311723 24112 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.314548 24112 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:14.296955 24313 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:14.310156 21145 rpcz_store.cc:276] 0709 13:07:12.548065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.548192 (+   127us) service_pool.cc:224] Handling call
0709 13:07:12.576169 (+ 27977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.719833 (+143664us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:12.747015 (+ 27182us) log.cc:844] Serialized 14 byte log entry
0709 13:07:12.747856 (+   841us) log.cc:844] Serialized 14 byte log entry
0709 13:07:12.760982 (+ 13126us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:12.761026 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:12.761073 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.876817 (+1115744us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b18000dbd00. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:14.293705 (+416888us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b5400424ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:14.293764 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.293902 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13193,"prepare.queue_time_us":106927,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":97,"raft.queue_time_us":85230,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":56789,"spinlock_wait_cycles":508155904}
I20260709 13:07:14.239090 24318 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.316862 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:14.316864 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:14.318099 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:14.318338 24318 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:14.318764 21147 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:14.319749 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:14.320858 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:14.321856 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:14.322600 21147 raft_consensus.cc:2410] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:07:14.303709 24286 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:14.323485 24286 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.323518 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:14.324014 24286 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:14.325409 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 679) took 1651 ms. Trace:
I20260709 13:07:14.325843 21797 rpcz_store.cc:276] 0709 13:07:12.674393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.674924 (+   531us) service_pool.cc:224] Handling call
0709 13:07:12.675363 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:12.720543 (+ 45180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.720586 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:12.749295 (+ 28709us) log.cc:844] Serialized 26 byte log entry
0709 13:07:12.843204 (+ 93909us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:12.843256 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.324991 (+1481735us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b54004934d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:14.325041 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.325106 (+    65us) raft_consensus.cc:1685] finished
0709 13:07:14.325118 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.325340 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5978,"raft.queue_time_us":6384,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":18811,"spinlock_wait_cycles":89034112,"thread_start_us":93447,"threads_started":1,"wal-append.queue_time_us":329719}
I20260709 13:07:14.328418 21799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 781) took 2849 ms. Trace:
I20260709 13:07:14.328833 21799 rpcz_store.cc:276] 0709 13:07:11.478918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.479088 (+   170us) service_pool.cc:224] Handling call
0709 13:07:11.520614 (+ 41526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.956683 (+1436069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.328129 (+1371446us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540046aad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:14.328319 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3301172608}
I20260709 13:07:14.343581 21145 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:14.352327 23943 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:14.362253 21066 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:14.297369 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:14.374022 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:14.375849 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78565.1us
I20260709 13:07:14.377140 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:14.378376 21145 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:14.378816 21218 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:14.375838 21066 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:14.383587 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:14.385682 21218 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:14.387887 21066 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:07:14.388319 21066 pending_rounds.cc:85] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 3
I20260709 13:07:14.388653 21066 pending_rounds.cc:107] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:07:14.388952 21066 raft_consensus.cc:2889] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:14.392820 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:14.393810 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:14.365778 24274 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.366752 24136 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:14.400461 24315 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.403061 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:14.408222 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:14.408350 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:14.412956 21064 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:14.417554 21218 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:14.384977 24027 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:14.385385 24318 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:14.390385 24281 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:14.387419 24303 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.389302 24143 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:14.433166 21220 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:14.385211 24089 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:14.363430 24310 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.395449 24129 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.458022 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:14.459059 21212 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:14.471774 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:14.474757 21070 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
W20260709 13:07:14.478147 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 90918.4us
I20260709 13:07:14.480255 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:14.483551 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 790) took 1767 ms. Trace:
I20260709 13:07:14.453706 24143 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.411298 24274 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.465005 24197 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.492944 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:14.493893 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:14.495141 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:14.499821 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:14.501438 21139 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:07:14.472721 24111 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:14.444850 24027 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.446581 24318 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.438656 24136 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.509632 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:14.486971 24330 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.489326 24143 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:14.546797 24136 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:14.468019 24089 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.467185 24193 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.544503 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:14.545328 24318 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:14.459921 24303 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.454094 24281 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.468245 24129 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.451905 24315 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:14.549662 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:14.726083 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.853269 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.916385 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.810974 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:14.914835 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:14.910918 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.674160 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.811779 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.658519 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.933212 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:14.493505 24254 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:14.820112 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:14.764367 24107 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.001s
I20260709 13:07:14.990222 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:14.990921 21070 raft_consensus.cc:2410] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:07:14.917876 24112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.015s	sys 0.018s
I20260709 13:07:14.905356 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2082) took 1951 ms. Trace:
W20260709 13:07:15.017611 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.066s	sys 0.117s
I20260709 13:07:14.822639 24284 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:15.022066 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.066s	sys 0.117s
I20260709 13:07:15.021987 21139 rpcz_store.cc:276] 0709 13:07:12.954165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.500821 (+1546656us) service_pool.cc:224] Handling call
0709 13:07:14.501097 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.501758 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.905261 (+403503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:15.030992 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.050370 21070 raft_consensus.cc:2410] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:07:15.035784 21146 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:07:14.786859 24027 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:14.545351 24109 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.778597 24255 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.617925 23980 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:15.057729 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:14.510417 24197 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:15.028520 24112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.015s	sys 0.019s
I20260709 13:07:15.006656 24157 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.992309 24331 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.998677 24332 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:15.087253 21215 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.022s	sys 0.027s
W20260709 13:07:15.117442 21215 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.023s	sys 0.027s
I20260709 13:07:14.542275 24111 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:15.089578 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.129469 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.130282 21212 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:15.046031 24107 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:15.106966 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 117006us
I20260709 13:07:15.111850 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.138981 21070 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:15.137687 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.140939 21069 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:15.020488 24254 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.144176 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.144817 21070 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:15.073683 24284 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:15.136736 23943 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.000s
I20260709 13:07:15.080819 24109 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.073231 24304 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.165860 24340 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:15.160152 24226 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.116854 24157 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.114573 24255 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:14.743307 24330 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.176044 21141 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
W20260709 13:07:15.169708 24207 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.025s	sys 0.046s
I20260709 13:07:15.194103 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.276190 21217 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:15.195874 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.196528 23952 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.204733 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2083) took 2249 ms. Trace:
I20260709 13:07:15.211966 24201 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:15.214458 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:15.221351 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.223965 24048 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:15.236524 24088 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.243224 24282 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:15.253499 24026 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:15.258203 24111 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:15.261291 21072 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:15.323249 21072 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:15.267243 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.268496 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.159404 24274 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:15.297411 21146 rpcz_store.cc:276] 0709 13:07:12.955447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.997292 (+2041845us) service_pool.cc:224] Handling call
0709 13:07:14.997610 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.051450 (+ 53840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.204395 (+152945us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b5400279a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:15.204648 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68069760}
I20260709 13:07:15.314440 21216 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:15.319304 21139 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:07:15.283850 24207 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.025s	sys 0.046s
W20260709 13:07:15.333485 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 242953us
I20260709 13:07:15.334664 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.335870 21146 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:07:15.336354 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.309780 24208 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.337924 21799 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:15.341902 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.338541 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:15.343107 21793 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:15.340370 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.344101 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.339455 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 335) took 2295 ms. Trace:
I20260709 13:07:15.344880 21146 rpcz_store.cc:276] 0709 13:07:13.044240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.335261 (+2291021us) service_pool.cc:224] Handling call
0709 13:07:15.335617 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.339121 (+  3504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.339395 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:15.120613 24331 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.315820 24333 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.123662 24332 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:15.350471 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:15.355536 21146 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:15.321187 24026 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:15.350776 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.350665 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.356130 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:15.365288 21066 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:15.304855 24201 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:15.179709 24226 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:15.337656 24328 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.276s	user 0.000s	sys 0.000s
I20260709 13:07:15.344780 21217 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:15.377542 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.186690 24304 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.363790 24282 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.368808 21796 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:15.356501 21184 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:15.340727 24048 consensus_queue.cc:1243] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:07:15.379575 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.382550 21219 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:15.309921 23952 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.385643 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:15.341391 24130 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:15.387635 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.404753 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.406450 21219 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:15.357851 24129 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:15.409371 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:15.375349 24256 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.345906 24088 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:07:15.411751 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:15.414633 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:15.418372 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.418808 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.427934 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.445134 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.413427 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:15.179924 24340 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.349093 24208 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.452909 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.445134 24048 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:07:15.461529 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.468909 21219 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:15.400766 24141 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:15.467964 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.459419 21066 raft_consensus.cc:2955] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:07:15.474304 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.477147 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.478106 21218 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:15.406672 24193 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:15.454054 24286 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.130s	user 0.007s	sys 0.005s
I20260709 13:07:15.449673 24313 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:15.450196 24027 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:15.456197 24089 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.493918 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:15.489934 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 247735us
I20260709 13:07:15.493691 21221 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:15.467839 24198 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:15.501746 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.482563 24141 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.445333 24130 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:15.511318 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2501) took 1019 ms. Trace:
I20260709 13:07:15.527405 21221 rpcz_store.cc:276] 0709 13:07:14.492256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.350413 (+858157us) service_pool.cc:224] Handling call
0709 13:07:15.511231 (+160818us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:15.497721 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:15.529037 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.511569 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:15.514482 24331 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.457762 24256 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:15.367687 24333 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:15.465314 24303 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:15.402287 24282 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:15.518026 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 151612us
I20260709 13:07:15.411811 23943 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:07:15.486980 24315 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:15.537868 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.549280 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.502833 24027 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:15.489789 24107 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:15.552573 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 54982.9us
I20260709 13:07:15.533432 24339 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:15.556089 21218 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:15.566107 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:15.531491 24141 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:15.560199 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:15.525058 24254 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.529986 24109 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.551728 21138 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:15.487926 24286 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.574424 24286 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.575167 21147 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:15.578778 21218 raft_consensus.cc:1217] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 4.5->[6.6-6.6]   Dedup: 6.6->[]
I20260709 13:07:15.579568 21218 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:07:15.580318 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2495) took 1117 ms. Trace:
I20260709 13:07:15.580731 21218 rpcz_store.cc:276] 0709 13:07:14.462865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.578209 (+1115344us) service_pool.cc:224] Handling call
0709 13:07:15.578652 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.579985 (+  1333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.580244 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:15.584167 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.585048 21065 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:15.587865 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.551527 24198 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:15.480170 24026 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:15.592344 21218 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:15.568923 24027 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:15.568724 24157 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.593915 21219 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:15.609887 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2508) took 1096 ms. Trace:
I20260709 13:07:15.610427 21219 rpcz_store.cc:276] 0709 13:07:14.513085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.581928 (+1068843us) service_pool.cc:224] Handling call
0709 13:07:15.593680 (+ 11752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.609535 (+ 15855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.609803 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:15.594125 21065 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:15.554510 24284 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:15.592362 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2500) took 1105 ms. Trace:
W20260709 13:07:15.595920 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 139650us
I20260709 13:07:15.569825 24088 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:15.598804 24340 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.588450 24255 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:15.590896 24254 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:15.612699 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2510) took 1099 ms. Trace:
I20260709 13:07:15.613754 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 969) took 1219 ms. Trace:
I20260709 13:07:15.617986 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:15.618702 21221 rpcz_store.cc:276] 0709 13:07:14.487024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.557113 (+1070089us) service_pool.cc:224] Handling call
0709 13:07:15.592237 (+ 35124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:15.621260 21212 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:15.596517 24112 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:07:15.635905 21218 rpcz_store.cc:276] 0709 13:07:14.513745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.584811 (+1071066us) service_pool.cc:224] Handling call
0709 13:07:15.592102 (+  7291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.604181 (+ 12079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.612612 (+  8431us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:15.636523 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2509) took 1123 ms. Trace:
I20260709 13:07:15.639451 21212 rpcz_store.cc:276] 0709 13:07:14.513409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.109356 (+595947us) service_pool.cc:224] Handling call
0709 13:07:15.636444 (+527088us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:15.635488 21213 rpcz_store.cc:276] 0709 13:07:14.394419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.394581 (+   162us) service_pool.cc:224] Handling call
0709 13:07:14.445580 (+ 50999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.445784 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:14.481705 (+ 35921us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.252765 (+771060us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b1800389f80. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:07:15.613309 (+360544us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:15.613388 (+    79us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:15.613450 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.613542 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.613691 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35880,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":9496,"raft.queue_time_us":17269,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":19528,"spinlock_wait_cycles":7276416,"thread_start_us":360210,"threads_started":2,"wal-append.queue_time_us":1133103}
I20260709 13:07:15.601706 24330 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.511405 24313 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.667762 24208 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:15.676605 24198 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:15.678542 24350 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:07:15.674799 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.684088 21221 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:15.684036 21212 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:07:15.687264 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46388.7us
I20260709 13:07:15.694223 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.635684 24315 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.670369 24336 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:15.486503 24201 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:15.709478 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.710711 21065 raft_consensus.cc:2410] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:07:15.700634 21064 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:07:15.716120 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:15.717821 21064 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:07:15.721666 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.725845 24333 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:15.682674 23952 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:15.684613 24254 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:15.732950 24201 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:15.691026 24088 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:15.689918 24304 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.748314 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 328) took 3722 ms. Trace:
I20260709 13:07:15.754586 21214 rpcz_store.cc:276] 0709 13:07:12.007732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.007875 (+   143us) service_pool.cc:224] Handling call
0709 13:07:12.008146 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.433004 (+424858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.727525 (+3294521us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b54000bc7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:15.729751 (+  2226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975878400}
I20260709 13:07:15.767644 21219 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:15.780009 24088 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.782625 24254 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:15.732950 24226 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:15.814131 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 236608us
I20260709 13:07:15.812374 24048 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:15.818849 24048 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:15.819489 24355 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:07:15.820940 24301 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.915s	user 0.000s	sys 0.014s
I20260709 13:07:15.822571 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 405) took 2692 ms. Trace:
I20260709 13:07:15.823092 21067 rpcz_store.cc:276] 0709 13:07:13.130255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.130511 (+   256us) service_pool.cc:224] Handling call
0709 13:07:13.167810 (+ 37299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.168162 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.168191 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:13.176862 (+  8671us) log.cc:844] Serialized 26 byte log entry
0709 13:07:13.780326 (+603464us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:13.780368 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.780434 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.746335 (+1965901us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b240013a5e8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f2b59c4 00007f7f3f2b3acb 00007f7f3f46edc9 00007f7f3f46ea55 00007f7f3f46e09d 00007f7f420a010e 00007f7f420b788c 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221
0709 13:07:15.822207 (+ 75872us) raft_consensus.cc:1685] finished
0709 13:07:15.822236 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.822470 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6003,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":7601,"spinlock_wait_cycles":16571776,"thread_start_us":603083,"threads_started":1,"wal-append.queue_time_us":647496}
I20260709 13:07:15.746946 24224 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:15.702608 24315 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:15.739491 24284 consensus_queue.cc:1175] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9384358ce43f4e3fac2ae44bdf11c389" responder_term: 8 status { last_received { term: 8 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 8 index: 5 } }
I20260709 13:07:15.865294 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:15.856047 24282 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.834278 24304 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.835528 21214 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:07:15.904703 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.883039 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54122.9us
I20260709 13:07:15.889590 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:15.883649 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:15.914228 21218 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:15.835316 24224 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:15.928239 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.930861 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.878378 24340 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:15.934064 24040 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:15.935168 24139 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
W20260709 13:07:15.946637 24344 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:07:15.935217 24189 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:15.948024 24189 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:15.936985 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2085) took 2973 ms. Trace:
I20260709 13:07:15.949211 21139 rpcz_store.cc:276] 0709 13:07:12.963189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.052800 (+2089611us) service_pool.cc:224] Handling call
0709 13:07:15.053223 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.334751 (+281528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.936745 (+601994us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54002b3ed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:15.936928 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611051776}
W20260709 13:07:15.950485 24358 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:07:15.950421 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:15.951022 21139 raft_consensus.cc:2376] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7. Re-sending same reply.
I20260709 13:07:15.951906 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:15.938265 24194 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:15.939472 24351 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
I20260709 13:07:14.042167 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:15.943761 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.967993 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.933228 21214 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 3.
W20260709 13:07:15.945075 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61512us
I20260709 13:07:15.978969 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:15.879914 24332 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:15.878885 24048 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:15.941707 24353 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:07:15.946357 24192 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:16.019135 24192 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:16.019487 24366 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:15.938709 24141 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:15.959424 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:15.983721 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.981002 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:16.024461 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 43570.5us
I20260709 13:07:16.020166 24192 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:16.025418 23695 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:16.025611 24354 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:16.025882 23695 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.026222 24192 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:15.945541 24040 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:16.023885 21139 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:16.031584 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.027731 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 336) took 2965 ms. Trace:
I20260709 13:07:16.032413 21146 rpcz_store.cc:276] 0709 13:07:13.061892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.472754 (+2410862us) service_pool.cc:224] Handling call
0709 13:07:15.473090 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.473421 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:15.473449 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:15.493163 (+ 19714us) log.cc:844] Serialized 26 byte log entry
0709 13:07:15.690372 (+197209us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:15.690445 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.690571 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:16.027512 (+336941us) raft_consensus.cc:1685] finished
0709 13:07:16.027532 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.027676 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36690,"thread_start_us":138725,"threads_started":1,"wal-append.queue_time_us":501735}
I20260709 13:07:16.034219 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.035118 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.946384 24282 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:16.036558 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:16.037962 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:16.020049 24366 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.026255 23695 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:16.039433 21139 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:16.015936 24340 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:15.931358 24304 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:16.051640 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:15.953294 24281 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:16.050806 21142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2678) took 2746 ms (client timeout 2211 ms). Trace:
W20260709 13:07:16.057102 21142 rpcz_store.cc:269] 0709 13:07:13.297396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.512503 (+215107us) service_pool.cc:224] Handling call
0709 13:07:16.043951 (+2531448us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b540036d654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:16.044063 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":511821568}
I20260709 13:07:16.023667 24303 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:16.048753 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.039222 24282 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.037132 21146 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
W20260709 13:07:16.033649 24143 consensus_meta.cc:287] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.486s	user 0.006s	sys 0.007s
I20260709 13:07:16.063446 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:16.064365 21146 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:16.036755 24313 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:16.060402 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 360690us
I20260709 13:07:16.065342 21143 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:16.067116 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.069008 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.069595 21146 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:16.070551 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.070731 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.041919 24048 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:16.068313 24143 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.085222 24310 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:16.037812 24332 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:07:16.085772 24259 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:16.039112 24286 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:07:16.088562 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:16.072609 21142 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 4.
W20260709 13:07:16.091367 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58158.3us
I20260709 13:07:16.116667 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:16.072117 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.065554 24282 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:16.026229 24354 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.128343 21063 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:16.144449 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:16.122459 24340 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.066416 24303 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:16.040033 24224 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:16.098683 24048 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.105698 21139 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:16.029541 24189 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.118562 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.117341 21221 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:16.122076 24226 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:16.123611 24286 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.483994 21798 rpcz_store.cc:276] 0709 13:07:12.716484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.716643 (+   159us) service_pool.cc:224] Handling call
0709 13:07:12.716923 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.717119 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:12.955429 (+238310us) log.cc:844] Serialized 14 byte log entry
0709 13:07:14.341415 (+1385986us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b180040f360. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287
0709 13:07:14.385011 (+ 43596us) log.cc:844] Serialized 14 byte log entry
0709 13:07:14.469199 (+ 84188us) log.cc:844] Serialized 14 byte log entry
0709 13:07:14.472370 (+  3171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:14.472418 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:14.472467 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.472535 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.483486 (+ 10951us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27274,"prepare.queue_time_us":420526,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":1942,"raft.queue_time_us":65157,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":33072,"spinlock_wait_cycles":9921920,"thread_start_us":516242,"threads_started":5,"wal-append.queue_time_us":221493,"wal-append.run_cpu_time_us":1153,"wal-append.run_wall_time_us":1050650}
I20260709 13:07:16.396222 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:16.342159 21145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 346) took 2107 ms (client timeout 2082 ms). Trace:
I20260709 13:07:16.385249 21796 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:16.284238 21797 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:16.399600 21145 rpcz_store.cc:269] 0709 13:07:14.212155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.316525 (+104370us) service_pool.cc:224] Handling call
0709 13:07:16.319755 (+2003230us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b54002bbe54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:16.319917 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59497216}
W20260709 13:07:16.232579 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.167392 21212 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
W20260709 13:07:16.366616 21765 leader_election.cc:336] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.076s (SENT)
I20260709 13:07:16.401997 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.366516 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:16.218250 21143 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:16.403088 21145 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:07:16.403633 21143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2687) took 3032 ms (client timeout 1692 ms). Trace:
I20260709 13:07:16.403981 21187 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:16.375883 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.147645 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:16.404152 21143 rpcz_store.cc:269] 0709 13:07:13.371394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.482569 (+111175us) service_pool.cc:224] Handling call
0709 13:07:14.148821 (+666252us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b54002457a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42491a1d 00007f7f424781a4 00007f7f424fa447 00007f7f42512fe7 00007f7f425147de 00007f7f4252027c 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591
0709 13:07:16.045390 (+1896569us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b5800770fd0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f424780aa 00007f7f424fa447 00007f7f42512fe7 00007f7f425147de 00007f7f4252027c 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c
0709 13:07:16.403516 (+358126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810657024}
I20260709 13:07:16.405370 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:16.406476 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:16.402715 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:16.407192 21793 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:07:16.411499 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:16.412387 21139 raft_consensus.cc:2376] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9. Re-sending same reply.
I20260709 13:07:16.414249 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:16.415817 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:16.416920 21142 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:16.417881 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2688) took 3046 ms. Trace:
I20260709 13:07:16.418324 21142 rpcz_store.cc:276] 0709 13:07:13.371677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.416266 (+3044589us) service_pool.cc:224] Handling call
0709 13:07:16.416680 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.417556 (+   876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.417812 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:16.262542 24365 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 13:07:16.324995 24364 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:07:16.425650 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 338) took 2601 ms. Trace:
I20260709 13:07:16.430979 21146 rpcz_store.cc:276] 0709 13:07:13.823855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.424772 (+2600917us) service_pool.cc:224] Handling call
0709 13:07:16.425175 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.425355 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.425379 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.425419 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:16.425450 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.425501 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.425596 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.433252 21142 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:16.433892 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2697) took 3059 ms. Trace:
W20260709 13:07:16.434538 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67951.3us
I20260709 13:07:16.434577 21142 rpcz_store.cc:276] 0709 13:07:13.382144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.419515 (+3037371us) service_pool.cc:224] Handling call
0709 13:07:16.433071 (+ 13556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.433667 (+   596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.433839 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.436190 21142 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:16.436854 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 341) took 2611 ms. Trace:
I20260709 13:07:16.437276 21142 rpcz_store.cc:276] 0709 13:07:13.825049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.435657 (+2610608us) service_pool.cc:224] Handling call
0709 13:07:16.435905 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.436581 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.436799 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.149925 24259 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:16.406046 24129 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.377229 24354 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:16.173626 24141 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:16.325667 24366 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:16.326121 24304 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:16.433826 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 339) took 2609 ms. Trace:
I20260709 13:07:16.413887 24223 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:07:16.447726 24027 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.818s	user 0.008s	sys 0.005s
I20260709 13:07:16.450480 24129 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:16.420195 24315 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:16.415733 24143 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:16.465246 24343 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.873s	user 0.000s	sys 0.001s
I20260709 13:07:16.470820 24026 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:16.413781 24286 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.472267 24261 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:16.422911 24361 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:16.406423 24313 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:16.414143 24198 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:16.448861 21146 rpcz_store.cc:276] 0709 13:07:13.824526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.432946 (+2608420us) service_pool.cc:224] Handling call
0709 13:07:16.433259 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.433416 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.433442 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.433479 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:16.433513 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.433666 (+   153us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.433774 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.430079 21139 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:07:16.581324 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2690) took 3209 ms. Trace:
I20260709 13:07:16.581847 21139 rpcz_store.cc:276] 0709 13:07:13.372232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.425300 (+3053068us) service_pool.cc:224] Handling call
0709 13:07:16.426489 (+  1189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.580936 (+154447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.581254 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.476483 21143 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:16.427223 24332 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:16.514528 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.585296 21142 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:16.448591 24259 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:16.530373 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 106120us
W20260709 13:07:16.577167 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.576915 21145 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:16.523871 24189 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:16.526749 24345 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.927s	user 0.002s	sys 0.002s
I20260709 13:07:16.428210 24367 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.163s
I20260709 13:07:16.448956 24229 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:07:16.584832 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.432520 24226 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:16.583230 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2685) took 3212 ms. Trace:
I20260709 13:07:16.598127 21143 rpcz_store.cc:276] 0709 13:07:13.370691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.421772 (+3051081us) service_pool.cc:224] Handling call
0709 13:07:16.426442 (+  4670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.582903 (+156461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.583164 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.590077 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:16.599189 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:16.591662 23695 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.002s
I20260709 13:07:16.524147 24027 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.619513 23695 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.621089 24027 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:16.621587 23695 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:16.576020 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:16.599912 24226 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:16.622617 24226 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:16.624567 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:16.625209 21801 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:16.626236 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:16.627171 21217 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:07:16.631755 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:16.631752 21217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2515) took 1689 ms (client timeout 2171 ms). Trace:
W20260709 13:07:16.633347 21217 rpcz_store.cc:269] 0709 13:07:14.942414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.474047 (+531633us) service_pool.cc:224] Handling call
0709 13:07:16.631569 (+1157522us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540013cc54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:16.631694 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2628209408}
I20260709 13:07:16.593094 21139 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
W20260709 13:07:16.599987 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.590366 21146 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:16.635740 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2095) took 2727 ms. Trace:
I20260709 13:07:16.601756 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:16.636072 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2096) took 2727 ms. Trace:
I20260709 13:07:16.602519 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:16.636444 21139 rpcz_store.cc:276] 0709 13:07:13.909064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.592482 (+2683418us) service_pool.cc:224] Handling call
0709 13:07:16.592864 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.635858 (+ 42994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.636025 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.636889 21221 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
W20260709 13:07:16.606019 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.585216 24198 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:16.594063 24313 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.608516 24303 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:16.609711 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 972) took 2174 ms. Trace:
I20260709 13:07:16.613396 24048 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:16.613567 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2700) took 1274 ms. Trace:
I20260709 13:07:16.614724 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 344) took 2503 ms. Trace:
I20260709 13:07:16.611512 24130 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:16.617175 24197 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:16.594686 24332 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:16.634184 24286 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:16.637655 21139 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:16.661937 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2098) took 2752 ms. Trace:
I20260709 13:07:16.662283 21139 rpcz_store.cc:276] 0709 13:07:13.909594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.637213 (+2727619us) service_pool.cc:224] Handling call
0709 13:07:16.637496 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.661669 (+ 24173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.661880 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.664006 21139 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:16.664690 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 342) took 2561 ms. Trace:
I20260709 13:07:16.665071 21139 rpcz_store.cc:276] 0709 13:07:14.103668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.663164 (+2559496us) service_pool.cc:224] Handling call
0709 13:07:16.663585 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.664409 (+   824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.664626 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:16.592070 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 465820us
W20260709 13:07:16.577688 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 161979us
I20260709 13:07:16.636092 21146 rpcz_store.cc:276] 0709 13:07:13.908810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.589946 (+2681136us) service_pool.cc:224] Handling call
0709 13:07:16.590200 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.635493 (+ 45293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.635689 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.638652 24198 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:16.641047 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 978) took 1614 ms. Trace:
I20260709 13:07:16.673478 21218 rpcz_store.cc:276] 0709 13:07:15.026947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.642143 (+615196us) service_pool.cc:224] Handling call
0709 13:07:15.674415 (+ 32272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.914651 (+240236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.640728 (+726077us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54004434d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:16.640966 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550675328}
I20260709 13:07:16.642460 21215 rpcz_store.cc:276] 0709 13:07:14.435216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.444202 (+  8986us) service_pool.cc:224] Handling call
0709 13:07:14.468375 (+ 24173us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:15.122336 (+653961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:15.122371 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:15.132126 (+  9755us) log.cc:844] Serialized 52 byte log entry
0709 13:07:15.402186 (+270060us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:15.402221 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.402287 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:16.464643 (+1062356us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:16.609422 (+144779us) raft_consensus.cc:1685] finished
0709 13:07:16.609442 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.609652 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27947,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":3457920,"thread_start_us":916284,"threads_started":2,"wal-append.queue_time_us":392909}
I20260709 13:07:16.644428 21138 rpcz_store.cc:276] 0709 13:07:15.339236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.385254 (+ 46018us) service_pool.cc:224] Handling call
0709 13:07:16.613404 (+1228150us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54002968d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:16.613508 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232741632}
I20260709 13:07:16.647385 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:16.649279 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:16.675666 21801 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:16.677211 24226 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.649931 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:16.679656 24226 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:16.639433 24282 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.515s	user 0.002s	sys 0.009s
I20260709 13:07:16.680505 24282 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.680325 24226 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:16.681452 24226 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.681962 24226 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:16.682714 24282 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:16.683521 24332 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:16.683948 21138 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:07:16.660147 21035 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:16.644569 21147 rpcz_store.cc:276] 0709 13:07:14.110981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.323120 (+212139us) service_pool.cc:224] Handling call
0709 13:07:16.614520 (+2291400us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400258ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:16.614666 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2876446464}
I20260709 13:07:16.672658 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:16.700809 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.701561 21066 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:07:16.747032 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.742089 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:16.742542 21147 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:16.742545 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2698) took 1403 ms. Trace:
I20260709 13:07:16.757561 21138 rpcz_store.cc:276] 0709 13:07:15.338701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.683365 (+1344664us) service_pool.cc:224] Handling call
0709 13:07:16.683725 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.684376 (+   651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.742469 (+ 58093us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.705583 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.744385 21801 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:16.759850 21066 raft_consensus.cc:2410] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:07:16.761941 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:16.742730 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 67537us
W20260709 13:07:16.762729 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 76074us
I20260709 13:07:16.769317 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.667812 24313 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:16.687028 24048 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:07:16.774891 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2703) took 1431 ms. Trace:
I20260709 13:07:16.776278 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 421) took 2574 ms. Trace:
I20260709 13:07:16.778303 21147 rpcz_store.cc:276] 0709 13:07:15.343935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.684740 (+1340805us) service_pool.cc:224] Handling call
0709 13:07:16.742360 (+ 57620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.774608 (+ 32248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.774835 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.699396 24302 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:16.773698 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.776525 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.777660 24192 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:16.776645 24247 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:07:16.781459 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.767642 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:16.781774 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.781652 24192 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:16.782126 21069 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:16.782407 21071 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:16.782830 21138 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:16.783113 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.783449 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2103) took 1278 ms. Trace:
I20260709 13:07:16.784096 21069 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:07:16.783841 21138 rpcz_store.cc:276] 0709 13:07:15.504910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.782249 (+1277339us) service_pool.cc:224] Handling call
0709 13:07:16.782543 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.783204 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.783395 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.785140 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.786265 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.783952 24192 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:16.786862 21067 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:16.787485 21069 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:16.697912 24388 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:16.783187 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.801828 21071 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:16.778427 21068 rpcz_store.cc:276] 0709 13:07:14.201909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.202061 (+   152us) service_pool.cc:224] Handling call
0709 13:07:14.212617 (+ 10556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:14.231622 (+ 19005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.231662 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:14.232143 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:07:16.473806 (+2241663us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b18003e7460. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:07:16.690878 (+217072us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:16.690919 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.691001 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:16.751748 (+ 60747us) raft_consensus.cc:1685] finished
0709 13:07:16.751779 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.776188 (+ 24409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8404,"raft.queue_time_us":13449,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":90,"spinlock_wait_cycles":14910208,"thread_start_us":216716,"threads_started":1,"wal-append.queue_time_us":2512145}
I20260709 13:07:16.702977 24189 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:16.802336 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:16.698163 24282 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:07:16.811080 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:16.824317 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.825718 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.826349 21067 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:16.826879 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2513) took 2312 ms. Trace:
I20260709 13:07:16.775995 24286 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.776932 24303 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:16.800071 24048 consensus_queue.cc:1243] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:16.800459 24388 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:16.809103 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:16.813220 24247 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:16.875906 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:16.883936 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:16.884157 21217 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:07:16.884685 21220 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:16.789908 24246 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:07:16.827260 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:16.890777 21067 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:16.893234 21217 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:16.888813 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:16.836951 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:16.845075 21792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 334) took 2934 ms (client timeout 2734 ms). Trace:
W20260709 13:07:16.845423 21794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 682) took 3445 ms (client timeout 1668 ms). Trace:
I20260709 13:07:16.849946 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2090) took 2947 ms. Trace:
I20260709 13:07:16.799662 24379 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.022s
W20260709 13:07:16.896294 21794 rpcz_store.cc:269] 0709 13:07:13.399916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.400153 (+   237us) service_pool.cc:224] Handling call
0709 13:07:16.845248 (+3445095us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b5400380254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:16.845360 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1735238528}
W20260709 13:07:16.895226 21792 rpcz_store.cc:269] 0709 13:07:13.910365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.910494 (+   129us) service_pool.cc:224] Handling call
0709 13:07:16.844746 (+2934252us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b54003bb854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:16.844940 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124753664}
W20260709 13:07:16.845846 21141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 348) took 2299 ms (client timeout 1571 ms). Trace:
W20260709 13:07:16.897557 21141 rpcz_store.cc:269] 0709 13:07:14.546590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.914484 (+367894us) service_pool.cc:224] Handling call
0709 13:07:16.845596 (+1931112us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54004240d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:16.845765 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594811648}
I20260709 13:07:16.806046 24189 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.899194 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2109) took 1373 ms. Trace:
I20260709 13:07:16.899612 21141 rpcz_store.cc:276] 0709 13:07:15.526159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.898421 (+1372262us) service_pool.cc:224] Handling call
0709 13:07:16.898719 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.898896 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.898918 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.898954 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:16.898988 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.899036 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.899130 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:16.846333 21216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2498) took 2376 ms (client timeout 1723 ms). Trace:
W20260709 13:07:16.900645 21216 rpcz_store.cc:269] 0709 13:07:14.470411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.493885 (+ 23474us) service_pool.cc:224] Handling call
0709 13:07:16.846149 (+2352264us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400442ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:16.846253 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589017344}
I20260709 13:07:16.865077 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2689) took 3493 ms. Trace:
I20260709 13:07:16.901685 21145 rpcz_store.cc:276] 0709 13:07:13.371939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.413610 (+3041671us) service_pool.cc:224] Handling call
0709 13:07:16.435541 (+ 21931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.588325 (+152784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.864767 (+276442us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540030aa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:16.864997 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322727168}
I20260709 13:07:16.869217 21184 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:16.870478 21801 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:16.895426 21142 rpcz_store.cc:276] 0709 13:07:13.902796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.438055 (+2535259us) service_pool.cc:224] Handling call
0709 13:07:16.438316 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.585738 (+147422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.849637 (+263899us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54002b48d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:16.849874 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337507584}
I20260709 13:07:16.904983 21142 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:07:16.827328 21219 rpcz_store.cc:276] 0709 13:07:14.514776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.611543 (+1096767us) service_pool.cc:224] Handling call
0709 13:07:15.642624 (+ 31081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.770276 (+127652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.826628 (+1056352us) spinlock_profiling.cc:243] Waited 71.6 ms on lock 0x7b54002ebb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:16.826813 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164697984}
I20260709 13:07:16.895326 21068 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:16.923666 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:16.926576 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 85580.9us
W20260709 13:07:16.927870 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.928238 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:16.930348 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 121313us
I20260709 13:07:16.930305 24332 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:16.931563 24332 consensus_queue.cc:1243] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:07:16.931829 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:16.932263 24332 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:16.932787 24332 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.933176 24332 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:16.935704 24323 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:16.936344 24387 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:16.936820 24387 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:16.937211 24387 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:16.936348 24323 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.950328 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:16.840945 24109 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:16.937208 24197 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.835988 24318 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:16.952602 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:16.883692 24247 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:16.939935 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:16.953732 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:16.974027 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.958600 21067 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:16.837358 24388 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:16.939563 24089 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:16.967748 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:16.970146 21217 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:16.956147 24323 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:16.925704 24189 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:16.955051 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:16.835338 24286 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:16.836158 24303 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.001536 21068 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:16.829326 24226 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:16.932927 24376 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:07:17.028120 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.028926 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.030529 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.034461 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.037451 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:16.853561 24259 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.004570 21217 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
W20260709 13:07:16.991918 24386 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:07:16.969844 24318 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:17.042692 24318 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.015451 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.044458 21219 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:17.044973 21214 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:17.045384 21214 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:17.045537 24226 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.046334 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:17.046168 21214 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.005719 24089 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:16.989427 24197 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:16.952710 24143 consensus_queue.cc:765] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Error trying to read ahead of the log while preparing peer request: Incomplete: Op with index 7 is ahead of the local log (next sequential op: 7). Destination peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 8, Last known committed idx: 7, Time since last communication: 0.344s
I20260709 13:07:16.841248 24198 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:17.039459 21212 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:17.035065 21220 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:16.991453 24192 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:16.992301 24247 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.049427 21217 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:07:17.062559 21214 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:17.101289 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.114750 21143 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:17.062714 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:17.102501 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54287us
I20260709 13:07:17.068928 21064 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:07:17.012480 24293 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:17.142194 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.155918 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:17.156487 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.157068 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.159725 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.040831 24129 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:07:17.164300 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.062839 24194 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:07:17.163472 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 275682us
I20260709 13:07:17.071336 24317 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.172049 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.172796 21212 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:17.074159 24275 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:17.191506 24317 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.193856 24317 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.194478 24089 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.196669 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2097) took 3287 ms. Trace:
I20260709 13:07:17.196611 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:17.197146 21143 rpcz_store.cc:276] 0709 13:07:13.909344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.598982 (+2689638us) service_pool.cc:224] Handling call
0709 13:07:16.601617 (+  2635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.115275 (+513658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.196456 (+ 81181us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b540030b954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:17.196616 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179606912}
I20260709 13:07:17.194041 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:17.202184 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.203673 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.204429 21212 raft_consensus.cc:2368] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:17.176672 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:17.178871 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5280908390001287s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2782s for a response
I20260709 13:07:17.206936 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:17.179989 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:17.208550 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:17.120774 24256 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:17.047472 24296 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.117s
I20260709 13:07:17.164649 24293 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.185598 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.234493 24313 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:17.234704 21217 raft_consensus.cc:2376] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7. Re-sending same reply.
I20260709 13:07:17.235045 24313 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:17.236147 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.236479 24293 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.237519 24313 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:17.237988 24259 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:17.238060 24313 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.239581 24303 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.238600 24313 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.224318 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.244609 24198 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:17.245054 21221 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
W20260709 13:07:17.183439 24386 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:07:17.222853 21220 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:17.230182 24296 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:17.231989 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.233332 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:17.252698 21067 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:17.233815 24143 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:17.233863 24317 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:17.118484 24192 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.237488 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.262966 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:17.263648 21217 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:17.219811 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:17.264467 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44794.8us
I20260709 13:07:17.256776 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.265538 21212 raft_consensus.cc:2376] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8. Re-sending same reply.
I20260709 13:07:17.264461 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.242053 24197 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:17.265561 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:17.264541 21214 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:17.217203 24194 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:17.272693 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:17.217558 24294 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:17.274331 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.283864 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.284842 21217 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:17.250160 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.288249 21220 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:17.286685 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.283025 21067 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:17.290845 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 684) took 1978 ms. Trace:
I20260709 13:07:17.291234 21797 rpcz_store.cc:276] 0709 13:07:15.312130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.312295 (+   165us) service_pool.cc:224] Handling call
0709 13:07:15.338229 (+ 25934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.413975 (+1075746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.290613 (+876638us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54005e01d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:17.290787 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175056768}
I20260709 13:07:17.289906 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:17.292563 21065 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:17.292774 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:17.293629 21070 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:17.294627 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:17.298043 21070 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:17.304103 24247 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.306305 24247 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.251132 24296 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:17.287703 24294 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:17.313584 24211 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:17.314061 24294 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.316655 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.317518 21212 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:17.319653 21065 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:17.271574 24198 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:17.321146 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:17.322854 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2104) took 1817 ms. Trace:
I20260709 13:07:17.324180 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:17.315491 24294 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:17.323807 21138 rpcz_store.cc:276] 0709 13:07:15.505299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.785307 (+1280008us) service_pool.cc:224] Handling call
0709 13:07:16.785615 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.785810 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:17.005347 (+219537us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.321920 (+316573us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.322133 (+   213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:17.322469 (+   336us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:17.322532 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.322677 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:17.322798 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7944,"prepare.queue_time_us":37703,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":5465,"thread_start_us":526457,"threads_started":2}
I20260709 13:07:17.302032 24284 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:17.308717 24197 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:17.268157 24143 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.273161 24317 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.343127 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.345544 21221 raft_consensus.cc:2376] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4. Re-sending same reply.
I20260709 13:07:17.346801 21138 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:17.329784 24333 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:17.351155 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 231770us
I20260709 13:07:17.336437 24284 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:17.335912 24197 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.329346 24388 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.358325 21064 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:17.370998 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.372674 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.373965 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.374919 21800 raft_consensus.cc:2410] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:07:17.376206 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:17.352437 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:17.313954 24296 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:17.354193 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.369837 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.380940 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.381255 21794 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:17.382252 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:17.383803 24395 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:07:17.366145 24284 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:17.378038 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.344655 24332 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.352074 24309 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.386080 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 347) took 2839 ms. Trace:
I20260709 13:07:17.377274 21067 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:17.396294 21217 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:17.390457 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.367575 24197 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:17.376911 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:17.392616 21065 raft_consensus.cc:2410] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:07:17.393401 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.367168 24313 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:17.394284 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.406232 21798 raft_consensus.cc:2410] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:17.389298 21070 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:17.408001 21070 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:17.401602 21795 raft_consensus.cc:2410] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:07:17.395675 21146 rpcz_store.cc:276] 0709 13:07:14.546303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.673449 (+2127146us) service_pool.cc:224] Handling call
0709 13:07:16.673924 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:16.674442 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.674485 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:16.701218 (+ 26733us) log.cc:844] Serialized 52 byte log entry
0709 13:07:16.956026 (+254808us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:16.988114 (+ 32088us) log.cc:844] Serialized 14 byte log entry
0709 13:07:16.988309 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.988387 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:17.385767 (+397380us) raft_consensus.cc:1685] finished
0709 13:07:17.385794 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:17.386000 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8877,"prepare.queue_time_us":211637,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":4235,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":23973,"thread_start_us":208634,"threads_started":1,"wal-append.queue_time_us":514343}
I20260709 13:07:17.396615 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.409587 21213 raft_consensus.cc:2376] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8. Re-sending same reply.
W20260709 13:07:17.412199 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.415483 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.416085 21145 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:17.416195 21217 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:17.420071 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:17.433893 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:17.435739 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:17.372715 24323 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.442503 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:17.405172 24296 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:17.353068 24194 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.450290 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:17.457824 21143 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:17.458647 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.459823 21213 raft_consensus.cc:2368] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:17.438148 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:17.456826 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:17.440790 24275 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:17.437572 23987 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:17.464257 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:17.451133 24375 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:17.476835 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:17.455405 24338 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:17.452777 24309 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:07:17.463547 24192 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.472718 23980 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:17.481983 24129 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:17.482177 24367 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:17.482375 24259 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:07:17.497694 24311 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 3.355s	user 0.006s	sys 0.014s
I20260709 13:07:17.495065 23980 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:17.499850 23980 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:17.503320 24400 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:07:17.464871 24275 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:17.505024 24275 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:17.512030 24317 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:17.530799 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.544909 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:17.546060 21213 raft_consensus.cc:2376] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9. Re-sending same reply.
I20260709 13:07:17.489862 23987 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:07:17.549602 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1971440380001468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 13:07:17.550155 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1971440380001468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 13:07:17.550491 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1590908390001005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31920000000000004s for a response
W20260709 13:07:17.550791 21036 connection.cc:256] RPC call timeout handler was delayed by -1.075644038000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 13:07:17.551054 21036 connection.cc:256] RPC call timeout handler was delayed by -1.075644038000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 13:07:17.555105 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.558518 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:17.559893 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 257538us
I20260709 13:07:17.493386 24143 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.571718 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:17.572350 21146 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:17.575088 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:17.493324 24303 consensus_queue.cc:765] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Error trying to read ahead of the log while preparing peer request: Incomplete: Op with index 7 is ahead of the local log (next sequential op: 7). Destination peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 8, Last known committed idx: 7, Time since last communication: 0.907s
I20260709 13:07:17.549142 24192 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:17.580554 24192 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.582482 24192 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:17.583379 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.584133 21213 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:17.584983 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:17.586520 21213 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:17.544622 24388 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.549540 24317 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:17.557338 23987 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.567981 24332 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:17.598287 24303 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:07:17.614152 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.611016 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:17.608395 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:17.596908 24143 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:17.601188 23987 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:17.622627 24189 consensus_meta.cc:287] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.003s
I20260709 13:07:17.622771 24197 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.623162 24189 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.625156 24197 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:17.603715 24317 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.625034 24040 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.626205 24040 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:17.626581 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:17.627543 24317 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:17.627884 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:17.629359 21065 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:07:17.632176 21215 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:07:17.633154 24387 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.695s	user 0.010s	sys 0.005s
I20260709 13:07:17.633699 21071 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:17.633569 24387 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:17.635610 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2676) took 4350 ms (client timeout 1723 ms). Trace:
I20260709 13:07:17.636081 24387 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:17.635886 21072 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:17.636221 21140 rpcz_store.cc:269] 0709 13:07:13.285508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.301437 (+ 15929us) service_pool.cc:224] Handling call
0709 13:07:17.635321 (+4333884us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54004236d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:17.635493 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485001472}
I20260709 13:07:17.637147 21072 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:17.620959 24303 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:17.639840 24303 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:17.643429 24157 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:17.644116 24157 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:17.645383 24157 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.643432 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:17.650815 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:17.618093 21068 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
W20260709 13:07:17.656514 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 218481us
W20260709 13:07:17.658164 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.657488 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:17.660833 24197 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:17.661331 24197 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:17.661755 24197 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:17.664170 24281 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:17.664706 24281 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.665107 24281 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:17.672003 24333 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:17.672822 24333 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:17.675513 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1903) took 2893 ms (client timeout 3811 ms). Trace:
W20260709 13:07:17.676096 21063 rpcz_store.cc:269] 0709 13:07:14.781818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.782524 (+   706us) service_pool.cc:224] Handling call
0709 13:07:17.675037 (+2892513us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400214b54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:17.675374 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2546398080}
I20260709 13:07:17.679656 21063 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:07:17.680783 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2086) took 4713 ms. Trace:
I20260709 13:07:17.680435 24284 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:17.683915 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.687841 21144 rpcz_store.cc:276] 0709 13:07:12.966916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.208333 (+2241417us) service_pool.cc:224] Handling call
0709 13:07:16.846408 (+1638075us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.846857 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:16.847198 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.016921 (+169723us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:17.017473 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:07:17.017611 (+   138us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:17.017648 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.017735 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:17.046045 (+ 28310us) raft_consensus.cc:1685] finished
0709 13:07:17.046065 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:17.680582 (+634517us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54004240d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:17.680703 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31519,"prepare.queue_time_us":135,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":34326,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":215,"spinlock_wait_cycles":3766236416,"thread_start_us":137526,"threads_started":1,"wal-append.queue_time_us":194945}
I20260709 13:07:17.683970 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:17.691474 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:17.684432 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.687207 21187 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:17.655339 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46992.6us
W20260709 13:07:17.685532 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.690510 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.691732 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.688987 21069 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:17.648406 23943 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:17.692385 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.647120 24157 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:17.698663 21066 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:07:17.715395 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:17.692767 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:17.625121 24189 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:17.660058 24367 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:17.702162 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.740914 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:17.692101 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:17.736920 24414 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:07:17.706336 21072 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:07:17.758183 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.659879 24143 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.754220 24189 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:17.714130 24413 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:17.766232 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 399) took 5067 ms. Trace:
I20260709 13:07:17.766987 21072 rpcz_store.cc:276] 0709 13:07:12.691829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.691981 (+   152us) service_pool.cc:224] Handling call
0709 13:07:12.698249 (+  6268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.261158 (+2562909us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54003ec720. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42493dc0 00007f7f4251e1c0 00007f7f4251f231 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c
0709 13:07:17.635574 (+2374416us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b54003ec720. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4249a06f 00007f7f42512f60 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62
0709 13:07:17.758423 (+122849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.758645 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783977600}
I20260709 13:07:17.774278 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:17.789471 21142 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:17.790802 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:17.791893 21142 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 4: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:17.792840 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:17.793555 21142 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:17.698606 24089 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:07:17.745723 24367 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:17.796914 24367 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:17.776294 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:17.781208 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:17.803956 21140 raft_consensus.cc:2376] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7. Re-sending same reply.
I20260709 13:07:17.805365 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:17.805706 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:17.806318 21140 raft_consensus.cc:2368] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:17.783466 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:17.808686 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:17.752858 24415 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:17.810629 21069 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
W20260709 13:07:17.811733 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.812258 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:17.768958 23868 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:07:17.814486 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:17.788367 21147 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:17.816056 21140 raft_consensus.cc:2368] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:17.822554 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2107) took 2309 ms. Trace:
W20260709 13:07:17.830466 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.814554 24048 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.841218 24048 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.837935 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:17.787093 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:17.833209 21796 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:17.844161 21072 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:17.834506 24367 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.834878 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:17.788235 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 350) took 1838 ms. Trace:
I20260709 13:07:17.823160 21147 rpcz_store.cc:276] 0709 13:07:15.512972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.884083 (+1371111us) service_pool.cc:224] Handling call
0709 13:07:17.788177 (+904094us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.815442 (+ 27265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.822266 (+  6824us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b54001eb4d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:17.822466 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1924402944}
I20260709 13:07:17.852399 21141 rpcz_store.cc:276] 0709 13:07:15.949719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.938350 (+988631us) service_pool.cc:224] Handling call
0709 13:07:16.938762 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.939209 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:16.965151 (+ 25942us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.354838 (+389687us) spinlock_profiling.cc:243] Waited 8.14 ms on lock 0x7b18000063a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3fd2a6 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1
0709 13:07:17.356186 (+  1348us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b18000063a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:17.640091 (+283905us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b18000063a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:07:17.768939 (+128848us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.777781 (+  8842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:17.787836 (+ 10055us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:17.787903 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.787995 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:17.788154 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23786,"prepare.queue_time_us":26941,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":90,"raft.queue_time_us":10960,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":24949,"spinlock_wait_cycles":92145920,"thread_start_us":125067,"threads_started":1}
I20260709 13:07:17.813830 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:17.864233 21069 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:17.869518 21071 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:17.899029 21071 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:17.889946 24048 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:17.891659 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:17.893733 23868 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.869365 21799 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:17.895540 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:17.912417 21144 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:17.880016 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:07:17.913802 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.755321 23943 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:07:17.909616 21142 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:17.914819 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:17.897650 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:17.922235 21063 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:07:17.922650 21072 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:17.916687 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:17.917626 21071 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:17.935995 21071 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:17.928401 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:17.938633 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 379884us
I20260709 13:07:17.939870 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:17.942035 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:17.945230 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:17.784382 24419 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:17.780265 24185 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.138s
W20260709 13:07:17.971804 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.977892 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 284611us
I20260709 13:07:17.910034 24143 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:17.996410 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51268.3us
I20260709 13:07:17.997864 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:18.011359 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 64188.7us
I20260709 13:07:17.908039 24415 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:17.933303 24273 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:17.942158 24279 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:07:18.014830 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.939633 24391 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:07:18.016491 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1916) took 1167 ms. Trace:
I20260709 13:07:18.022162 21065 rpcz_store.cc:276] 0709 13:07:16.849533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.630137 (+780604us) service_pool.cc:224] Handling call
0709 13:07:17.630389 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.630563 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:17.630858 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.016110 (+385252us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.016170 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:18.016219 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.016295 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.016429 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62178,"prepare.queue_time_us":1002,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"thread_start_us":317938,"threads_started":1}
I20260709 13:07:17.984138 24313 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:17.945039 24312 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:17.944897 24317 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:18.012303 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:17.953454 24246 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:17.935060 23868 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.022956 24430 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:07:18.037284 21071 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:07:18.052421 21071 pending_rounds.cc:85] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 6
I20260709 13:07:18.018441 24273 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:18.072579 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.074147 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 382091us
I20260709 13:07:18.075162 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:18.037784 24279 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:18.020891 24048 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:18.052686 21071 pending_rounds.cc:107] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:07:18.093999 21071 raft_consensus.cc:2889] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:18.032485 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:18.049693 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:18.030210 24302 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
W20260709 13:07:17.396221 24121 consensus_queue.cc:1175] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" responder_term: 7 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: [6.7-6.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:17.989624 24303 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.054550 23943 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:18.054939 24284 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:18.059772 24331 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:18.061472 24197 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.026816 24226 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
W20260709 13:07:18.062506 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.118474 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 959) took 1354 ms. Trace:
W20260709 13:07:18.064137 24417 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
W20260709 13:07:18.119151 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.066484 24367 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:18.119648 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.076864 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 359082us
I20260709 13:07:18.013495 23980 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.085170 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:18.038154 24278 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:18.075223 24273 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:18.101444 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2099) took 3983 ms. Trace:
W20260709 13:07:18.130224 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 118009us
I20260709 13:07:18.130522 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:18.105119 24246 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:18.129145 21139 rpcz_store.cc:276] 0709 13:07:14.117821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.665991 (+2548170us) service_pool.cc:224] Handling call
0709 13:07:16.666366 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.669814 (+  3448us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:16.670188 (+   374us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.811642 (+1141454us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b18001d6aa0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:07:18.101102 (+289460us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.101154 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:18.101204 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.101270 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.101389 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":141,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":14951,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":3734,"spinlock_wait_cycles":42221312,"thread_start_us":281687,"threads_started":1,"wal-append.queue_time_us":1428122}
W20260709 13:07:18.105811 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.076382 24419 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:18.111920 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 62359.1us
I20260709 13:07:18.155364 21795 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:18.156797 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:18.112079 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79661.2us
I20260709 13:07:18.041217 24312 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.111495 24247 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:18.067747 24293 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.120450 21068 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:18.118902 21064 rpcz_store.cc:276] 0709 13:07:16.764343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.374006 (+609663us) service_pool.cc:224] Handling call
0709 13:07:17.374298 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.374614 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:07:17.756113 (+381499us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.118014 (+361901us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.118065 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:18.118117 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.118180 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.118416 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":77146,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":39,"raft.queue_time_us":78,"thread_start_us":724883,"threads_started":2}
W20260709 13:07:18.121572 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.115489 24109 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:18.124825 24197 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:18.126461 24303 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:18.126271 24229 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:18.115192 24130 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:18.035549 21063 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:18.130946 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:17.949625 24330 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:18.166222 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 960) took 1401 ms. Trace:
I20260709 13:07:18.131858 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:18.099324 23987 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.119735 24376 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:18.128531 23761 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:18.154099 24246 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:18.153134 24340 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:18.131436 24103 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:07:18.131260 24211 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:07:18.164014 24422 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:18.195531 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 64629.1us
I20260709 13:07:18.205281 21036 leader_election.cc:400] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:18.205590 21036 leader_election.cc:403] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:07:18.206197 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:18.207602 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:18.201905 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1929) took 1160 ms. Trace:
I20260709 13:07:18.201236 24303 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:18.170881 24109 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.119928 24415 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:18.182999 24331 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:07:18.186477 24381 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:07:18.222754 24286 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:18.165169 24297 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:18.189218 21067 rpcz_store.cc:276] 0709 13:07:16.764564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.404525 (+639961us) service_pool.cc:224] Handling call
0709 13:07:17.437990 (+ 33465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:17.438259 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:17.438297 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:17.438860 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:07:17.601868 (+163008us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:17.601910 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.601978 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:18.165938 (+563960us) raft_consensus.cc:1685] finished
0709 13:07:18.165965 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.166141 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":161923,"threads_started":1,"wal-append.queue_time_us":326156}
I20260709 13:07:18.226501 21070 rpcz_store.cc:276] 0709 13:07:17.041658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.689942 (+648284us) service_pool.cc:224] Handling call
0709 13:07:17.690252 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.690464 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:17.743805 (+ 53341us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.201230 (+457425us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.201465 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.201523 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:18.201588 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.201678 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.201833 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4759,"prepare.queue_time_us":55551,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":110,"raft.queue_time_us":6376,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":6164,"thread_start_us":451586,"threads_started":1}
I20260709 13:07:18.189596 24130 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.228401 21798 raft_consensus.cc:2368] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:07:18.189358 24197 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:18.165597 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 175241us
I20260709 13:07:18.232061 24109 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:18.204648 23761 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:18.226254 24275 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.003s
W20260709 13:07:18.225487 24425 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:07:18.200255 24376 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:18.234647 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:18.259220 24413 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:18.260615 24381 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:07:18.318439 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 83914.4us
I20260709 13:07:18.319454 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:18.320956 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:18.222246 24040 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.000s
I20260709 13:07:18.212662 24313 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:18.187582 24278 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 13:07:18.236425 24424 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:07:18.237339 23987 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:18.237761 23980 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:18.237950 24423 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 13:07:18.243412 24296 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.789s	user 0.010s	sys 0.002s
I20260709 13:07:18.247444 24312 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:18.249785 24422 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:18.249953 24333 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.000s
I20260709 13:07:18.195072 24279 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.183450 24293 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.252701 23868 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:18.255151 24281 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.590s	user 0.012s	sys 0.000s
I20260709 13:07:18.260811 24286 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:18.285722 24297 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:18.283847 24130 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:18.285889 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:18.310000 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.264313 24323 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:18.326002 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:18.353267 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.342430 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 399325us
I20260709 13:07:18.330428 24303 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:07:18.360019 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 228226us
I20260709 13:07:18.373298 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 370) took 1056 ms. Trace:
I20260709 13:07:18.373795 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:18.373735 21219 rpcz_store.cc:276] 0709 13:07:17.317194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.589280 (+272086us) service_pool.cc:224] Handling call
0709 13:07:17.589602 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:17.589979 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:17.590013 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:17.590460 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:07:17.930494 (+340034us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:17.930538 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.930616 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:18.373073 (+442457us) raft_consensus.cc:1685] finished
0709 13:07:18.373092 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.373238 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":339704,"threads_started":1,"wal-append.queue_time_us":538293}
I20260709 13:07:18.374962 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:18.323247 24302 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:18.332569 24338 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:18.286504 24197 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.334561 24143 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:18.343542 24313 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:18.378119 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 92352us
I20260709 13:07:18.322137 24275 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.340445 24422 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:18.383652 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57750.8us
I20260709 13:07:18.387416 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:18.389112 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:18.392894 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:18.393388 21067 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:18.361948 24286 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:18.389227 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:18.392895 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 258060us
I20260709 13:07:18.394348 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:18.396150 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 235662us
I20260709 13:07:18.396981 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:18.400846 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:18.395430 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 213092us
I20260709 13:07:18.376893 24317 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:18.384698 24331 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:18.381937 24323 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.359959 24333 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.383396 24312 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:18.404805 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:18.405934 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:18.386201 24211 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:07:18.386997 24381 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:18.409489 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:18.420122 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:18.420522 21796 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEARNER]: Advancing to term 5
I20260709 13:07:18.363476 24281 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.413594 21217 raft_consensus.cc:1217] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Deduplicated request from leader. Original: 5.3->[6.4-6.4]   Dedup: 6.4->[]
I20260709 13:07:18.423357 21217 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:07:18.414902 21215 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:18.387332 24303 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:18.383502 24436 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:18.426474 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:18.427248 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:18.427721 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:18.428715 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:18.431196 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.396602 24422 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:18.399160 24410 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:18.434967 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:18.424592 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:18.436084 21213 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:18.408285 24143 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:18.429802 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:18.436771 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:18.437043 21067 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:18.389714 24367 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:07:18.389209 24338 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:18.505244 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.423794 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:18.378718 24376 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:18.547979 24376 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:18.554241 21140 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
W20260709 13:07:18.555981 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.557226 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.437598 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:18.561410 21218 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:18.383083 23761 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.560194 21217 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:18.404965 24129 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:07:18.564404 21220 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:18.564714 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:18.568874 21144 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:07:18.460471 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:18.416814 24302 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:18.572753 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:18.583076 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 182284us
I20260709 13:07:18.585915 21147 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:18.587513 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:18.586190 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:18.424733 24330 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:18.597010 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:18.604597 21213 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:18.545441 23980 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.603204 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:18.606308 24302 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:18.608959 24410 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:18.608696 23980 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.609406 24410 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:18.609887 24410 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:18.593016 24436 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:18.611392 24436 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.616753 24446 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.609841 24302 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.617550 24446 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.426553 24040 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.169s
I20260709 13:07:18.621687 24040 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.622401 24040 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.643260 21214 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
W20260709 13:07:18.645609 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 256434us
W20260709 13:07:18.646327 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.460307 24437 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:18.421036 24323 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.668898 24278 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:07:18.669763 24432 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
W20260709 13:07:18.668900 24438 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
W20260709 13:07:18.679061 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75926.8us
I20260709 13:07:18.653890 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:18.700275 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.847321 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.710772 21036 leader_election.cc:400] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:18.847801 21036 leader_election.cc:403] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:07:18.731686 24333 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:18.819746 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 395126us
I20260709 13:07:18.715271 21217 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:18.735955 24197 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:18.730592 24419 consensus_meta.cc:287] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.565s	user 0.000s	sys 0.008s
I20260709 13:07:18.661141 24437 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.724452 24281 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:18.545256 24286 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:18.717135 24273 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.004s
I20260709 13:07:18.600905 24312 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:07:18.721974 24247 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:18.598265 24313 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:18.756240 24323 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:18.728499 24446 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:18.753242 24275 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:18.662644 24449 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:18.802228 21796 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEARNER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:07:18.677242 24444 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:07:18.836848 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.835451 21147 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:18.675673 24303 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:18.855408 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.883178 21796 pending_rounds.cc:85] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 2
I20260709 13:07:18.887636 21796 pending_rounds.cc:107] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 3.3
W20260709 13:07:18.887856 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.869807 24419 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:18.857829 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 147222us
I20260709 13:07:18.891703 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:18.869722 24437 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:18.436509 24246 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:18.733300 24293 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:18.864741 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 444901us
W20260709 13:07:18.870837 21187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.126s	user 0.021s	sys 0.027s
W20260709 13:07:18.955621 21187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.211s	user 0.021s	sys 0.028s
W20260709 13:07:18.956549 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 1.26379e+06us
I20260709 13:07:18.435665 24297 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.847545 24441 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:18.876329 24247 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:18.856173 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.888417 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.887949 21796 raft_consensus.cc:2889] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:18.917543 24130 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.010s
I20260709 13:07:18.675835 24143 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:18.898152 24224 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:18.908614 24040 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:18.876946 24452 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
W20260709 13:07:18.991837 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 426867us
I20260709 13:07:18.880065 24330 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.996363 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:18.895823 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:18.873219 21184 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.824s (SENT) [suppressed 37 similar messages]
I20260709 13:07:18.960381 21794 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:18.889063 24312 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:18.884757 24444 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:18.871968 24415 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:18.895931 24286 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:18.889706 24449 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:18.863363 21218 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:07:18.973758 21141 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:18.978609 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.986035 21793 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.652s	user 0.012s	sys 0.004s
I20260709 13:07:18.955680 24185 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.878805 24273 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.991745 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:18.973168 24130 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:19.008128 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.987388 24310 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:18.963315 24197 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:07:19.029011 21793 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:19.033702 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:19.069805 21796 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
W20260709 13:07:19.092984 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.065308 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:19.066370 24444 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:18.991271 24259 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.045s
W20260709 13:07:19.004648 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 108928us
I20260709 13:07:19.015215 24312 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:19.020219 24275 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:18.969092 24441 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.112751 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:19.103901 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 107611us
I20260709 13:07:19.037683 24089 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:07:19.133281 21071 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
W20260709 13:07:19.133427 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.134258 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:19.086191 24194 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.103017 24457 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.107942 24447 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.056354 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.169816 21143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2118) took 1966 ms (client timeout 2180 ms). Trace:
W20260709 13:07:19.170198 21143 rpcz_store.cc:269] 0709 13:07:17.206558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.206687 (+   129us) service_pool.cc:224] Handling call
0709 13:07:19.169633 (+1962946us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400279554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:19.169759 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575125376}
I20260709 13:07:19.038730 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:19.109684 24310 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:19.015916 24247 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:19.136528 24376 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.588s	user 0.014s	sys 0.004s
I20260709 13:07:19.135818 24275 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:19.181243 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 530589us
W20260709 13:07:19.182143 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 290505us
I20260709 13:07:19.190075 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:19.191036 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:19.192476 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:19.192919 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:19.197400 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:19.145766 24455 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:07:19.112226 24452 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:07:19.224542 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:19.199003 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51199.3us
I20260709 13:07:19.017865 23980 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.180019 24444 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:19.199810 21064 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:19.167614 24442 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:19.103266 24185 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.137434 24460 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:07:19.228755 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 262859us
I20260709 13:07:19.144286 24457 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.227214 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:19.229449 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:19.199553 24453 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:19.236960 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:19.168983 24194 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:19.242059 21215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.010s	sys 0.010s
W20260709 13:07:19.246904 21215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.010s	sys 0.010s
I20260709 13:07:19.179782 24447 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.228449 24197 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:19.272125 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 398922us
W20260709 13:07:19.276484 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.239919 24452 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:19.248428 24089 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:19.268349 24442 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:07:19.312747 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.235522 24467 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:19.253026 24471 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:19.268417 24211 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:19.187919 24449 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.349658 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:19.270633 24143 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:19.314806 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 170943us
I20260709 13:07:19.334138 21796 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:19.322252 21066 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:19.238754 24415 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
W20260709 13:07:19.315641 24418 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.326s	user 0.011s	sys 0.000s
I20260709 13:07:19.317196 24442 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:19.353371 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.352638 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.363637 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.282401 24453 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.067s
W20260709 13:07:19.367206 21066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 974) took 1575 ms (client timeout 1864 ms). Trace:
W20260709 13:07:19.372699 21066 rpcz_store.cc:269] 0709 13:07:17.776757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.805370 (+ 28613us) service_pool.cc:224] Handling call
0709 13:07:19.321973 (+1516603us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b54002144a8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f482f9924 00007f7f474e2227 00007f7f474e1ab8 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6
0709 13:07:19.352076 (+ 30103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148221312}
I20260709 13:07:19.377439 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:19.342564 24211 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:19.408874 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.416031 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 188976us
I20260709 13:07:19.279526 24310 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:19.414814 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 174784us
I20260709 13:07:19.283043 24275 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.414971 24465 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.369314 24472 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:19.373656 24442 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:19.417465 24458 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 13:07:19.415572 24211 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:19.414278 24459 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:19.409665 24293 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:07:19.476100 24194 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:19.476878 24463 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.275s	user 0.001s	sys 0.001s
I20260709 13:07:19.479256 24185 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.480942 24273 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.486358 24457 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.501849 24415 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:19.502763 24415 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:19.449175 24469 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:07:19.451680 24462 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:07:19.461082 24211 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.468021 24441 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.468916 24246 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:19.522236 24312 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:19.673183 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.700641 24246 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:19.701915 24246 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:19.462914 24109 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.140s	user 0.010s	sys 0.005s
I20260709 13:07:19.704154 21795 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:07:19.677114 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 364) took 1238 ms. Trace:
I20260709 13:07:19.454114 24143 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:19.592960 24224 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.601s	user 0.015s	sys 0.004s
I20260709 13:07:19.706403 21142 rpcz_store.cc:276] 0709 13:07:18.438534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.438692 (+   158us) service_pool.cc:224] Handling call
0709 13:07:18.439036 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.439420 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.439445 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.439938 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:07:18.992705 (+552767us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:18.992748 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.992821 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.676170 (+683349us) raft_consensus.cc:1685] finished
0709 13:07:19.676246 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.676708 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":177096,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":337324,"threads_started":1,"wal-append.queue_time_us":569467}
I20260709 13:07:19.684214 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:19.711741 24259 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.712457 24297 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.712874 24259 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.713483 24297 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.714512 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 365) took 1272 ms. Trace:
W20260709 13:07:19.689050 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 102871us
I20260709 13:07:19.716264 21072 raft_consensus.cc:2410] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:07:19.717338 24224 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:19.717862 24224 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:19.718340 24224 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:19.528403 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 811) took 1068 ms. Trace:
I20260709 13:07:19.719696 21794 rpcz_store.cc:276] 0709 13:07:18.459886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.460216 (+   330us) service_pool.cc:224] Handling call
0709 13:07:19.528222 (+1068006us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400682254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:19.528337 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861773184}
I20260709 13:07:19.704206 24109 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:19.720324 24048 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:19.721776 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:19.722672 24279 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:19.690727 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:19.723233 24279 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:19.672176 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.725121 24279 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.726683 24279 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:19.692561 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:19.728690 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:19.729135 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:19.731217 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:19.731972 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:19.732331 21138 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:19.732615 24297 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:19.733088 21139 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:19.733479 24286 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.734529 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:19.518234 24293 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:19.735673 24293 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:19.719955 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:19.737236 24293 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.737787 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:19.735671 21143 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:19.744452 21142 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:19.744819 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1011) took 1331 ms. Trace:
I20260709 13:07:19.745213 21220 rpcz_store.cc:276] 0709 13:07:18.412754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.414074 (+  1320us) service_pool.cc:224] Handling call
0709 13:07:18.414317 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.584030 (+169713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.744201 (+1160171us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b540013b354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:19.744558 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344747776}
I20260709 13:07:19.747351 24211 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:19.518460 21798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 809) took 1803 ms (client timeout 2070 ms). Trace:
W20260709 13:07:19.748523 21798 rpcz_store.cc:269] 0709 13:07:17.714980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.715115 (+   135us) service_pool.cc:224] Handling call
0709 13:07:19.518236 (+1803121us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b540060efd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:19.518375 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146160000}
I20260709 13:07:19.747547 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:19.749788 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:19.750277 21147 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:19.750648 24194 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:19.751914 24247 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:19.753607 24194 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:07:19.687819 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.721807 24278 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.757485 24246 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:19.757836 24278 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.758298 24198 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:07:19.681133 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.760514 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.761559 24278 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.762324 24278 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:19.751169 21800 raft_consensus.cc:2410] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:19.758303 24246 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:19.763868 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48930.1us
I20260709 13:07:19.734484 24286 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.714970 21141 rpcz_store.cc:276] 0709 13:07:18.441911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.442056 (+   145us) service_pool.cc:224] Handling call
0709 13:07:18.442370 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.028454 (+586084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.713917 (+685463us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b5400423bd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:19.714275 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221507968}
I20260709 13:07:19.765061 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:19.766410 24310 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:19.766758 24286 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:19.766865 24310 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:19.804723 24310 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:19.808120 21793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 698) took 2117 ms (client timeout 1824 ms). Trace:
I20260709 13:07:19.697499 24026 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:19.808656 21801 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:07:19.808646 21793 rpcz_store.cc:269] 0709 13:07:17.690274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.690425 (+   151us) service_pool.cc:224] Handling call
0709 13:07:19.807686 (+2117261us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b5400700154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:19.807891 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346067840}
W20260709 13:07:19.692627 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 450342us
I20260709 13:07:19.767182 24226 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:19.808763 24026 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:19.813683 24026 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.722997 24109 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:07:19.769424 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2125) took 2174 ms. Trace:
I20260709 13:07:19.811275 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 352) took 1389 ms. Trace:
I20260709 13:07:19.815554 21146 rpcz_store.cc:276] 0709 13:07:17.594705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.594863 (+   158us) service_pool.cc:224] Handling call
0709 13:07:17.595140 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.332062 (+736922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.332094 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.332146 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:18.332189 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.768855 (+1436666us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b54004254d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:19.768931 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.769195 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":322,"raft.queue_time_us":563,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":2821,"spinlock_wait_cycles":1693872384}
I20260709 13:07:19.785166 24312 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:19.818003 24293 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:19.792230 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:19.819734 24281 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:19.820396 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:19.821153 24281 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:19.821655 24281 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:19.821820 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:19.822257 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52418.6us
I20260709 13:07:19.793598 21064 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:19.823184 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:19.795148 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:19.801530 24130 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:19.585922 24422 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.006s	user 0.007s	sys 0.006s
W20260709 13:07:19.472298 24401 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 2.022s	user 0.007s	sys 0.011s
I20260709 13:07:19.602090 24442 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.593048 24440 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:19.543063 24338 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.811578 24226 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:19.604333 24437 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.013s
I20260709 13:07:19.523622 24447 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.510761 24459 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:19.906008 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:19.907012 21799 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:19.907261 24459 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:19.469631 24464 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:07:19.429801 24465 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:19.455790 24466 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260709 13:07:19.620115 24467 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:19.448359 24468 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:07:19.431206 24470 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:07:19.978384 24453 consensus_queue.cc:1243] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:07:19.981220 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 370) took 1527 ms. Trace:
W20260709 13:07:19.983163 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.601624 24415 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:19.739225 21069 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:19.985028 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:19.985812 21069 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:19.986410 24472 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:19.985950 24281 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:19.550907 24410 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.941s	user 0.015s	sys 0.011s
I20260709 13:07:19.986852 24472 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:19.790132 24259 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.788760 24211 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:19.816228 21801 rpcz_store.cc:276] 0709 13:07:18.421782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.421919 (+   137us) service_pool.cc:224] Handling call
0709 13:07:19.807992 (+1386073us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.809514 (+  1522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.810883 (+  1369us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400700150. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:19.811176 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3186617600}
I20260709 13:07:19.823244 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:19.825399 21064 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:19.826956 21065 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:19.763092 24278 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.882296 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:19.825332 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:19.888808 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:19.900131 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:19.869055 24026 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:19.830849 24442 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:19.825839 24284 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:19.935582 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:19.949556 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 182404us
I20260709 13:07:19.931269 24224 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.830920 24437 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:19.828004 24422 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.830332 24226 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:19.969159 24318 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:19.969755 21070 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.000s	sys 0.016s
W20260709 13:07:19.970052 21067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.000s	sys 0.009s
W20260709 13:07:19.971385 21068 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.001s	sys 0.006s
W20260709 13:07:19.971895 21187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.004s	sys 0.005s
W20260709 13:07:20.048813 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 223656us
I20260709 13:07:20.049170 24226 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:20.049546 24415 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:20.050666 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:20.051160 21219 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:20.051645 24209 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:20.052135 21218 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:20.052491 24496 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.052386 24209 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.053287 24496 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.054749 24453 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:20.055367 21070 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.089s	user 0.000s	sys 0.017s
W20260709 13:07:20.056919 21068 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.002s	sys 0.006s
I20260709 13:07:19.970866 24465 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:19.986689 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:20.019412 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 257933us
I20260709 13:07:19.987556 24410 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.020757 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.023018 24437 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:20.025072 24224 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.025568 21145 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:20.026235 24310 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.013444 24211 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:20.028472 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:20.029743 21213 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:20.030577 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:20.031585 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:20.035617 21143 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:19.981706 21140 rpcz_store.cc:276] 0709 13:07:18.453944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.555093 (+101149us) service_pool.cc:224] Handling call
0709 13:07:18.572815 (+ 17722us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.877313 (+304498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.877342 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.877889 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.182235 (+304346us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:19.420109 (+237874us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.429137 (+  9028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.429244 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.980751 (+551507us) raft_consensus.cc:1685] finished
0709 13:07:19.980787 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.981027 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106367,"prepare.queue_time_us":53637,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":114520,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":53612,"spinlock_wait_cycles":28544,"thread_start_us":739596,"threads_started":3,"wal-append.queue_time_us":393398}
I20260709 13:07:20.039132 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:19.829360 24440 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.040944 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:20.055362 24453 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.091784 24496 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:20.092221 21067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.000s	sys 0.009s
I20260709 13:07:20.093009 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:20.093854 21212 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:07:20.056525 24197 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:07:20.095582 24504 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:20.096187 24470 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:20.096722 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:19.987653 24281 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:20.137470 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 44568.7us
W20260709 13:07:20.143859 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 104852us
I20260709 13:07:20.143724 21072 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:20.147827 21069 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:20.150388 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.154850 24194 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:20.146917 21065 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:20.159548 21063 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
W20260709 13:07:20.160547 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.160995 24247 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:20.161599 24247 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:20.161695 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:20.162223 24247 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:20.163944 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:20.165578 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.166281 24185 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:07:20.166792 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.166857 24185 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:20.167384 24185 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:20.171404 24279 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:20.171614 24419 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:20.172286 24419 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.173512 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:20.173815 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:20.167408 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.175127 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:20.180837 24278 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:20.181401 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:20.184190 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:20.166349 21072 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:20.196727 21071 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:20.197772 24504 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.199807 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1001) took 3166 ms. Trace:
I20260709 13:07:20.200321 24504 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.202047 24275 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:20.205135 21145 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
W20260709 13:07:20.207862 24507 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:07:20.209237 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52626.5us
I20260709 13:07:20.210089 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:20.210980 21145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 367) took 1764 ms (client timeout 2125 ms). Trace:
I20260709 13:07:20.211191 21215 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
W20260709 13:07:20.211478 21145 rpcz_store.cc:269] 0709 13:07:18.447039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.447173 (+   134us) service_pool.cc:224] Handling call
0709 13:07:20.210735 (+1763562us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400447ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:20.210894 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3313814272}
I20260709 13:07:20.210114 24185 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.186292 21035 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:20.155458 24194 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:20.224212 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 256393us
I20260709 13:07:20.229585 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:20.225497 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:20.236653 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:20.242375 23980 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:20.243719 24189 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:20.244297 24189 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:20.244544 24410 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:20.245761 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 453) took 1262 ms. Trace:
I20260709 13:07:20.245723 24189 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.246766 24209 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:20.247961 24422 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:20.250056 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 445) took 2033 ms. Trace:
I20260709 13:07:20.250481 21063 rpcz_store.cc:276] 0709 13:07:18.216976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.225020 (+  8044us) service_pool.cc:224] Handling call
0709 13:07:18.250702 (+ 25682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.160235 (+1909533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.249792 (+ 89557us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b5400215554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:20.249987 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4294619008}
I20260709 13:07:20.251981 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.252746 21063 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:20.253814 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:20.254359 21063 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:20.255966 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 457) took 1114 ms. Trace:
I20260709 13:07:20.256502 21071 rpcz_store.cc:276] 0709 13:07:19.141267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.151483 (+ 10216us) service_pool.cc:224] Handling call
0709 13:07:19.151915 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.197320 (+1045405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.255685 (+ 58365us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b540011bcd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:20.255894 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2262372096}
I20260709 13:07:20.257772 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.232120 24504 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:20.258790 24504 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.258810 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:20.246392 21068 rpcz_store.cc:276] 0709 13:07:18.967648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.967823 (+   175us) service_pool.cc:224] Handling call
0709 13:07:18.968196 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.189876 (+221680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.189919 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.194254 (+  4335us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.374851 (+180597us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:20.091457 (+716606us) log.cc:844] Serialized 14 byte log entry
0709 13:07:20.148402 (+ 56945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.194031 (+ 45629us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.229322 (+ 35291us) raft_consensus.cc:1685] finished
0709 13:07:20.229343 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.229501 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8533,"prepare.queue_time_us":105167,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":132794,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":16496,"thread_start_us":993631,"threads_started":3,"wal-append.queue_time_us":379974}
I20260709 13:07:20.260963 24504 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.261252 24310 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:20.261873 21064 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:20.262696 21071 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 13:07:20.261909 24493 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:20.264178 24419 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:20.267627 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 145472us
W20260709 13:07:20.268023 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 88851.8us
I20260709 13:07:20.262096 21068 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:20.268635 21036 leader_election.cc:400] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:20.270010 21068 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:20.271334 21068 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:20.271726 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.272284 24211 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.272056 20968 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 4, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 4 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:07:20.262362 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 167226us
I20260709 13:07:20.270763 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:20.280577 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:20.282883 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:20.286409 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.287696 21072 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:20.228502 24194 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:20.292290 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:20.279613 21767 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:20.278570 24211 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:20.301338 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:20.200361 21216 rpcz_store.cc:276] 0709 13:07:17.033398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.033904 (+   506us) service_pool.cc:224] Handling call
0709 13:07:17.034286 (+   382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:17.102787 (+ 68501us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:17.547344 (+444557us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.871673 (+324329us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:17.872591 (+   918us) log.cc:844] Serialized 52 byte log entry
0709 13:07:17.872798 (+   207us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:17.873138 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.506481 (+1633343us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b1800425f80. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f45f143c7 00007f7f425269a7 00007f7f425360bf 00007f7f42536038
0709 13:07:19.506780 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 13:07:20.198846 (+692066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.199177 (+   331us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.199257 (+    80us) raft_consensus.cc:1685] finished
0709 13:07:20.199301 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.199562 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166766,"prepare.queue_time_us":1972415,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":132,"raft.queue_time_us":329669,"raft.run_cpu_time_us":585,"raft.run_wall_time_us":128944,"spinlock_wait_cycles":13047040,"thread_start_us":1010117,"threads_started":3,"wal-append.queue_time_us":1042131,"wal-append.run_cpu_time_us":2444,"wal-append.run_wall_time_us":1055282}
I20260709 13:07:20.228727 24278 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:20.290513 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.297016 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:20.307425 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 141737us
I20260709 13:07:20.292657 24422 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:20.310935 21214 raft_consensus.cc:2410] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:07:20.319864 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 62168us
I20260709 13:07:20.321846 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:20.315297 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:20.319975 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49315.6us
I20260709 13:07:20.333732 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:20.297245 23980 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 13:07:20.340400 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.308072 24459 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.351841 21036 leader_election.cc:400] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:20.352289 21036 leader_election.cc:403] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:20.356935 23761 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:07:20.368870 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:20.375370 21218 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:20.376976 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:20.377714 21218 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:20.382261 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:20.382928 24442 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:20.383819 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:20.384193 24442 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:20.384878 24445 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:20.385305 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:20.385506 24445 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:20.384996 24442 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:20.386713 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:20.403882 24310 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:20.386718 24445 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.404702 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:20.405265 23761 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:07:20.405472 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53838.6us
W20260709 13:07:20.405750 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 111120us
W20260709 13:07:20.406052 20967 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 223) took 26181 ms (client timeout 15000 ms). Trace:
I20260709 13:07:20.406008 23761 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:20.406570 24457 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:20.406939 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:20.407251 24457 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.411465 24457 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:20.411342 24459 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:20.412365 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 984) took 1382 ms. Trace:
I20260709 13:07:20.413710 24459 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.412724 21067 rpcz_store.cc:276] 0709 13:07:19.029736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.032426 (+  2690us) service_pool.cc:224] Handling call
0709 13:07:19.032774 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.033203 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:19.286262 (+253059us) log.cc:844] Serialized 14 byte log entry
0709 13:07:20.215445 (+929183us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:20.271663 (+ 56218us) log.cc:844] Serialized 26 byte log entry
0709 13:07:20.271822 (+   159us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:20.271857 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.271949 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.412136 (+140187us) raft_consensus.cc:1685] finished
0709 13:07:20.412154 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.412310 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15872,"prepare.queue_time_us":391372,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":3970,"raft.queue_time_us":50878,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":4741,"thread_start_us":1019590,"threads_started":2,"wal-append.queue_time_us":666420}
I20260709 13:07:20.415395 24129 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.416003 24129 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:20.407987 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.418074 24129 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.419328 24441 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:20.419816 24441 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.408398 24460 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:20.341866 23868 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:20.342164 24185 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.347216 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:20.405354 24445 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:20.309382 24472 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.407308 23761 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.354498 24278 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.475589 23761 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:20.475835 24471 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.476473 24471 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.476982 24278 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:20.477908 24189 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:20.478854 24471 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.480329 21221 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
W20260709 13:07:20.482273 21221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2539) took 3415 ms (client timeout 2173 ms). Trace:
W20260709 13:07:20.482673 21221 rpcz_store.cc:269] 0709 13:07:17.066979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.350497 (+283518us) service_pool.cc:224] Handling call
0709 13:07:20.482002 (+3131505us) spinlock_profiling.cc:243] Waited 2.99 s on lock 0x7b540013b854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:20.482146 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6868190080}
I20260709 13:07:20.483731 24453 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:20.474675 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:20.491560 24471 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:20.420532 24460 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.494951 24460 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:20.495445 24381 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:20.503778 24440 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.427119 21072 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
W20260709 13:07:20.421356 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66425us
W20260709 13:07:20.391530 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51905.3us
I20260709 13:07:20.420178 24441 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:20.354859 20968 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:20.522061 21069 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:20.523304 21068 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:20.434374 21063 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:20.411608 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:20.405534 21215 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:20.529898 21065 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:20.414935 24459 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 13:07:20.538677 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:20.540676 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 216410us
I20260709 13:07:20.541738 21218 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:07:20.552165 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.409730 24259 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:20.515902 23868 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.375432 24279 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:20.568344 20968 catalog_manager.cc:5697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:20.570950 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:20.576437 21215 raft_consensus.cc:2410] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:07:20.588078 21214 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:20.574476 23868 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:20.566709 24440 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:07:20.590677 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47227.3us
I20260709 13:07:20.566143 24259 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.591780 21220 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:20.606166 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:20.607059 21144 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:20.607421 24279 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.607831 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.608724 21147 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:07:20.609764 24279 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:20.611217 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:20.613768 24279 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.614461 24279 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:20.615032 24026 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.513s	user 0.012s	sys 0.000s
I20260709 13:07:20.617825 24457 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.621596 23980 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:20.622541 23868 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:20.624790 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.623406 24279 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:20.630343 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48100.6us
W20260709 13:07:20.631493 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.632373 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:20.634263 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.635480 21145 raft_consensus.cc:2410] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:07:20.638954 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:20.599119 24185 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:07:20.645324 24185 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:20.645910 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 451) took 1682 ms. Trace:
W20260709 13:07:20.646494 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 156201us
I20260709 13:07:20.646834 24516 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:20.647670 24516 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:20.649767 24517 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:20.650209 24517 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:20.648835 24516 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.651250 24517 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.652590 24517 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:20.606398 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.607158 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:20.656666 21143 raft_consensus.cc:2410] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:20.660025 21141 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11.
I20260709 13:07:20.596936 20968 catalog_manager.cc:5697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 5, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 5 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:20.646324 21070 rpcz_store.cc:276] 0709 13:07:18.963715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.964605 (+   890us) service_pool.cc:224] Handling call
0709 13:07:18.965013 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:20.055870 (+1090857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:20.055927 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:20.056508 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:07:20.230711 (+174203us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:20.230766 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.300349 (+ 69583us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.645701 (+345352us) raft_consensus.cc:1685] finished
0709 13:07:20.645720 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.645859 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1159,"raft.queue_time_us":54651,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":41421,"thread_start_us":1262951,"threads_started":2,"wal-append.queue_time_us":587123}
I20260709 13:07:20.654996 24259 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:20.681419 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.708617 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52999.2us
I20260709 13:07:20.715646 21141 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:20.719316 24457 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.716784 20968 catalog_manager.cc:5697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 5 to 7, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 7 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:20.729427 21143 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:07:20.668452 24504 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:20.720541 24247 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.005s
I20260709 13:07:20.663017 24194 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.742902 21144 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:20.743432 24247 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.743850 24460 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:20.745901 24509 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.426s	user 0.000s	sys 0.003s
I20260709 13:07:20.745982 24259 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.728354 24453 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.753054 20968 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:20.766908 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:20.768766 24259 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.768929 24281 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:20.406637 20967 rpcz_store.cc:269] 0709 13:06:54.194062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.195509 (+  1447us) service_pool.cc:224] Handling call
0709 13:07:20.375131 (+26179622us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:10.906012 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:10.906736 (+   724us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:10.906797 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:07:10.906835 (+    38us) tablet.cc:662] Decoding operations
0709 13:07:10.908023 (+  1188us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:10.908152 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:10.908207 (+    55us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:07:10.909414 (+  1207us) tablet.cc:701] Row locks acquired
0709 13:07:10.909487 (+    73us) write_op.cc:260] PREPARE: finished
0709 13:07:10.909734 (+   247us) write_op.cc:270] Start()
0709 13:07:10.909951 (+   217us) write_op.cc:276] Timestamp: P: 1783602430909649 usec, L: 0
0709 13:07:10.909979 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:07:10.915505 (+  5526us) log.cc:844] Serialized 3262 byte log entry
0709 13:07:11.284984 (+369479us) op_driver.cc:464] REPLICATION: finished
0709 13:07:11.402751 (+117767us) write_op.cc:301] APPLY: starting
0709 13:07:11.402882 (+   131us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:11.403150 (+   268us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:11.403349 (+   199us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:11.573484 (+170135us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:11.641638 (+ 68154us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:07:11.673116 (+ 31478us) write_op.cc:312] APPLY: finished
0709 13:07:11.731818 (+ 58702us) log.cc:844] Serialized 65 byte log entry
0709 13:07:11.811754 (+ 79936us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:11.825807 (+ 14053us) write_op.cc:454] Released schema lock
0709 13:07:11.839606 (+ 13799us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:11.881778 (+ 42172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:11.882916 (+  1138us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":276633,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":110611,"apply.run_cpu_time_us":5920,"apply.run_wall_time_us":460379,"num_ops":8,"prepare.queue_time_us":172939,"prepare.run_cpu_time_us":22056,"prepare.run_wall_time_us":54604,"raft.queue_time_us":159082,"raft.run_cpu_time_us":14135,"raft.run_wall_time_us":187947,"replication_time_us":322684,"spinlock_wait_cycles":64384,"thread_start_us":400326,"threads_started":4,"wal-append.queue_time_us":47083,"wal-append.run_cpu_time_us":19860,"wal-append.run_wall_time_us":1896640}]]}
I20260709 13:07:20.775367 21141 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:07:20.778558 21146 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:07:20.781169 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 148966us
W20260709 13:07:20.779023 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 367962us
I20260709 13:07:20.786078 21143 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:20.765683 24026 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:20.761580 24504 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.784431 24318 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:20.769593 24281 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:20.802155 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8759307399999443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 13:07:20.806957 20968 catalog_manager.cc:5697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 9, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:20.808604 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.814399 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.815349 21144 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:07:20.817838 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 203998us
W20260709 13:07:20.818075 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 80764.9us
I20260709 13:07:20.814071 20968 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 5, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 5 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:20.780998 24453 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.822669 21146 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:20.825464 21143 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:20.823988 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:20.829144 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.830001 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:20.830149 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.810642 21141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:20.830849 21145 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:20.833177 24504 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.834344 24440 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:20.834885 24440 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:20.839386 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.833178 21792 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:20.831153 21141 raft_consensus.cc:606] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:20.840548 21141 raft_consensus.cc:614] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:20.805851 24457 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:07:20.848919 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.849691 21144 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:20.827975 20968 catalog_manager.cc:5697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:20.851207 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.854367 24504 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.812019 24026 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.857051 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.856536 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.856557 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.867594 21143 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:20.857563 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.847849 24185 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:20.869076 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:20.872110 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.875272 21144 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:20.873294 24460 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.874312 21141 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:20.853969 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:20.878041 21145 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:20.877851 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.878945 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:20.855131 24318 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:20.878695 21140 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:20.886612 21147 raft_consensus.cc:2368] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
W20260709 13:07:20.887543 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.889429 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.890355 21144 raft_consensus.cc:2368] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:20.877787 20968 catalog_manager.cc:5697] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 2, term changed from 1 to 3, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to <none>, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:07:20.883123 21036 leader_election.cc:336] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.173s (SENT)
I20260709 13:07:20.854929 24504 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:20.895083 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:20.899226 21145 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:20.893728 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:20.893186 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.896278 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:20.899914 24471 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.900019 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.906747 21146 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:20.900625 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.906965 21071 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:20.900718 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.901147 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.884030 23980 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:20.903647 24310 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:20.885993 24185 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:20.903091 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.910657 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.912499 21144 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:20.913609 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:20.888397 24247 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:20.914324 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 112523us
I20260709 13:07:20.917209 21141 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:20.916949 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.918915 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.927547 21147 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:20.932266 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:20.932964 21063 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:20.934752 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.909821 24194 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:20.939167 21069 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
W20260709 13:07:20.939632 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 109748us
I20260709 13:07:20.940335 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:20.940871 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:20.941735 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.942516 24189 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:20.943306 24197 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:20.944099 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.944229 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:20.943753 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.945556 21141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:20.946236 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:20.947715 21144 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:20.948676 21139 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:20.949146 20968 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 3 to 5, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 5 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:20.953807 21147 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:20.954627 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:20.955917 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.956918 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:20.959174 24504 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:20.960183 21144 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:07:20.969722 21146 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:20.960990 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:20.925845 23980 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:20.976236 24247 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:20.976768 21144 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:20.914213 24471 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.948740 24281 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:20.973155 20968 catalog_manager.cc:5697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:20.977603 21140 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:20.976718 24247 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.989974 21069 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:20.970873 24457 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.993083 20968 catalog_manager.cc:5697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:20.998510 21147 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:21.001554 21142 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:07:21.003680 21143 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:20.947896 24310 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:20.980091 24504 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:21.004338 21141 raft_consensus.cc:606] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:21.013670 21141 raft_consensus.cc:614] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:21.014554 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:21.016367 21069 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
W20260709 13:07:21.020947 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.023010 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:21.024017 21147 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:21.037009 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.039400 24527 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:07:20.999881 24247 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:21.044301 20968 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 7, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 7 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:21.047453 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:21.048229 21067 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:21.000560 24453 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:21.051992 21147 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:21.052244 24310 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:21.055370 21110 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.018323 24185 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.055609 24247 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.058107 23761 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:21.058533 23761 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:21.058724 24445 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:21.046193 20968 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 6 to 8, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 8 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:21.097395 21143 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 3. Current term is 4. Ops: []
W20260709 13:07:21.099221 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 169447us
I20260709 13:07:21.100234 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:21.059455 24445 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:21.108381 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:21.113991 21140 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:21.107021 20968 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:21.116063 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.116430 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:21.117048 21144 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:21.119585 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.103132 24026 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:21.122222 20968 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:21.134217 24471 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:21.098511 24457 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:21.139582 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:21.140458 21143 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:21.141325 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:21.151645 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.152431 21220 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:21.153460 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.154295 21212 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:21.112351 24517 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.157606 24517 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.121872 23980 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.162325 24440 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:21.163996 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 51887us
I20260709 13:07:21.167177 24275 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:21.167776 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.168704 21185 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:21.168856 24275 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:21.169385 24275 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:21.170121 21185 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11.
I20260709 13:07:21.170481 21185 leader_election.cc:403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:21.171649 24281 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:21.172485 24281 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:07:21.176841 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 57358.4us
I20260709 13:07:21.177707 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.178673 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.128157 24197 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:21.179483 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.181524 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.182278 21216 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:21.186944 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.180596 21213 raft_consensus.cc:2410] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:07:21.187366 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.186451 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.192662 21218 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:21.134928 24279 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:21.189626 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.191251 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.170258 24275 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.200877 21063 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:21.185318 24478 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:21.192886 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.208542 21216 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:21.213565 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46312.3us
I20260709 13:07:21.217489 21214 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:21.059487 23761 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.219765 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.222510 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.223304 21218 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:21.225939 20968 catalog_manager.cc:5697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:21.220754 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.231297 21212 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:21.226768 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:21.207353 24259 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:07:21.238179 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.239295 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.236326 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.124188 24445 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.249236 21213 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:21.246063 20968 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:21.250073 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.255919 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.256889 21221 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:21.257999 21221 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:21.263180 21140 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:21.263841 21221 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:21.265578 21221 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:21.267133 21221 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 4. (term mismatch)
I20260709 13:07:21.267560 21221 pending_rounds.cc:85] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 3
I20260709 13:07:21.267834 21221 pending_rounds.cc:107] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 13:07:21.268172 21221 raft_consensus.cc:2889] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:21.269443 24459 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:21.269879 24459 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.269990 21221 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:21.270311 24459 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:21.271459 21221 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:21.276552 24457 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:21.277257 24440 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:21.277446 24457 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.277932 24457 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:21.278414 24447 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:21.248184 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.280965 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.283767 21212 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:21.284811 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.286423 21215 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:21.290644 20968 catalog_manager.cc:5697] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 2 to 4, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 4 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:07:21.292248 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.294440 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 417004us
I20260709 13:07:21.295003 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.302299 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.303452 21218 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:21.285769 24275 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:21.299961 24441 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.765s	user 0.008s	sys 0.004s
I20260709 13:07:21.305369 20968 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 6, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:21.298959 24532 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:07:21.321894 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:21.340240 21063 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:07:21.343999 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.347775 24517 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:21.357537 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.361181 24471 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:21.344724 24441 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.365509 21070 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:21.366190 24441 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:21.374166 23868 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:21.374876 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.375933 21218 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
W20260709 13:07:21.377599 24506 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.068s	user 0.000s	sys 0.002s
I20260709 13:07:21.377497 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.381498 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.382207 21214 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:21.377691 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.387064 21213 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:21.388545 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.394100 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:21.381506 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 191936us
I20260709 13:07:21.400767 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.403261 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:21.405311 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 45999.4us
I20260709 13:07:21.402396 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.415093 21212 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:21.418669 21213 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:07:21.341820 24528 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
W20260709 13:07:21.422122 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.430584 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 108863us
I20260709 13:07:21.431730 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.434778 24318 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:21.436038 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.435312 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.436893 21214 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:07:21.442078 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.450429 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:21.452262 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:21.453147 21798 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:21.456383 24453 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:21.457026 24453 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.457500 24453 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:21.461427 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:21.468083 23761 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:21.471961 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1935) took 1738 ms. Trace:
I20260709 13:07:21.471841 24189 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:07:21.473083 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.468230 24445 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:21.473623 24535 raft_consensus.cc:2764] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:07:21.474049 24445 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:21.474723 24247 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:21.474606 24445 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.476394 24247 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:21.476792 24247 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.476850 24445 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:21.478377 21065 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:21.479964 21065 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:07:21.481485 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.473217 24189 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:21.487128 24189 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:21.488328 24189 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:21.489473 24189 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:21.491643 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:21.492394 21070 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:21.492269 24419 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:21.492910 24419 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:21.493398 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:21.494177 21070 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:21.493793 24419 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.495369 24457 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:21.497850 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 384) took 1155 ms. Trace:
I20260709 13:07:21.498459 24472 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:21.499197 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 468) took 1321 ms. Trace:
I20260709 13:07:21.499665 21068 rpcz_store.cc:276] 0709 13:07:20.178159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.272136 (+ 93977us) service_pool.cc:224] Handling call
0709 13:07:20.523113 (+250977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.526946 (+  3833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.498947 (+972001us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540026e150. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:21.499085 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576538496}
I20260709 13:07:21.472967 21138 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:07:21.504029 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.432745 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.493712 24459 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:21.487102 21798 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:21.494987 24419 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:21.477157 24247 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:21.486917 24281 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11.
I20260709 13:07:21.517393 21216 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:21.495060 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:21.472534 21066 rpcz_store.cc:276] 0709 13:07:19.733933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.739418 (+  5485us) service_pool.cc:224] Handling call
0709 13:07:19.739938 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.740906 (+   968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.740936 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.741593 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.946621 (+205028us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:19.946663 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.946728 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:21.376828 (+1430100us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:21.471600 (+ 94772us) raft_consensus.cc:1685] finished
0709 13:07:21.471624 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.471870 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1791,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":139,"spinlock_wait_cycles":34514816,"thread_start_us":204576,"threads_started":1,"wal-append.queue_time_us":403383}
I20260709 13:07:21.517431 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.498225 21217 rpcz_store.cc:276] 0709 13:07:20.342399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.345041 (+  2642us) service_pool.cc:224] Handling call
0709 13:07:20.367747 (+ 22706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.368098 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:20.658862 (+290764us) log.cc:844] Serialized 14 byte log entry
0709 13:07:21.497223 (+838361us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b1800466720. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a
0709 13:07:21.497441 (+   218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:21.497486 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:21.497531 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.497589 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.497689 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":366560,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":36,"raft.queue_time_us":120,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":8027904,"thread_start_us":397919,"threads_started":2,"wal-append.queue_time_us":260639}
I20260709 13:07:21.515715 21035 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:21.531805 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 97160.6us
I20260709 13:07:21.518005 21063 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:21.524659 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:21.524153 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:21.527264 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:21.533499 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:21.534083 21138 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:21.504361 24445 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:07:21.527968 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 104758us
I20260709 13:07:21.536216 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.537401 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:21.515456 24535 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:21.547606 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:21.543995 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:21.546892 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:21.550041 21801 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:21.544301 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.555003 21792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 712) took 1341 ms. Trace:
I20260709 13:07:21.557281 21072 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:21.555451 21792 rpcz_store.cc:276] 0709 13:07:20.213170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.215912 (+  2742us) service_pool.cc:224] Handling call
0709 13:07:20.235137 (+ 19225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.840366 (+605229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.550083 (+709717us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b5400380254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:21.554926 (+  4843us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1220632320}
I20260709 13:07:21.535501 24226 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:21.564384 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.566401 21796 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:21.535678 24536 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:21.552285 24445 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:21.535315 24441 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 13:07:21.592602 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55966.9us
I20260709 13:07:21.598240 21220 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:21.600557 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.563874 24226 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:21.566161 23868 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:21.559754 24281 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:07:21.580870 24517 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:07:21.613976 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 443988us
I20260709 13:07:21.614610 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.610596 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.622911 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.623584 21216 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:21.623522 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.612401 21218 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:21.621162 21217 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:21.593309 24259 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:21.626116 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.629261 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.594439 24536 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:21.629392 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:21.639026 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 43835.6us
I20260709 13:07:21.616397 24226 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:21.640091 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.622269 24517 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:21.630949 24478 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:21.653834 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.646492 24310 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:21.636682 24279 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:21.657819 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:21.658586 21213 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:21.639546 24318 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:21.629995 24530 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:07:21.652469 24517 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:21.669248 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:21.680116 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:21.656203 23868 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:21.691143 21216 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:21.692821 24547 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:21.692692 24259 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:21.693424 24547 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:21.694353 24547 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:21.695840 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.657202 24310 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:21.655961 24281 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 13:07:21.708950 24538 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:07:21.685328 23868 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:21.711647 23868 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:21.695595 24471 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:21.714874 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65799.9us
I20260709 13:07:21.708132 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:21.719257 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.685843 24445 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:21.697983 24459 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:21.725322 24318 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:21.725876 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:21.726696 21139 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:21.729875 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:21.703114 24310 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:21.701413 24460 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:21.737229 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.732586 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:21.738032 24537 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.262s	user 0.001s	sys 0.001s
I20260709 13:07:21.739579 24459 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:21.776070 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 359254us
I20260709 13:07:21.778280 21213 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:21.781992 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:21.757050 24471 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:21.770288 24460 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:21.816052 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 98363.9us
I20260709 13:07:21.794275 24281 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:21.822916 21801 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:21.826057 24281 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:21.792781 24471 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:21.829519 24540 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:07:21.736253 24536 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.830683 21036 leader_election.cc:400] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:21.846877 21036 leader_election.cc:403] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:21.843391 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:21.739710 24478 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:21.779920 24279 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:21.865773 24542 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:07:21.866688 24545 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:21.867136 24545 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.867578 24545 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:21.867218 24281 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:21.868739 24281 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:21.870119 24247 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:21.871992 24543 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:21.872305 24247 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:21.872920 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:21.873960 24548 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:07:21.830179 24549 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:07:21.876003 24539 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:07:21.877315 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:21.877600 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 60641.6us
I20260709 13:07:21.877898 24535 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:21.878373 21036 leader_election.cc:400] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:07:21.878454 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:21.878731 21036 leader_election.cc:403] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:07:21.875162 24189 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.879354 24535 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5.
I20260709 13:07:21.879992 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:21.880856 24535 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:21.881250 24535 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:21.881647 24535 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:21.881800 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:21.879992 24189 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:21.885700 24189 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:21.886307 24189 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.827057 24550 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:07:21.795589 24459 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:07:21.899132 21138 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:21.886855 24189 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:21.906332 24189 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:21.917400 24197 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:21.919869 24530 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.846550 24471 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:21.925350 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:21.926759 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 294524us
I20260709 13:07:21.928830 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:21.920523 24548 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.932158 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:21.960084 21138 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
W20260709 13:07:21.935456 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 356996us
I20260709 13:07:21.930565 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:21.930565 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 386690us
W20260709 13:07:21.947186 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.961676 21185 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:21.963001 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:21.970183 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:22.008935 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:22.011449 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49816.1us
I20260709 13:07:21.936795 24197 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:22.014248 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52663us
I20260709 13:07:22.021117 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:22.025691 21216 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
W20260709 13:07:22.025714 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 293185us
I20260709 13:07:22.034912 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:22.037138 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:21.995126 24460 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:22.045848 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:22.047526 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:22.024307 23761 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:22.052783 24457 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:22.045197 23868 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:22.056953 24535 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:22.058614 23868 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:22.023967 24419 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:22.059835 21070 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:22.062316 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:21.957413 24553 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:07:22.026762 24281 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:21.961839 24530 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:22.059039 24535 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:22.068526 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:22.074213 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:22.061465 24453 consensus_meta.cc:287] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.603s	user 0.014s	sys 0.000s
I20260709 13:07:22.059535 24247 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:21.995486 24548 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:22.062759 23868 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:07:22.081498 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 199752us
I20260709 13:07:22.072331 24026 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.082266 21792 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:07:22.063218 23761 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:22.073958 24457 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:22.059785 24459 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:07:22.067436 24571 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.146198 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:22.119755 24549 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:22.070701 24226 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:22.076678 24445 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:07:22.156919 21064 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:22.128531 24281 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:22.138288 24259 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:22.168226 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:22.082135 24453 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:22.172056 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 103627us
I20260709 13:07:22.139583 24536 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:07:22.148963 24460 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:22.182073 21212 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:22.161412 24549 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.165035 24457 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:22.141299 24535 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:07:22.140087 24026 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:22.189087 21221 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:22.189618 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 153449us
I20260709 13:07:22.191849 24472 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:22.190712 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:22.142582 24197 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:22.174283 23868 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:07:22.154166 24563 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:07:22.224234 21064 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:22.209415 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1022) took 1116 ms. Trace:
I20260709 13:07:22.213335 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:22.214144 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:22.227480 21215 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:22.166304 24571 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:22.215543 24551 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:07:22.216002 24275 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:22.187315 24549 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:22.205303 24552 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:07:22.205770 24541 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:07:22.225049 21069 rpcz_store.cc:276] 0709 13:07:21.092942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.480419 (+387477us) service_pool.cc:224] Handling call
0709 13:07:21.480820 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:21.481261 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:21.481290 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:21.481957 (+   667us) log.cc:844] Serialized 52 byte log entry
0709 13:07:21.923677 (+441720us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:21.923748 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.923853 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.209180 (+285327us) raft_consensus.cc:1685] finished
0709 13:07:22.209201 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.209355 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":441262,"threads_started":1,"wal-append.queue_time_us":725526}
I20260709 13:07:22.171489 24562 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:07:22.265056 23761 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:22.268697 24536 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:22.269958 24453 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:22.297431 21220 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:22.216536 24281 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:22.304029 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 169585us
W20260709 13:07:22.307919 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 84002.8us
I20260709 13:07:22.309060 21185 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:22.321264 24275 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:22.322844 24275 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:22.329309 24555 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.261s	user 0.001s	sys 0.001s
I20260709 13:07:22.300743 24197 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:22.333665 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:22.218418 24472 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:22.340030 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:22.300635 24536 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:22.342272 24548 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:22.315863 24563 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:07:22.335958 24457 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:22.457242 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:22.457854 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 92431.5us
I20260709 13:07:22.470436 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:22.286769 24460 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:22.461642 21212 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:07:22.472090 21215 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:07:22.483120 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.484588 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1017) took 1566 ms. Trace:
I20260709 13:07:22.492374 21067 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:07:22.494999 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:22.485037 21071 rpcz_store.cc:276] 0709 13:07:20.917979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.924568 (+  6589us) service_pool.cc:224] Handling call
0709 13:07:20.943892 (+ 19324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:21.325829 (+381937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:21.325876 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:21.346869 (+ 20993us) log.cc:844] Serialized 26 byte log entry
0709 13:07:21.779979 (+433110us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:21.780036 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.780130 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.484286 (+704156us) raft_consensus.cc:1685] finished
0709 13:07:22.484315 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.484503 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99431,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":814092,"threads_started":2,"wal-append.queue_time_us":578584}
W20260709 13:07:22.476395 24517 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.004s
W20260709 13:07:22.510874 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 201891us
I20260709 13:07:22.470134 24535 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:07:22.482700 24310 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.733s	user 0.012s	sys 0.004s
W20260709 13:07:22.485605 24585 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.184s	user 0.000s	sys 0.003s
I20260709 13:07:22.515717 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:22.500181 24528 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.483506 24453 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:22.479892 24595 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:22.526108 21138 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:07:22.495398 24578 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.164s	user 0.000s	sys 0.003s
I20260709 13:07:22.493059 24519 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:22.549731 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 54859.3us
I20260709 13:07:22.550608 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:22.551822 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:22.552851 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:22.556546 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:22.560642 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:22.515271 24226 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:22.573390 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 103026us
I20260709 13:07:22.779301 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:22.781618 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 225140us
I20260709 13:07:22.782795 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:22.535357 24595 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:22.542212 24478 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.558260 24279 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:22.527127 24517 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:22.501632 24592 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.283s
I20260709 13:07:22.530674 24310 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:22.782871 24459 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:22.844162 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2558) took 1660 ms. Trace:
I20260709 13:07:22.558784 24572 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:22.558928 24553 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:22.814515 24547 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:22.785136 24445 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:22.888113 21796 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:22.550165 24453 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:22.860184 24602 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:22.814110 24578 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
I20260709 13:07:22.821796 24226 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:22.524818 24536 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:22.892402 24445 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:22.896255 24536 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:22.897617 24445 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:22.898931 24595 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:22.899377 24602 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:22.900414 24595 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:22.901834 24602 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:22.903595 24586 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:07:22.836859 24571 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:22.851790 21219 rpcz_store.cc:276] 0709 13:07:21.183570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.183723 (+   153us) service_pool.cc:224] Handling call
0709 13:07:21.214020 (+ 30297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:21.301423 (+ 87403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:21.301468 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:21.314930 (+ 13462us) log.cc:844] Serialized 26 byte log entry
0709 13:07:21.788703 (+473773us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:21.815932 (+ 27229us) log.cc:844] Serialized 14 byte log entry
0709 13:07:21.816150 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.968470 (+152320us) spinlock_profiling.cc:243] Waited 76.4 ms on lock 0x7b54002ebb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:21.968517 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.813284 (+844767us) raft_consensus.cc:1685] finished
0709 13:07:22.843905 (+ 30621us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.844085 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3609,"prepare.queue_time_us":27070,"prepare.run_cpu_time_us":160,"prepare.run_wall_time_us":5364,"raft.queue_time_us":62744,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":12096,"spinlock_wait_cycles":175659776,"thread_start_us":399312,"threads_started":1,"wal-append.queue_time_us":614002}
I20260709 13:07:22.904834 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 392) took 1296 ms. Trace:
I20260709 13:07:22.892911 21072 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:22.541966 24528 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:22.850478 24318 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:22.906719 21217 rpcz_store.cc:276] 0709 13:07:21.608266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.656149 (+ 47883us) service_pool.cc:224] Handling call
0709 13:07:21.677858 (+ 21709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.014870 (+337012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.014946 (+    76us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.032609 (+ 17663us) log.cc:844] Serialized 26 byte log entry
0709 13:07:22.147404 (+114795us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:22.147477 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.147651 (+   174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.904598 (+756947us) raft_consensus.cc:1685] finished
0709 13:07:22.904622 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.904773 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83122,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":11131,"thread_start_us":406725,"threads_started":2,"wal-append.queue_time_us":434164}
I20260709 13:07:22.857470 24026 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:22.844820 24478 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:22.919458 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 358864us
I20260709 13:07:22.922044 24594 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:22.922632 24545 consensus_meta.cc:287] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.055s	user 0.011s	sys 0.006s
I20260709 13:07:22.572278 24519 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:22.923130 24592 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:22.922029 24472 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:22.927484 24571 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.942605 24026 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:22.942653 24318 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:22.945771 24535 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:22.950248 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 167438us
W20260709 13:07:22.954692 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.011364 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 57284.3us
W20260709 13:07:23.030480 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.027915 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 247155us
I20260709 13:07:23.032420 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:23.008515 24594 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.026365 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:23.027326 24318 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:23.020195 24580 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.508s	user 0.000s	sys 0.001s
I20260709 13:07:23.023624 24565 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:23.019874 24592 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:22.936398 24604 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.115s
I20260709 13:07:23.030686 24275 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.043334 24478 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.059005 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.075850 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.022431 24528 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.040840 24547 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.048525 24565 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:23.041054 24584 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:23.041385 24594 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:23.079568 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.019538 24460 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:07:23.076330 21142 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:23.090152 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:23.093853 21142 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:07:23.090370 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.086840 21147 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:23.012584 24545 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.086030 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.047551 24026 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:23.047667 24607 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:23.021097 24571 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.108006 24026 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.110652 24571 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.091239 21145 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:23.111990 24603 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:07:23.112865 24614 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 13:07:23.091679 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.007s	sys 0.043s
I20260709 13:07:23.087409 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.100607 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.116266 21143 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
W20260709 13:07:23.117025 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.007s	sys 0.043s
W20260709 13:07:23.105019 24575 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 13:07:23.105939 24471 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.138s	user 0.011s	sys 0.001s
W20260709 13:07:23.108553 24197 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.000s
I20260709 13:07:23.126559 24530 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:23.042975 24318 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.127503 24310 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.127713 24318 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:23.128113 24310 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:23.128485 24576 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.128603 24310 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.128999 24576 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.130604 24536 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:23.130801 24310 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.131601 24592 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.132416 24596 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:23.133028 24596 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.135207 24571 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.136201 24519 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.136471 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:23.118717 24535 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.138177 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:23.138501 24545 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.125108 24478 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:23.146358 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:23.125566 24471 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.135998 24571 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.141607 21138 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:23.081079 24275 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.099025 24604 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:07:23.146061 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.152884 24445 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:23.159449 24553 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:23.160450 24545 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.163064 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:23.232386 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 69405.1us
I20260709 13:07:23.170626 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:23.200155 21221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1052) took 1623 ms (client timeout 2143 ms). Trace:
W20260709 13:07:23.233562 21221 rpcz_store.cc:269] 0709 13:07:21.518537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.518691 (+   154us) service_pool.cc:224] Handling call
0709 13:07:23.141265 (+1622574us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b540034b7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:23.141385 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1493969280}
W20260709 13:07:23.107267 24556 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.979s	user 0.001s	sys 0.001s
I20260709 13:07:23.234144 24457 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:23.234236 24318 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:23.234990 24318 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:23.201437 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57989.9us
I20260709 13:07:23.237160 24318 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.237748 24318 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.238152 24318 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:23.145880 24528 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:23.239104 24602 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:23.239737 24623 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.240332 24530 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:23.240244 24623 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.240747 24530 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:23.224931 24604 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:23.241911 24604 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.242542 24623 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.097515 24584 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.245168 24584 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.246102 24517 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.251652 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:23.252642 21213 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:07:23.206105 21216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:07:23.102821 24453 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:23.116849 24607 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:23.090108 24611 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:23.099830 24608 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:23.493209 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.200740 24185 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:23.202207 24519 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:07:23.212724 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 180389us
I20260709 13:07:23.131108 24576 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.241783 24530 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.135075 24596 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:23.254580 21037 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.013s	sys 0.019s
I20260709 13:07:23.208238 21144 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: [6.7-7.8]
W20260709 13:07:23.519318 21037 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.013s	sys 0.019s
I20260709 13:07:23.250929 24517 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.090953 24547 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.340193 24310 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.525822 24596 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.525960 24310 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.526513 24596 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.528280 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1963) took 1606 ms. Trace:
I20260709 13:07:23.528749 24310 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.528681 21072 rpcz_store.cc:276] 0709 13:07:21.922114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.151579 (+229465us) service_pool.cc:224] Handling call
0709 13:07:23.528097 (+1376518us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b5400214154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:23.528197 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665125760}
I20260709 13:07:23.355001 24453 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:23.344995 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1025) took 1565 ms. Trace:
I20260709 13:07:23.423476 24612 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.529819 24453 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.530140 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.232092 24565 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.354179 24318 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:23.530606 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.530831 21072 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:23.532109 24453 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.528681 24596 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:23.499554 24611 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.287186 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:23.535602 21212 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:23.536736 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:23.536937 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:23.537292 21212 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:23.537819 24441 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:23.538226 24441 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.538583 24441 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:23.354151 24613 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.282405 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.539927 21797 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:23.539577 24613 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.282405 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:23.278925 24607 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:23.353886 24618 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:23.239781 24528 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:07:23.541493 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:23.541862 24613 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:23.279040 24281 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.542629 24281 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.543134 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1949) took 2618 ms. Trace:
I20260709 13:07:23.541493 24618 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:23.544708 24600 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.409s	user 0.001s	sys 0.001s
I20260709 13:07:23.544842 24281 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.257256 24445 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.545992 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1064) took 1348 ms. Trace:
I20260709 13:07:23.546072 24618 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:23.546883 24445 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.215565 24478 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.547091 24517 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.543526 21067 rpcz_store.cc:276] 0709 13:07:20.924970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.049980 (+125010us) service_pool.cc:224] Handling call
0709 13:07:22.486061 (+1436081us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.500060 (+ 13999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.542930 (+1042870us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54000dc350. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:23.543055 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3301912320}
I20260709 13:07:23.548796 24478 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.419086 24589 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:23.549191 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:23.549511 24589 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.549865 24516 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:23.289475 21217 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
W20260709 13:07:23.284380 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 113861us
I20260709 13:07:23.307224 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.552040 21068 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:23.422042 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 717) took 1770 ms. Trace:
I20260709 13:07:23.382584 24590 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:23.257158 24545 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.466639 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1059) took 1271 ms. Trace:
I20260709 13:07:23.552866 21796 rpcz_store.cc:276] 0709 13:07:21.651847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.652037 (+   190us) service_pool.cc:224] Handling call
0709 13:07:21.668032 (+ 15995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.891600 (+1223568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.421716 (+530116us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400700dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:23.421971 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2805378432}
I20260709 13:07:23.303251 24524 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:23.573275 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:23.283202 23868 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:23.419507 24609 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:07:23.243103 24604 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:23.282123 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.586400 21141 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:23.289000 21799 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:07:23.553511 24595 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.653s	user 0.013s	sys 0.000s
I20260709 13:07:23.260394 24584 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:07:23.382889 24571 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.522418 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.524142 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.523425 24519 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:23.542729 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:23.631019 21064 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:23.632460 21213 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:23.549326 24517 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:23.640218 24594 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.000s
I20260709 13:07:23.640713 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 394) took 1146 ms. Trace:
I20260709 13:07:23.587225 24590 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:23.616752 23868 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.643250 24524 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:23.644109 23868 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.644276 24524 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.647111 24545 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.647666 21799 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEARNER]: Advancing to term 6
I20260709 13:07:23.649163 24547 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.553594 21184 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:23.650004 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.323546 24623 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.499053 24549 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.203s	user 0.006s	sys 0.006s
I20260709 13:07:23.567664 21216 raft_consensus.cc:606] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:23.569182 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.657989 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.546365 21218 rpcz_store.cc:276] 0709 13:07:22.197299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.215818 (+ 18519us) service_pool.cc:224] Handling call
0709 13:07:22.216144 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.297240 (+ 81096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.299492 (+  2252us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.302993 (+  3501us) log.cc:844] Serialized 26 byte log entry
0709 13:07:22.530481 (+227488us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:22.530527 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.530718 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.544085 (+1013367us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:23.545700 (+  1615us) raft_consensus.cc:1685] finished
0709 13:07:23.545720 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.545936 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64537,"raft.queue_time_us":186401,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":152,"spinlock_wait_cycles":13374336,"thread_start_us":248101,"threads_started":2,"wal-append.queue_time_us":752435}
I20260709 13:07:23.259163 24553 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:23.573756 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 48923.5us
I20260709 13:07:23.492820 24608 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.579779 21068 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:23.529733 21066 rpcz_store.cc:276] 0709 13:07:21.779140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.779357 (+   217us) service_pool.cc:224] Handling call
0709 13:07:21.789137 (+  9780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.789417 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:22.044840 (+255423us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.035718 (+990878us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b1800415cc0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a
0709 13:07:23.344516 (+308798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.344586 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:23.344656 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.344755 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.344915 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15642,"prepare.queue_time_us":357180,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":114464,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":214,"spinlock_wait_cycles":2769664,"thread_start_us":656464,"threads_started":3,"wal-append.queue_time_us":1476457}
I20260709 13:07:23.583797 21793 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:23.586081 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.310506 21219 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:07:23.589509 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.572508 21214 rpcz_store.cc:276] 0709 13:07:22.194904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.195006 (+   102us) service_pool.cc:224] Handling call
0709 13:07:22.195309 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.457531 (+262222us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:22.500010 (+ 42479us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.072524 (+572514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.072970 (+   446us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:23.073246 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.466315 (+393069us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400442fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:23.466377 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.466562 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18517,"prepare.queue_time_us":42569,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":167415,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":10489,"spinlock_wait_cycles":554296704,"thread_start_us":556220,"threads_started":2,"wal-append.queue_time_us":850168}
I20260709 13:07:23.592494 21212 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:23.602738 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.603511 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.530483 24612 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.613158 24530 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:23.585613 24618 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.615938 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.549244 24565 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:23.626204 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:23.741696 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 115568us
I20260709 13:07:23.742599 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:23.742830 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:23.633302 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.745066 21064 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:23.631714 24519 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.637771 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:23.749015 21213 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:23.638367 24471 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.638759 24611 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.638885 24275 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.641096 21146 rpcz_store.cc:276] 0709 13:07:22.494544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.494720 (+   176us) service_pool.cc:224] Handling call
0709 13:07:22.521662 (+ 26942us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:22.553565 (+ 31903us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.553605 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:22.578797 (+ 25192us) log.cc:844] Serialized 52 byte log entry
0709 13:07:22.947191 (+368394us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:23.065692 (+118501us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.066002 (+   310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.066106 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.640500 (+574394us) raft_consensus.cc:1685] finished
0709 13:07:23.640520 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.640654 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31499,"prepare.queue_time_us":683172,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":32592,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":234100,"thread_start_us":199609,"threads_started":2,"wal-append.queue_time_us":250716}
I20260709 13:07:23.646965 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.551185 21217 pending_rounds.cc:85] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 8
I20260709 13:07:23.343832 21069 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:07:23.650266 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 96742.5us
I20260709 13:07:23.551138 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:23.676489 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.672710 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:23.686450 21218 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:23.737433 21796 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:23.738086 21219 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:23.739651 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.744311 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:23.751564 21214 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:23.753993 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.755963 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.864657 21066 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:23.864559 21064 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:23.763854 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:23.644124 24590 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:23.776949 21144 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:07:23.865501 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.777945 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.768914 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.776182 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.780453 21185 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:23.771268 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.775689 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.850001 21799 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:23.785178 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:23.675889 24593 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:23.849136 21067 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:23.873411 21799 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:07:23.878923 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 93817.2us
I20260709 13:07:23.835377 21217 pending_rounds.cc:107] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:07:23.859625 21792 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:23.880630 21217 raft_consensus.cc:2889] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:23.862628 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.769544 21215 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:23.850061 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:23.868878 21142 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:23.780669 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:23.657631 24310 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:23.884989 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.866245 21068 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:23.744138 24576 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.884931 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 140705us
I20260709 13:07:23.886103 21064 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:23.888443 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.888952 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.753278 24613 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:23.750339 24571 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:23.736210 24608 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:23.889587 21794 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:23.872442 24593 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:23.893580 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.893610 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:23.640690 24594 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.894848 21145 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:23.676640 24549 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:23.897029 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 47049.4us
I20260709 13:07:23.898204 21218 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:23.930927 21219 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:23.890736 24275 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:07:23.973753 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 193430us
I20260709 13:07:23.769866 24632 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.900799 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.904858 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.907269 21143 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:23.676847 24623 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:23.911093 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 174283us
I20260709 13:07:23.952404 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:23.952253 21066 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
W20260709 13:07:23.953181 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46588.4us
I20260709 13:07:23.963135 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:23.748833 24618 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.965706 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:23.889178 24590 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:23.892746 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:23.897039 24471 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.892726 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:23.778906 24553 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:23.901597 24604 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.896499 24576 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.892318 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:23.976423 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 82911.5us
I20260709 13:07:23.904583 24478 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:23.945883 24613 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.974607 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:23.903995 24310 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.786412 24557 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Committing config change with OpId 7.7: config changed from index 3 to 7, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:23.977082 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:24.044896 21070 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:24.661621 21796 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:23.988745 21219 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:24.657672 21064 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:23.958827 24593 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:24.139544 21801 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:23.978421 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:24.234628 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 343407us
I20260709 13:07:24.667788 21067 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:24.658157 21218 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
W20260709 13:07:24.283128 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 288066us
I20260709 13:07:23.966027 24606 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:24.689071 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:24.690099 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:24.694383 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1976) took 1090 ms. Trace:
I20260709 13:07:24.694900 21067 rpcz_store.cc:276] 0709 13:07:23.604396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.888152 (+283756us) service_pool.cc:224] Handling call
0709 13:07:24.694320 (+806168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:24.692164 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:24.696123 24453 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:24.699208 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:24.700111 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:24.702739 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1049) took 1028 ms. Trace:
I20260709 13:07:24.703146 21064 rpcz_store.cc:276] 0709 13:07:23.673822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.699861 (+1026039us) service_pool.cc:224] Handling call
0709 13:07:24.702666 (+  2805us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:24.645102 24624 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.241s	user 0.000s	sys 0.000s
I20260709 13:07:24.699580 21069 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:07:23.970374 24545 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:24.045326 24226 consensus_meta.cc:287] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.124s	user 0.010s	sys 0.001s
I20260709 13:07:24.708455 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:24.486909 24536 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.356s	user 0.009s	sys 0.000s
W20260709 13:07:24.600961 24637 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.463s	user 0.001s	sys 0.000s
I20260709 13:07:23.974148 24275 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.710506 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:24.670156 24565 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.920s	user 0.010s	sys 0.000s
I20260709 13:07:24.031194 24447 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:24.670116 24611 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.976220 24459 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.759s
I20260709 13:07:24.283792 24472 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.467s
I20260709 13:07:24.309448 24026 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.717710 24226 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:24.725099 24530 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.980s	user 0.005s	sys 0.005s
I20260709 13:07:24.673264 24478 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.959951 24194 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.749s
I20260709 13:07:24.746282 24572 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:07:24.031071 24197 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:24.725276 24310 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:24.716894 24590 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:07:24.726624 24623 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:24.693969 24471 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:24.727545 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:24.728706 24596 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:24.867722 24596 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:24.870232 24596 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:24.730280 24549 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:24.713971 24553 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.138046 24281 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:24.871515 24553 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:24.030685 24562 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:24.872684 24562 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.874325 24553 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:24.492116 24615 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:24.874810 24562 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:24.875186 24615 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:24.875752 24615 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:24.309182 24610 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:24.879632 24610 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.744498 21066 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:24.882295 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 495) took 1099 ms. Trace:
I20260709 13:07:24.882556 24610 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:24.882776 21066 rpcz_store.cc:276] 0709 13:07:23.783037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.690139 (+907102us) service_pool.cc:224] Handling call
0709 13:07:24.882212 (+192073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:24.883764 21068 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:24.884160 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:24.715649 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1973) took 1143 ms. Trace:
I20260709 13:07:24.885226 21069 rpcz_store.cc:276] 0709 13:07:23.572359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.957140 (+384781us) service_pool.cc:224] Handling call
0709 13:07:24.715571 (+758431us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:24.886392 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:24.887151 21069 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:24.888305 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:24.716130 24571 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:24.889045 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1971) took 1366 ms. Trace:
I20260709 13:07:24.889268 24571 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:24.890127 24571 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:24.894477 24563 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.716495 24547 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.895090 24563 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.895557 24547 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.897236 24563 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:24.308177 24588 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.447s
W20260709 13:07:24.761370 21800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.013s	sys 0.060s
I20260709 13:07:24.670800 24604 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:24.898042 21800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.013s	sys 0.061s
I20260709 13:07:24.898429 24588 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:24.898833 24588 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:24.899919 24604 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:24.758994 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:24.900007 24588 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:24.900975 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 142083us
I20260709 13:07:24.901279 24588 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:24.902014 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46560 (request call id 11) took 1784 ms. Trace:
I20260709 13:07:24.902400 21216 rpcz_store.cc:276] 0709 13:07:23.145655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.145813 (+   158us) service_pool.cc:224] Handling call
0709 13:07:24.901836 (+1756023us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b540013b354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42516853 00007f7f474e460a 00007f7f40a6a7d4 00007f7f40a6a762 00007f7f40a6a691 00007f7f40a6a60c 00007f7f40a698c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:24.901917 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":723218176}
I20260709 13:07:24.745615 24536 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:24.747056 24275 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:24.740430 24453 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.907413 24562 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.907575 24453 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:24.907987 24562 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.908874 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:24.909902 24453 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:24.910152 24562 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:24.910359 24516 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:24.901298 24612 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:24.910866 21215 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:24.911211 24612 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:24.910880 24516 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.912158 24516 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:24.753767 24595 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:24.758249 24447 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.914311 21067 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:24.914510 24447 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:24.705958 24606 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:24.915763 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1041) took 1266 ms. Trace:
I20260709 13:07:24.916121 21067 rpcz_store.cc:276] 0709 13:07:23.649463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.727223 (+1077760us) service_pool.cc:224] Handling call
0709 13:07:24.915574 (+188351us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b540011c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:24.915704 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127572224}
I20260709 13:07:24.917227 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:24.917333 24606 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:24.813841 21038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.010s	sys 0.026s
I20260709 13:07:24.917888 21067 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
W20260709 13:07:24.918167 21038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.010s	sys 0.032s
W20260709 13:07:24.918376 24640 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.609s	user 0.000s	sys 0.011s
I20260709 13:07:24.761269 24318 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.919556 24318 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.921725 24318 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:24.763144 24565 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:24.731525 24545 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.923799 24592 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:24.923987 24545 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:24.689445 24632 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:24.843715 21221 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:24.781383 24445 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:24.925019 24445 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:24.925925 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 396) took 2725 ms. Trace:
I20260709 13:07:24.926172 24632 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:24.818428 24441 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.275s	user 0.006s	sys 0.006s
I20260709 13:07:24.926312 21220 rpcz_store.cc:276] 0709 13:07:22.200388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.329173 (+128785us) service_pool.cc:224] Handling call
0709 13:07:22.329793 (+   620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.330257 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:22.330547 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.202320 (+871773us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:23.203424 (+  1104us) log.cc:844] Serialized 508 byte log entry
0709 13:07:23.203560 (+   136us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:23.203592 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.203655 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:24.925716 (+1722061us) raft_consensus.cc:1685] finished
0709 13:07:24.925736 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:24.925866 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4999,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":145445,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":75,"thread_start_us":848179,"threads_started":2,"wal-append.queue_time_us":2463297}
I20260709 13:07:24.927466 24441 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:24.928006 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 485) took 1644 ms. Trace:
I20260709 13:07:24.928447 21070 rpcz_store.cc:276] 0709 13:07:23.291395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.306446 (+ 15051us) service_pool.cc:224] Handling call
0709 13:07:23.306724 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:24.660283 (+1353559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.927776 (+267493us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b54002ad5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:24.927945 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1667271040}
I20260709 13:07:24.929883 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:24.930047 24441 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:24.810813 24478 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:24.930474 21070 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:24.933179 24519 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:24.934361 24519 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:24.934775 24519 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:24.859441 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 490) took 1474 ms. Trace:
I20260709 13:07:24.843540 24590 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:24.935447 21063 rpcz_store.cc:276] 0709 13:07:23.384911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.385047 (+   136us) service_pool.cc:224] Handling call
0709 13:07:23.419876 (+ 34829us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:23.464991 (+ 45115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.465036 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:23.483345 (+ 18309us) log.cc:844] Serialized 26 byte log entry
0709 13:07:23.650784 (+167439us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:23.650832 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.650904 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:24.859059 (+1208155us) raft_consensus.cc:1685] finished
0709 13:07:24.859090 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:24.859310 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28690,"raft.queue_time_us":28836,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":13097,"thread_start_us":120498,"threads_started":1,"wal-append.queue_time_us":390459}
I20260709 13:07:24.843217 24310 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.842809 24594 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:24.860760 24627 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:07:24.871618 24281 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.673111 24613 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:24.959237 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:24.959904 21063 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:24.806216 24026 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:23.977535 24607 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.922019 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:24.924315 24592 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:24.935942 21798 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:24.931214 21070 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:07:24.889654 21072 rpcz_store.cc:276] 0709 13:07:23.522285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.563133 (+ 40848us) service_pool.cc:224] Handling call
0709 13:07:23.607633 (+ 44500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.645728 (+ 38095us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:07:23.646017 (+   289us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.973761 (+327744us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.974078 (+   317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.974145 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:23.974219 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.888730 (+914511us) spinlock_profiling.cc:243] Waited 37.9 ms on lock 0x7b540011add4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:24.888793 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:24.888948 (+   155us) inbound_call.cc:177] Queueing success response
0709 13:07:24.918085 (+ 29137us) spinlock_profiling.cc:243] Waited 94.4 ms on lock 0x7b240013a4c8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f2b59c4 00007f7f3f2b3acb 00007f7f3f46edc9 00007f7f3f46ea55 00007f7f3f46e09d 00007f7f420a010e 00007f7f420b788c 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221
Metrics: {"mutex_wait_us":79392,"prepare.queue_time_us":2066,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":13705,"raft.queue_time_us":45022,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"spinlock_wait_cycles":388179968,"thread_start_us":270757,"threads_started":1,"wal-append.queue_time_us":662780}
I20260709 13:07:24.951241 24604 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.949673 24310 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:24.918411 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1979) took 1313 ms. Trace:
I20260709 13:07:24.943334 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:24.970930 21067 rpcz_store.cc:276] 0709 13:07:23.605514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.916952 (+1311438us) service_pool.cc:224] Handling call
0709 13:07:24.918347 (+  1395us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:24.762708 24611 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:24.889058 21069 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:24.950986 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 718) took 2916 ms. Trace:
I20260709 13:07:24.948312 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:24.976182 21801 rpcz_store.cc:276] 0709 13:07:22.020338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.026751 (+  6413us) service_pool.cc:224] Handling call
0709 13:07:22.027190 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:24.693545 (+2666355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.936029 (+242484us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54004934d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:24.936259 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4857403776}
I20260709 13:07:24.976871 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:24.978477 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:24.946424 24590 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:24.897464 24547 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:24.912169 24626 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.942s	user 0.011s	sys 0.000s
I20260709 13:07:24.936005 24445 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:24.841621 24528 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.121s
I20260709 13:07:24.960813 24576 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:24.987082 21063 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:24.989008 21066 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:24.992501 21064 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:25.003854 21767 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:24.991413 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.005375 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.009845 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:24.980939 24632 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:24.979020 24459 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:25.011677 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.016296 21063 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:25.012626 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:24.989620 24226 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:24.989914 24565 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:24.989773 24318 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:25.019230 24197 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:25.022884 24318 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:25.024979 24197 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.025368 24318 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:25.025494 24197 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:25.016016 24459 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.027549 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 370) took 1237 ms. Trace:
I20260709 13:07:25.027639 24459 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:25.027590 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.019508 24226 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:24.989624 24447 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.029350 24226 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.029479 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.030450 21063 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:25.019337 24596 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:25.031383 24447 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:25.031589 24596 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.032040 24596 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:24.979816 24604 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.032466 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.033092 21065 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:25.033917 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.034502 21065 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:25.022258 24565 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.007251 24618 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:24.973003 24519 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.013895 23761 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:25.031405 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.042088 21063 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:25.035255 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.043915 21065 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:07:25.018930 24563 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:07:25.045866 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 1.07461e+06us
I20260709 13:07:25.050908 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.051715 21063 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:25.013618 24562 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:25.056167 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.056708 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1036) took 1408 ms. Trace:
I20260709 13:07:25.028057 21800 rpcz_store.cc:276] 0709 13:07:23.790394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.795467 (+  5073us) service_pool.cc:224] Handling call
0709 13:07:23.888347 (+ 92880us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:23.888707 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.888735 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:23.912697 (+ 23962us) log.cc:844] Serialized 26 byte log entry
0709 13:07:24.898487 (+985790us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:24.898541 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.898609 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.027310 (+128701us) raft_consensus.cc:1685] finished
0709 13:07:25.027332 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.027476 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48915,"raft.queue_time_us":439,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":8792,"thread_start_us":760066,"threads_started":1,"wal-append.queue_time_us":999033}
I20260709 13:07:25.056890 21072 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
W20260709 13:07:24.966738 24636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.014s	sys 0.050s
I20260709 13:07:25.057220 21065 rpcz_store.cc:276] 0709 13:07:23.648173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.035046 (+1386873us) service_pool.cc:224] Handling call
0709 13:07:25.056632 (+ 21586us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.083055 21064 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:25.059576 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:25.090698 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:25.084589 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.096772 21065 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:25.086113 21063 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:25.035758 24565 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:25.033871 24604 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:25.098949 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1980) took 1493 ms. Trace:
I20260709 13:07:25.099398 21068 rpcz_store.cc:276] 0709 13:07:23.605877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.891825 (+285948us) service_pool.cc:224] Handling call
0709 13:07:25.098797 (+1206972us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54003c9e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:25.098896 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508105088}
I20260709 13:07:25.103649 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.105262 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.110041 21070 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:07:25.110481 21070 pending_rounds.cc:85] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 4
I20260709 13:07:25.110692 21070 pending_rounds.cc:107] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:07:25.110908 21070 raft_consensus.cc:2889] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:25.111135 21065 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:07:25.111716 21065 pending_rounds.cc:85] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 5
I20260709 13:07:25.112138 21065 pending_rounds.cc:107] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:07:25.111794 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.102589 21067 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:25.112798 21065 raft_consensus.cc:2889] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:25.113320 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1038) took 1464 ms. Trace:
I20260709 13:07:25.113698 21067 rpcz_store.cc:276] 0709 13:07:23.648667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.101739 (+1453072us) service_pool.cc:224] Handling call
0709 13:07:25.102189 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.112991 (+ 10802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.113249 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:25.084349 24636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.014s	sys 0.051s
I20260709 13:07:25.051180 24563 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.115895 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.116966 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.120407 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:25.124126 21063 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:25.087952 24447 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.115947 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:25.146238 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1042) took 1496 ms. Trace:
I20260709 13:07:25.146628 21063 rpcz_store.cc:276] 0709 13:07:23.649705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.143415 (+1493710us) service_pool.cc:224] Handling call
0709 13:07:25.145810 (+  2395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.145960 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.145983 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.146019 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:25.146047 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.146096 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.146185 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.139078 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1037) took 1490 ms. Trace:
I20260709 13:07:25.059130 24562 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:25.139964 24549 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:25.059695 23761 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:25.144018 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:25.142416 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.148859 21070 rpcz_store.cc:276] 0709 13:07:23.648422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.109434 (+1461012us) service_pool.cc:224] Handling call
0709 13:07:25.109711 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.138806 (+ 29095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.139001 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.152729 21067 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:25.102948 24589 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:25.157085 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1040) took 1507 ms. Trace:
I20260709 13:07:25.156658 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1974) took 1553 ms. Trace:
I20260709 13:07:25.089020 24519 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:25.156343 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.143862 24563 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:25.161418 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.156339 21767 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:25.157424 21067 rpcz_store.cc:276] 0709 13:07:23.649193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.142666 (+1493473us) service_pool.cc:224] Handling call
0709 13:07:25.152416 (+  9750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.156833 (+  4417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.157030 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:25.171265 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 481239us
I20260709 13:07:25.173070 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:25.157953 21065 rpcz_store.cc:276] 0709 13:07:23.603660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.110300 (+1506640us) service_pool.cc:224] Handling call
0709 13:07:25.110701 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.156416 (+ 45715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.156604 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.137391 24604 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:25.137548 24618 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:25.140165 24606 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:07:25.179685 21139 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.009s	sys 0.007s
W20260709 13:07:25.157668 24643 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.167s	user 0.001s	sys 0.001s
W20260709 13:07:25.207041 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45686.8us
I20260709 13:07:25.159902 24549 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.155239 24562 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:25.211867 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:25.207950 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.000s	sys 0.018s
I20260709 13:07:25.102550 24593 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.216796 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1044) took 1566 ms. Trace:
I20260709 13:07:25.219842 21072 rpcz_store.cc:276] 0709 13:07:23.650215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.176071 (+1525856us) service_pool.cc:224] Handling call
0709 13:07:25.216271 (+ 40200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.216452 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.216479 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.216537 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:25.216579 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.216632 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.216737 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.151991 24584 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:07:25.222847 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 484) took 1960 ms. Trace:
I20260709 13:07:25.224086 24584 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:25.224615 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.224841 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:25.224716 24584 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.225139 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:25.225521 21792 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:25.226461 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:25.227171 21796 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
W20260709 13:07:25.215400 21110 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.012s	sys 0.006s
I20260709 13:07:25.159384 24548 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:07:25.233480 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:25.234064 21216 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:25.151801 24612 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
W20260709 13:07:25.228113 21110 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.012s	sys 0.007s
W20260709 13:07:25.241971 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 583031us
I20260709 13:07:25.194752 24441 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:07:25.228296 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 506162us
I20260709 13:07:25.215606 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:25.193383 24604 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.151527 24588 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:25.163453 24472 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:25.229228 24645 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:07:25.193706 24618 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.248611 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:25.255568 21112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.001s	sys 0.018s
I20260709 13:07:25.299566 21214 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
W20260709 13:07:25.299664 21112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.001s	sys 0.018s
I20260709 13:07:25.228765 24584 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:25.085492 24590 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.267863 21072 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:25.159641 24576 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:07:25.046666 24592 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:25.215087 24606 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:25.213482 24536 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:25.215355 24460 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:25.217793 24549 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:25.160046 24447 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.269428 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:25.232084 21220 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:25.345366 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 408) took 1387 ms. Trace:
I20260709 13:07:25.289933 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:25.221251 24247 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:25.234753 24642 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:07:25.297060 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:25.215346 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.000s	sys 0.019s
W20260709 13:07:25.357969 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 666164us
I20260709 13:07:25.315441 24441 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:25.298638 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 83099us
I20260709 13:07:25.306911 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1048) took 1633 ms. Trace:
I20260709 13:07:25.308800 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:25.362489 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.309571 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:25.209762 24553 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:07:25.311743 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2152) took 2254 ms. Trace:
I20260709 13:07:25.316423 21143 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:25.323416 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 723) took 1825 ms. Trace:
I20260709 13:07:25.229420 24594 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:25.315980 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1045) took 1643 ms. Trace:
W20260709 13:07:25.327039 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 115127us
I20260709 13:07:25.261715 24281 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:25.266408 24604 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:25.327744 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:25.266952 24588 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.348335 24590 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:25.263155 24607 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:25.298077 24472 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.339964 21069 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:25.341676 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:25.298419 24471 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:25.345793 21218 rpcz_store.cc:276] 0709 13:07:23.957982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.702649 (+744667us) service_pool.cc:224] Handling call
0709 13:07:24.702816 (+   167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:24.702947 (+   131us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:24.738298 (+ 35351us) log.cc:844] Serialized 14 byte log entry
0709 13:07:25.344998 (+606700us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.345053 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:25.345100 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.345175 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.345312 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2573,"prepare.queue_time_us":43431,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"thread_start_us":606284,"threads_started":2,"wal-append.queue_time_us":595734}
I20260709 13:07:25.446802 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:25.312415 24618 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:25.321732 24589 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:25.350627 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 403) took 1797 ms. Trace:
I20260709 13:07:25.349527 21216 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:25.402204 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:25.341065 24318 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.331503 24611 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:25.362979 21067 rpcz_store.cc:276] 0709 13:07:23.673561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.305739 (+1632178us) service_pool.cc:224] Handling call
0709 13:07:25.306219 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.306451 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.306491 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.306549 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:25.306596 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.306669 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.306830 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.461498 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.462203 21067 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:25.323747 23761 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.407174 21142 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:25.413292 21072 rpcz_store.cc:276] 0709 13:07:23.672603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.220840 (+1548237us) service_pool.cc:224] Handling call
0709 13:07:25.267418 (+ 46578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.315605 (+ 48187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.315902 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.412973 21798 rpcz_store.cc:276] 0709 13:07:23.483383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.483744 (+   361us) service_pool.cc:224] Handling call
0709 13:07:25.308539 (+1824795us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54006c1e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:25.308672 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3001343872}
I20260709 13:07:25.414265 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:25.480250 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 66231.8us
I20260709 13:07:25.416369 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:25.432858 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 400) took 2179 ms. Trace:
I20260709 13:07:25.411237 21139 rpcz_store.cc:276] 0709 13:07:23.057082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.057226 (+   144us) service_pool.cc:224] Handling call
0709 13:07:23.085641 (+ 28415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.448054 (+362413us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:23.585291 (+137237us) log.cc:844] Serialized 14 byte log entry
0709 13:07:25.180172 (+1594881us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.180232 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:25.180281 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.311413 (+131132us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54004240d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:25.311481 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.311665 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45943,"prepare.queue_time_us":182930,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":736849,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":6256,"spinlock_wait_cycles":831751040,"thread_start_us":875077,"threads_started":3,"wal-append.queue_time_us":1296424}
I20260709 13:07:25.481220 21144 raft_consensus.cc:606] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:25.482251 21144 raft_consensus.cc:614] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:25.433223 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:25.441942 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1047) took 1768 ms. Trace:
I20260709 13:07:25.481192 21219 rpcz_store.cc:276] 0709 13:07:23.253734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.254232 (+   498us) service_pool.cc:224] Handling call
0709 13:07:23.290609 (+ 36377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:24.738423 (+1447814us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:25.160789 (+422366us) log.cc:844] Serialized 14 byte log entry
0709 13:07:25.431860 (+271071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.431933 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:25.432003 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.432095 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.432792 (+   697us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16792,"prepare.queue_time_us":74432,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":580,"raft.queue_time_us":130,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":6003,"thread_start_us":566838,"threads_started":3}
I20260709 13:07:25.483835 21069 rpcz_store.cc:276] 0709 13:07:23.673259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.289992 (+1616733us) service_pool.cc:224] Handling call
0709 13:07:25.328139 (+ 38147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.441615 (+113476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.441874 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.441964 21138 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:25.451581 21145 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:25.452219 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:25.485962 21216 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:25.404560 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:25.451261 21213 rpcz_store.cc:276] 0709 13:07:23.552723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.787969 (+235246us) service_pool.cc:224] Handling call
0709 13:07:23.849746 (+ 61777us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:23.873015 (+ 23269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.873056 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:23.905492 (+ 32436us) log.cc:844] Serialized 26 byte log entry
0709 13:07:24.354597 (+449105us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:24.354646 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.354729 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.350390 (+995661us) raft_consensus.cc:1685] finished
0709 13:07:25.350414 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.350550 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112743,"raft.queue_time_us":34564,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":9300,"thread_start_us":448749,"threads_started":1,"wal-append.queue_time_us":937399}
I20260709 13:07:25.325090 24584 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:25.441622 24281 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:25.439678 24588 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:25.471048 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:25.450743 24544 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:07:25.491988 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:25.447921 24472 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:25.474704 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1035) took 1826 ms. Trace:
W20260709 13:07:25.550701 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.402590 24516 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:25.577862 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 106928us
I20260709 13:07:25.450786 24658 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:25.439177 24594 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:25.586148 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 181796us
I20260709 13:07:25.583895 21068 rpcz_store.cc:276] 0709 13:07:23.647871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.114453 (+1466582us) service_pool.cc:224] Handling call
0709 13:07:25.114781 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:25.138376 (+ 23595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.138407 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:25.162559 (+ 24152us) log.cc:844] Serialized 26 byte log entry
0709 13:07:25.435225 (+272666us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:25.435416 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.439578 (+  4162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.474355 (+ 34777us) raft_consensus.cc:1685] finished
0709 13:07:25.474386 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.474613 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23142,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":9338,"thread_start_us":187568,"threads_started":1,"wal-append.queue_time_us":253462}
I20260709 13:07:25.544999 24281 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:25.588400 24655 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:07:25.490275 24247 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
W20260709 13:07:25.598832 24197 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.573s	user 0.005s	sys 0.001s
I20260709 13:07:25.600117 24654 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:25.602463 24557 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.783s	user 0.008s	sys 0.000s
I20260709 13:07:25.661876 24663 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:25.662539 24571 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.772s	user 0.012s	sys 0.001s
I20260709 13:07:25.662982 24571 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.663467 24663 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:25.664952 24571 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:25.665338 24604 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.579631 24664 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:25.666044 24596 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.634s	user 0.008s	sys 0.002s
I20260709 13:07:25.666182 24664 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:25.666677 24478 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.736s	user 0.008s	sys 0.001s
W20260709 13:07:25.606674 24459 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.000s
I20260709 13:07:25.667220 24478 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.667363 24459 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.669555 24478 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:25.669683 24654 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:25.604578 24593 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:25.606174 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
W20260709 13:07:25.607949 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 116083us
I20260709 13:07:25.578562 24584 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:25.621335 24275 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.000s
I20260709 13:07:25.625593 24662 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.623941 21138 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
W20260709 13:07:25.629086 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 364370us
W20260709 13:07:25.652769 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 383420us
I20260709 13:07:25.653895 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:25.655710 21068 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:25.606226 24608 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.640s	user 0.006s	sys 0.004s
I20260709 13:07:25.613096 24516 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:25.607517 24447 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:25.660310 24668 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:07:25.660894 24318 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:25.661182 24659 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:07:25.583638 24441 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.666641 24664 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:25.451701 24550 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 13:07:25.666507 24596 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.607491 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 491) took 1825 ms. Trace:
I20260709 13:07:25.622356 24658 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:25.677729 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 254132us
I20260709 13:07:25.678193 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1081) took 1788 ms. Trace:
I20260709 13:07:25.447005 24612 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.134s
I20260709 13:07:25.641268 24281 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:25.669569 24613 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.710s	user 0.007s	sys 0.004s
I20260709 13:07:25.697041 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 492) took 1915 ms. Trace:
I20260709 13:07:25.492416 24606 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.669703 24459 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:25.665727 24571 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:25.658421 24197 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.676117 24587 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:25.938817 21072 rpcz_store.cc:276] 0709 13:07:23.781744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.579109 (+1797365us) service_pool.cc:224] Handling call
0709 13:07:25.606974 (+ 27865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.607158 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.607183 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.607222 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:25.607260 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.607335 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.607433 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.674532 24611 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:25.904366 21215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1078) took 2015 ms (client timeout 1507 ms). Trace:
I20260709 13:07:25.935763 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:25.957428 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 274216us
W20260709 13:07:25.958438 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.947387 21068 rpcz_store.cc:276] 0709 13:07:23.782120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.640414 (+1858294us) service_pool.cc:224] Handling call
0709 13:07:25.640830 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.696768 (+ 55938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.696980 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:25.937206 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 283772us
W20260709 13:07:25.956346 21215 rpcz_store.cc:269] 0709 13:07:23.889200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.896258 (+  7058us) service_pool.cc:224] Handling call
0709 13:07:25.904147 (+2007889us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54001257d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:25.904281 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555754368}
I20260709 13:07:25.960755 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.938578 21212 rpcz_store.cc:276] 0709 13:07:23.890114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.891974 (+  1860us) service_pool.cc:224] Handling call
0709 13:07:23.909924 (+ 17950us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:24.193046 (+283122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:24.193095 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:24.912990 (+719895us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b2c0011d108. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:07:24.913411 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 13:07:25.296295 (+382884us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:25.296352 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.296453 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.672259 (+375806us) raft_consensus.cc:1685] finished
0709 13:07:25.672278 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.678108 (+  5830us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120928,"raft.queue_time_us":274240,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":46456,"spinlock_wait_cycles":10533632,"thread_start_us":382421,"threads_started":1,"wal-append.queue_time_us":746440}
I20260709 13:07:25.698769 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:25.963109 21213 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:25.961736 21068 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:25.964097 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:25.857812 24670 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:07:25.968585 24611 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:25.970927 24611 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:25.910009 24584 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:25.823930 24615 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.948s	user 0.005s	sys 0.007s
I20260709 13:07:25.971527 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:25.971917 24615 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.958441 24587 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:25.974143 24587 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:25.974535 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:25.975704 24669 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:07:25.958575 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:25.723734 24565 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.581s	user 0.003s	sys 0.007s
W20260709 13:07:25.841044 24630 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.578s	user 0.004s	sys 0.012s
W20260709 13:07:25.982515 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.709409 24676 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.991313 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:25.991454 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:25.994243 21068 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:07:25.787072 24593 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:25.997366 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:25.710767 24607 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:26.001928 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:25.696847 24545 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.788304 24656 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:25.947507 24571 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:26.038563 24571 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:25.223516 21071 rpcz_store.cc:276] 0709 13:07:23.281090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.281469 (+   379us) service_pool.cc:224] Handling call
0709 13:07:23.321189 (+ 39720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.321429 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:23.343491 (+ 22062us) log.cc:844] Serialized 14 byte log entry
0709 13:07:24.860545 (+1517054us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b18000bdac0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:07:25.222102 (+361557us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.222437 (+   335us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:25.222506 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.222601 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.222767 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27992,"prepare.queue_time_us":21896,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":108,"spinlock_wait_cycles":14903680,"thread_start_us":361277,"threads_started":1}
I20260709 13:07:26.039551 24588 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.040478 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 493) took 2258 ms. Trace:
I20260709 13:07:26.040858 21071 rpcz_store.cc:276] 0709 13:07:23.782432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.039633 (+2257201us) service_pool.cc:224] Handling call
0709 13:07:26.039966 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.040141 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.040166 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.040207 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:26.040237 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.040288 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.040424 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:26.041946 24588 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:26.042523 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 498) took 2258 ms. Trace:
I20260709 13:07:26.042644 24588 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:25.709630 24608 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:25.861135 24604 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:25.907508 24654 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:25.953473 24658 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:25.990710 24318 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.020779 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:25.996892 24611 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:26.021965 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 59508.1us
I20260709 13:07:26.042896 21071 rpcz_store.cc:276] 0709 13:07:23.783953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.041752 (+2257799us) service_pool.cc:224] Handling call
0709 13:07:26.042056 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.042216 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.042239 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.042278 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:26.042307 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.042367 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.042474 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:26.026117 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:25.692665 24275 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.215250 21070 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:26.215672 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:26.216133 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:26.216727 21070 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:26.217764 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:26.218292 21070 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:07:26.057924 24673 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.219265 24673 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.221341 24673 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:26.221879 24673 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:26.222352 24673 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.222536 24571 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.043112 24588 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.023388 24672 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:26.223673 24588 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:25.955561 24613 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.225528 24613 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:26.226091 24318 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:26.211902 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 134626us
I20260709 13:07:26.009173 24593 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:26.007839 24544 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:25.697934 24662 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.225821 21070 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
W20260709 13:07:26.049569 24590 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.605s	user 0.008s	sys 0.001s
I20260709 13:07:26.227581 21217 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:26.230819 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:26.067405 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:26.251947 21798 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:26.252205 21219 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:26.043646 21214 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
W20260709 13:07:26.047611 24618 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.003s
W20260709 13:07:26.109061 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.043174 24519 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.880s	user 0.002s	sys 0.005s
W20260709 13:07:26.053201 21066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 494) took 2270 ms (client timeout 2174 ms). Trace:
I20260709 13:07:26.255796 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:26.256037 21066 rpcz_store.cc:269] 0709 13:07:23.782732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.883809 (+1101077us) service_pool.cc:224] Handling call
0709 13:07:26.052996 (+1169187us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540019dcd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:26.053124 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239123840}
W20260709 13:07:26.241062 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 243799us
I20260709 13:07:26.007432 24565 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.056764 24594 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:26.060938 24026 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:26.035530 24656 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:26.249612 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:26.272468 21070 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:26.229724 21072 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:26.203413 24630 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 13:07:26.053735 24563 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.847s	user 0.007s	sys 0.002s
I20260709 13:07:26.093144 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:26.066212 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:26.083251 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:26.211155 24557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.047s	sys 0.061s
W20260709 13:07:26.259402 21067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.019s
W20260709 13:07:26.284466 21067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.000s	sys 0.019s
I20260709 13:07:26.061590 24596 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:26.300109 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:26.225555 24592 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:26.264191 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 300213us
I20260709 13:07:26.300899 24563 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.302057 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:26.300877 21794 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
W20260709 13:07:26.303735 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 237636us
I20260709 13:07:26.251235 21071 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 3 to 7, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:26.266083 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:26.306998 21796 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:26.176517 24670 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:26.267335 21801 raft_consensus.cc:2410] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:07:26.240490 21767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 2.174s (SENT) [suppressed 2 similar messages]
I20260709 13:07:26.270004 21212 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:26.313668 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 73216.4us
I20260709 13:07:26.313812 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:26.244274 24544 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.012241 24676 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.282373 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:26.292637 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:26.299690 21793 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:07:26.299963 24557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.048s	sys 0.061s
I20260709 13:07:26.285455 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1050) took 2535 ms. Trace:
I20260709 13:07:26.308132 21216 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:26.303258 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:26.147934 24587 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.273006 24656 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:26.148301 24545 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.266017 24613 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:26.318033 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:26.263798 24604 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:26.318984 21070 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:26.364524 21070 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:07:26.364893 21070 pending_rounds.cc:85] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 5
I20260709 13:07:26.365110 21070 pending_rounds.cc:107] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:07:26.365373 21070 raft_consensus.cc:2889] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:26.365725 21072 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:26.322176 21215 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:07:26.283205 21065 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:26.271355 24672 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:07:26.340520 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:26.336588 21067 rpcz_store.cc:276] 0709 13:07:23.750145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.463503 (+1713358us) service_pool.cc:224] Handling call
0709 13:07:25.463792 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.463973 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:07:25.693943 (+229970us) log.cc:844] Serialized 14 byte log entry
0709 13:07:26.284985 (+591042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.285056 (+    71us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:26.285123 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.285211 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.285374 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":543114,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":1075,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":820249,"threads_started":2}
I20260709 13:07:26.346581 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:26.243432 24593 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:26.312052 24576 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:07:26.317224 24524 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.029s	sys 0.033s
I20260709 13:07:26.303262 24563 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:26.355379 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 114348us
I20260709 13:07:26.261540 24683 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:07:26.321197 24544 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:26.262858 24618 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.365990 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1981) took 1702 ms. Trace:
I20260709 13:07:26.431478 21070 rpcz_store.cc:276] 0709 13:07:24.664160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.363862 (+1699702us) service_pool.cc:224] Handling call
0709 13:07:26.364166 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.365765 (+  1599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.365938 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:26.367741 21065 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:26.300828 24026 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:26.324731 24275 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:26.316797 24594 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:07:26.339110 24611 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:26.353681 24596 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:07:26.415556 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 75131.8us
I20260709 13:07:26.421347 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1057) took 1517 ms. Trace:
I20260709 13:07:26.414484 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1059) took 1510 ms. Trace:
W20260709 13:07:26.429420 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.359925 24318 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:26.318071 21219 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:26.432647 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1055) took 1719 ms. Trace:
W20260709 13:07:26.414222 24524 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.030s	sys 0.034s
I20260709 13:07:26.435048 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 500) took 1525 ms. Trace:
I20260709 13:07:26.494590 21070 rpcz_store.cc:276] 0709 13:07:24.909208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.433945 (+1524737us) service_pool.cc:224] Handling call
0709 13:07:26.434578 (+   633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.434746 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.434774 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.434816 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:26.434850 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.434903 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.434997 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:26.363910 24688 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:26.434238 24690 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:26.447176 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:26.322855 24670 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:26.436776 24549 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.076s	user 0.011s	sys 0.002s
I20260709 13:07:26.359155 24519 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:07:26.488767 21065 rpcz_store.cc:276] 0709 13:07:24.712828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.367121 (+1654293us) service_pool.cc:224] Handling call
0709 13:07:26.367448 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.432349 (+ 64901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.432585 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:26.454382 24611 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.490536 24589 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:26.489223 21067 rpcz_store.cc:276] 0709 13:07:24.904291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.392271 (+1487980us) service_pool.cc:224] Handling call
0709 13:07:26.413356 (+ 21085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.413901 (+   545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.413967 (+    66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.414051 (+    84us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:26.414134 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.414236 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.414397 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:26.435118 24557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48444 (request call id 8) took 8833 ms. Trace:
I20260709 13:07:26.491423 21072 rpcz_store.cc:276] 0709 13:07:24.903783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.367192 (+1463409us) service_pool.cc:224] Handling call
0709 13:07:26.420681 (+ 53489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.420921 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.420958 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.421014 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:26.421067 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.421138 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.421279 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:26.492681 24472 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.909s	user 0.012s	sys 0.000s
I20260709 13:07:26.323611 24691 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:07:26.439539 24563 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:26.513252 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1051) took 2625 ms. Trace:
I20260709 13:07:26.515631 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1983) took 1554 ms. Trace:
W20260709 13:07:26.485008 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 128472us
W20260709 13:07:26.495733 24562 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.234s	user 0.013s	sys 0.004s
I20260709 13:07:26.530746 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:26.517666 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:26.503782 24318 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:26.491957 24596 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:26.517822 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:26.493647 24604 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:26.530927 21065 rpcz_store.cc:276] 0709 13:07:24.961062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.514747 (+1553685us) service_pool.cc:224] Handling call
0709 13:07:26.515070 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.515248 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.515332 (+    84us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.515382 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:26.515422 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.515472 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.515575 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:26.541616 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:26.519357 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.531374 21767 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:26.536206 21795 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:07:26.537027 21068 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.002s
I20260709 13:07:26.549307 21068 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:26.549998 21070 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:26.550554 21068 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:07:26.530815 21066 rpcz_store.cc:276] 0709 13:07:23.887511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.264049 (+2376538us) service_pool.cc:224] Handling call
0709 13:07:26.264394 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.264600 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:26.287214 (+ 22614us) log.cc:844] Serialized 14 byte log entry
0709 13:07:26.364067 (+ 76853us) log.cc:844] Serialized 14 byte log entry
0709 13:07:26.512746 (+148679us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.512828 (+    82us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:26.512909 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.513004 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.513172 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29088,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":61876,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":20448,"spinlock_wait_cycles":99843200,"thread_start_us":224610,"threads_started":2,"wal-append.queue_time_us":198748}
W20260709 13:07:26.539495 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 192997us
I20260709 13:07:26.514307 24590 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:26.514312 24673 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:26.522421 24611 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:26.526836 24639 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.820s	user 0.001s	sys 0.003s
I20260709 13:07:26.485141 24026 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.519025 24589 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:26.538568 24318 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.576177 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:26.518884 24549 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.541810 24596 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:26.554894 24565 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:26.540446 24694 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:07:26.557507 24571 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:26.537343 24472 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.590178 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:26.516279 24690 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.553244 24604 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.540763 24562 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.541616 24563 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.735915 24688 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.521148 24557 rpcz_store.cc:276] 0709 13:07:17.572039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.572975 (+   936us) service_pool.cc:224] Handling call
0709 13:07:17.948995 (+376020us) log.cc:844] Serialized 508 byte log entry
0709 13:07:19.529082 (+1580087us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54005d5f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42522735 00007f7f4251a9fa 00007f7f474e24c9 00007f7f40a5ec54 00007f7f40a5ebe2 00007f7f40a5eb11 00007f7f40a5ea8c 00007f7f40a5dd42 00007f7f4085748c 00007f7f408568c6
0709 13:07:26.370949 (+6841867us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3028,"raft.run_cpu_time_us":1085,"raft.run_wall_time_us":1085,"spinlock_wait_cycles":426102656}
I20260709 13:07:26.959419 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:26.971534 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 381452us
I20260709 13:07:26.972409 24528 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:26.972508 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:26.605786 24689 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:07:26.958421 24706 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.568399 24662 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:26.972990 24528 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:26.974061 24706 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.727142 24618 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:26.974583 24528 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:26.652873 24544 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:26.812227 24638 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 2.528s	user 0.002s	sys 0.015s
W20260709 13:07:26.606829 24692 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:07:26.595764 24318 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:26.626981 24194 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:26.731671 24545 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:26.590050 24607 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.001s
I20260709 13:07:26.776314 24576 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:26.979635 21218 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:26.644577 24664 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.000s
I20260709 13:07:26.980739 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:26.981006 21144 raft_consensus.cc:606] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:26.791484 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:26.647575 24698 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:07:26.980747 24706 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:26.964522 24670 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:26.982812 24549 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:26.651055 24281 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.668s	user 0.004s	sys 0.004s
I20260709 13:07:26.983440 24472 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:26.728924 21221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1082) took 2838 ms (client timeout 1648 ms). Trace:
I20260709 13:07:26.969743 24688 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:26.597309 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:26.985126 21064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1058) took 2081 ms (client timeout 2190 ms). Trace:
W20260709 13:07:26.634428 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.596009 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:26.626806 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:26.623039 21796 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:26.993120 24544 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:26.959929 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 503) took 1903 ms. Trace:
W20260709 13:07:26.996759 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.646808 24700 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:26.606280 24588 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:26.826645 24592 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.000s
I20260709 13:07:26.958951 24310 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:26.604099 24685 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:07:26.591454 24589 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:26.576418 23761 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:26.726447 24704 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 13:07:27.005151 21071 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.001s
W20260709 13:07:27.153437 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.195443 21066 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:27.194792 21221 rpcz_store.cc:269] 0709 13:07:23.890411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.890538 (+   127us) service_pool.cc:224] Handling call
0709 13:07:26.728716 (+2838178us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54004443d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:26.728847 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401958144}
I20260709 13:07:27.192862 21144 raft_consensus.cc:614] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:27.072096 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 72957us
W20260709 13:07:27.193550 21064 rpcz_store.cc:269] 0709 13:07:24.904056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.921774 (+ 17718us) service_pool.cc:224] Handling call
0709 13:07:26.984958 (+2063184us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b540026eb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:26.985072 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59682048}
W20260709 13:07:27.029378 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.015520 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1064) took 1340 ms. Trace:
I20260709 13:07:27.197692 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.198587 21064 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:27.193434 21067 rpcz_store.cc:276] 0709 13:07:25.056366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.521816 (+1465450us) service_pool.cc:224] Handling call
0709 13:07:26.532896 (+ 11080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.533086 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:26.689891 (+156805us) log.cc:844] Serialized 14 byte log entry
0709 13:07:26.959548 (+269657us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.959599 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:26.959648 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.959720 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.959864 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":425899,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":68,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":403697,"threads_started":2,"wal-append.queue_time_us":265804}
I20260709 13:07:27.193786 21216 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:26.724170 24587 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:26.965271 24197 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:26.820139 24611 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.043728 24607 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:27.090413 24596 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.043530 24194 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:27.219384 24194 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.220031 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.221325 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.222131 21067 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:27.222769 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.222904 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.208736 24673 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:27.133982 24658 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.886s	user 0.008s	sys 0.005s
I20260709 13:07:27.227952 24658 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.227921 24673 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:27.230194 24658 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.164089 24563 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:27.056617 24593 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.638s	user 0.007s	sys 0.004s
I20260709 13:07:27.213304 24441 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:27.066799 24711 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.223537 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.224051 24608 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.163910 24594 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.225169 24596 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.116400 24604 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.227525 21067 raft_consensus.cc:2368] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:07:27.197705 21068 rpcz_store.cc:276] 0709 13:07:25.675220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.551493 (+876273us) service_pool.cc:224] Handling call
0709 13:07:26.552437 (+   944us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:26.569424 (+ 16987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.569458 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:26.586718 (+ 17260us) log.cc:844] Serialized 26 byte log entry
0709 13:07:26.930252 (+343534us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:26.930310 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.930424 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:27.015206 (+ 84782us) raft_consensus.cc:1685] finished
0709 13:07:27.015233 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.015450 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5971,"raft.queue_time_us":29828,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5253,"thread_start_us":270181,"threads_started":1,"wal-append.queue_time_us":377567}
I20260709 13:07:27.230774 24673 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.210178 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.240095 21216 raft_consensus.cc:2368] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:27.240185 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.209478 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:27.240921 21068 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:27.241448 21212 raft_consensus.cc:2368] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:27.231767 21221 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:27.224807 24676 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.234162 21064 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:27.233579 21066 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:27.208288 24576 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:27.088624 24690 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.117796 24562 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.013970 24709 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:27.195298 23761 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.730902 24656 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.017813 24622 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:27.244253 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.247191 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.248474 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:27.235651 24557 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:27.251600 21066 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:27.207067 24281 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.234838 24310 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:27.254603 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.252357 21212 raft_consensus.cc:2368] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:27.256795 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.261956 21067 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.087805 20954 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0d78b35ac92d402db8a5e3fe45be52b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 13:07:27.196565 24664 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.267434 21213 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.266566 21064 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:27.250854 23761 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.235956 24593 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.268991 21220 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.274260 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.239838 24594 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.253752 24557 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:27.272224 20954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 184619us
I20260709 13:07:27.208118 24700 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.237736 24711 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.278911 21213 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.281272 21216 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.279521 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.281921 21212 raft_consensus.cc:2368] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 9.
I20260709 13:07:27.282866 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.279917 21184 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:27.280350 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46926.6us
I20260709 13:07:27.281040 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.285794 21066 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:27.285681 21068 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:27.287364 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.287914 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.288661 21068 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:27.288695 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:27.289961 24708 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:07:27.290249 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:27.295792 24693 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.635s	user 0.012s	sys 0.000s
I20260709 13:07:27.297472 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:27.298277 21212 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.299583 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.300794 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.300868 24656 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.301553 21066 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:27.302405 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.303066 21066 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:27.303819 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.304359 21066 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:27.305843 21212 raft_consensus.cc:2410] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:07:27.307041 21212 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:27.251802 24709 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.309945 24709 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.310228 21067 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:27.290062 24690 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:27.259720 24622 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.316394 21212 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:07:27.311275 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.321388 21067 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:27.323274 21068 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:07:27.331586 24557 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.332939 24557 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.333886 21067 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:27.334810 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 507) took 1120 ms. Trace:
I20260709 13:07:27.288070 24562 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.335215 21067 rpcz_store.cc:276] 0709 13:07:26.213960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.324158 (+1110198us) service_pool.cc:224] Handling call
0709 13:07:27.333656 (+  9498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.334475 (+   819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.334734 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:27.336421 24664 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.337886 24593 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.338871 24700 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.340001 24688 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.341042 24318 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.342314 21215 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.343770 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 506) took 1130 ms. Trace:
I20260709 13:07:27.344129 21068 rpcz_store.cc:276] 0709 13:07:26.213709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.317913 (+1104204us) service_pool.cc:224] Handling call
0709 13:07:27.322982 (+  5069us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.343511 (+ 20529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.343714 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:27.336107 24562 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.328063 21212 raft_consensus.cc:2955] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 3 to 7, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:07:27.346053 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 162237us
I20260709 13:07:27.349228 21068 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:27.351747 21066 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:27.271692 24310 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.358935 24588 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.359886 21215 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:27.361133 21215 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 13:07:27.363870 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.344134 24576 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.365154 21215 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:27.365794 21064 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:27.367420 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:27.348971 24690 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:27.361040 24562 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.373416 21215 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:27.379505 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 95545.4us
I20260709 13:07:27.380535 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:27.381361 21800 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.383555 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.387131 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.387910 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.401700 21215 pending_rounds.cc:85] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 7
I20260709 13:07:27.402237 21215 pending_rounds.cc:107] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:07:27.402573 21215 raft_consensus.cc:2889] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:27.406555 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.411633 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:27.412292 21797 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:27.412526 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.410987 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.375463 24576 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.421324 24516 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:27.433347 21797 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:27.393891 24528 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:07:27.393687 24602 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:07:27.441197 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:27.509552 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:27.509756 21794 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:27.461756 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57651.1us
I20260709 13:07:27.489396 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 416) took 1127 ms. Trace:
I20260709 13:07:27.479678 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:27.511569 21212 rpcz_store.cc:276] 0709 13:07:26.361641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.327242 (+965601us) service_pool.cc:224] Handling call
0709 13:07:27.327650 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.327917 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:27.370391 (+ 42474us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.489046 (+118655us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.489096 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:27.489144 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.489212 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.489328 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85529,"raft.run_wall_time_us":4309,"thread_start_us":47336,"threads_started":1,"wal-append.queue_time_us":34531}
I20260709 13:07:27.498693 21800 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:27.515683 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:27.505432 24732 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:07:27.487645 24622 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.431728 24516 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:27.520330 24516 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.520912 24602 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:27.521370 24602 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:27.521885 24516 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:27.522287 24602 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.523386 24602 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:27.525156 24026 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.527462 24576 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:27.545533 23761 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.532102 24588 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:27.530844 24281 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:27.559093 21219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 412) took 1860 ms (client timeout 1956 ms). Trace:
I20260709 13:07:27.483062 24589 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:27.561522 21219 rpcz_store.cc:269] 0709 13:07:25.698346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.698480 (+   134us) service_pool.cc:224] Handling call
0709 13:07:27.558926 (+1860446us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5400104354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:27.559042 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665054464}
I20260709 13:07:27.566062 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:27.566911 21796 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:27.559978 24576 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:27.434615 24690 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.571071 24576 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.576823 24711 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.579077 24310 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.565076 24588 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:27.580585 24588 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:27.583246 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.583719 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.547585 24594 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.585117 24589 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.568066 24318 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.610456 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 398) took 2359 ms. Trace:
I20260709 13:07:27.610918 21141 rpcz_store.cc:276] 0709 13:07:25.250678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.252391 (+  1713us) service_pool.cc:224] Handling call
0709 13:07:25.297606 (+ 45215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.297850 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:25.552564 (+254714us) log.cc:844] Serialized 14 byte log entry
0709 13:07:25.741216 (+188652us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.609962 (+1868746us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b18000d2280. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287 00007f7f42537208
0709 13:07:27.610113 (+   151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.610166 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:27.610214 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.610288 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.610394 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1874016,"prepare.queue_time_us":153997,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":7883,"raft.queue_time_us":1885254,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":3392,"spinlock_wait_cycles":5263744,"thread_start_us":389987,"threads_started":2,"wal-append.queue_time_us":506939,"wal-append.run_cpu_time_us":1744,"wal-append.run_wall_time_us":1001834}
I20260709 13:07:27.617764 24584 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.618355 24584 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.620862 24584 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.622547 21065 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:07:27.623889 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.624542 21066 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.626691 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:27.631472 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.633560 21066 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:27.634732 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.635627 21066 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.627841 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.638749 21065 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.640466 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.641089 21065 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.643000 21063 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:27.661470 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.668663 21066 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:27.667490 21065 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:27.671067 21767 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:27.671942 24528 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:27.672331 24528 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.672715 24528 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:27.677279 21767 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:27.679123 24602 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:27.679888 24602 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:27.681016 24602 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.688133 24663 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.687937 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:27.688719 24663 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.689706 21110 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:27.690837 24478 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:27.691468 24654 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:27.691643 24478 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:27.692319 24478 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:27.693313 21141 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:27.693950 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.694207 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.694744 21147 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:27.695107 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.695363 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.694689 21146 raft_consensus.cc:2368] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:27.695870 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.655953 24441 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:07:27.696861 21139 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:27.696964 24547 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:27.698738 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.654011 24589 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.700781 24589 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.701771 24547 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.702898 24589 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.704931 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.705248 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.705794 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.706120 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.705840 21146 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:27.708043 21142 raft_consensus.cc:2368] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
W20260709 13:07:27.708971 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.708922 21141 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:07:27.710501 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.717680 21139 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:27.719583 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.719883 21141 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:27.719929 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.720342 21147 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:27.720659 21142 raft_consensus.cc:2376] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8. Re-sending same reply.
I20260709 13:07:27.721505 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.721884 21140 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:27.722364 21036 leader_election.cc:400] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:27.690874 24663 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.722564 21143 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:27.722728 21036 leader_election.cc:403] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:27.729316 21796 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:07:27.722710 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.729789 21036 leader_election.cc:400] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:27.730098 21036 leader_election.cc:403] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:07:27.730022 21140 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:27.724089 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.724273 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.731388 21138 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:27.732159 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.732995 21143 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:27.734119 21143 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:27.735106 24516 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:27.736858 21036 leader_election.cc:400] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:27.735100 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2583) took 1006 ms. Trace:
I20260709 13:07:27.737255 21036 leader_election.cc:403] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:07:27.737565 21221 rpcz_store.cc:276] 0709 13:07:26.728763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.209403 (+480640us) service_pool.cc:224] Handling call
0709 13:07:27.734951 (+525548us) spinlock_profiling.cc:243] Waited 8.37 ms on lock 0x7b540013b354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:27.735047 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19257216}
I20260709 13:07:27.725500 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.741255 21036 leader_election.cc:400] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 9.
I20260709 13:07:27.741580 21036 leader_election.cc:403] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:27.726127 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 108954us
I20260709 13:07:27.746085 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.748188 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:27.727949 21794 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:27.719142 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.752368 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:27.719444 24553 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:07:27.759784 24460 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.760536 24460 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.751448 24310 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:27.763561 24724 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 9.
I20260709 13:07:27.764521 24547 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.765288 24460 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.751816 24663 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.766515 24663 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:27.768705 24663 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:27.768980 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.769593 21072 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:27.770457 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.772176 24615 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:27.773186 24611 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:27.775821 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.776443 21068 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:27.776945 24441 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:27.778855 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:27.779821 24653 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:27.780344 24653 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:27.781293 24653 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.784754 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:27.785789 21069 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.786350 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:27.751153 24594 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:27.789726 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.790330 21072 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.795575 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:27.796581 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:27.752182 24706 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.807265 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.807875 21139 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.814747 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:27.816008 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.817032 21072 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:27.817986 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.818528 21072 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:27.820760 21217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1083) took 3930 ms (client timeout 2134 ms). Trace:
W20260709 13:07:27.821254 21217 rpcz_store.cc:269] 0709 13:07:23.890700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.904601 (+ 13901us) service_pool.cc:224] Handling call
0709 13:07:27.820568 (+3915967us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54003b4cd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:27.820688 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3587183360}
I20260709 13:07:27.827512 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:27.819530 21141 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:07:27.828806 21801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 830) took 2730 ms (client timeout 2090 ms). Trace:
W20260709 13:07:27.830193 21801 rpcz_store.cc:269] 0709 13:07:25.098442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.137760 (+ 39318us) service_pool.cc:224] Handling call
0709 13:07:27.819974 (+2682214us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54005401d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:27.828717 (+  8743us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2555159808}
I20260709 13:07:27.793748 24460 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:27.793982 24690 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:27.804128 24654 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:27.808722 24663 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:27.848882 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.849412 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32968 (request call id 14) took 2243 ms. Trace:
I20260709 13:07:27.850121 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.851315 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.806911 24678 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:27.849839 21144 rpcz_store.cc:276] 0709 13:07:25.605857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.606021 (+   164us) service_pool.cc:224] Handling call
0709 13:07:27.849165 (+2243144us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54004231d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42516853 00007f7f474e460a 00007f7f40a6a7d4 00007f7f40a6a762 00007f7f40a6a691 00007f7f40a6a60c 00007f7f40a698c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:27.849309 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3009723776}
I20260709 13:07:27.838426 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.845247 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.847525 24663 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:27.836992 24460 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.835631 24676 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:27.834241 24548 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:27.820052 24589 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:07:27.835981 21068 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:27.865594 21143 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:27.879423 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.880206 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:27.866734 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.775882 24478 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.885244 21139 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:27.864099 24690 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:27.888891 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.889701 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.872300 24460 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:27.897354 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.804183 24530 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:07:27.803807 24553 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:07:27.872032 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.874390 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.909385 21138 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:27.812363 24557 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:07:27.805305 24607 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:07:27.845279 21145 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:07:27.891319 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.918798 21068 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:27.868243 21069 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:27.894266 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 499) took 4008 ms. Trace:
I20260709 13:07:27.919531 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.919925 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.895396 24594 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:27.894994 21218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1093) took 2629 ms (client timeout 2827 ms). Trace:
I20260709 13:07:27.920447 21145 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:27.919667 21071 rpcz_store.cc:276] 0709 13:07:23.885623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.229189 (+2343566us) service_pool.cc:224] Handling call
0709 13:07:26.250892 (+ 21703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.251125 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:27.194944 (+943819us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.618658 (+423714us) spinlock_profiling.cc:243] Waited 8.52 ms on lock 0x7b1800253ae0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:27.893636 (+274978us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b1800253ae0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287
0709 13:07:27.893865 (+   229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.893926 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:27.893983 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.894058 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.894196 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38554,"raft.run_wall_time_us":24941,"spinlock_wait_cycles":26867584,"thread_start_us":149389,"threads_started":2,"wal-append.queue_time_us":116488}
W20260709 13:07:27.920981 21218 rpcz_store.cc:269] 0709 13:07:25.265257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.446458 (+181201us) service_pool.cc:224] Handling call
0709 13:07:27.894787 (+2448329us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54002ee5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:27.894938 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3523202816}
I20260709 13:07:27.867758 24545 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:07:27.899991 21072 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:27.887641 21141 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:27.875759 24548 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.924919 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.925752 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:27.926746 21218 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.894270 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:27.911648 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.872812 24678 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:27.916705 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.867583 24445 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:07:27.920732 21212 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.922989 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:27.923736 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.928191 21143 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:27.932601 21142 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:27.933794 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.934820 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.935608 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.931738 21072 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:27.936304 21138 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:27.937456 21221 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.939942 21072 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:27.940858 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.947359 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:27.948972 21219 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:07:27.951637 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.952785 21214 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:07:27.941246 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.948006 21220 raft_consensus.cc:2376] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10. Re-sending same reply.
I20260709 13:07:27.960870 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.961671 21220 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:27.967938 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.929283 24548 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:27.887930 24663 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.918498 24576 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:27.973771 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.981465 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.982384 21138 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:27.981518 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.974161 21068 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:27.974498 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:27.985082 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.984956 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.987661 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:27.989145 21219 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:07:27.991739 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.993563 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:27.983548 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:27.941695 24607 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:28.002801 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.999936 21220 raft_consensus.cc:2368] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:28.007151 21140 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:28.008073 21145 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:27.961357 24654 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:27.931740 24528 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:27.971452 24545 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:28.011163 21214 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:28.016937 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:27.975745 24687 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:27.972117 24588 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:27.933313 24594 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:28.040769 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.013705 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.041816 21138 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:07:28.042580 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.019507 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.021238 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.014338 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:28.014937 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.048442 21139 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:27.925966 24662 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:07:28.011864 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.049814 21221 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:28.024538 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.052237 21219 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:28.066903 24247 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:28.069348 24478 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:28.073074 24528 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:28.072896 24678 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.015952 24615 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.080506 24615 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.040594 24687 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:28.084046 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.084693 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:28.085065 24687 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:28.086172 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:28.086457 24687 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:28.088920 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.089753 21216 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:28.090538 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.090643 24587 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:28.091959 21220 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:07:28.092583 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.092931 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.093602 21220 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:28.093675 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:28.094383 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:28.094995 21220 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:28.096230 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.097057 21215 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:28.067170 24662 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:28.097936 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:28.098366 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:28.099311 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.094409 21213 raft_consensus.cc:2368] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10.
I20260709 13:07:28.087193 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:28.107782 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.108035 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.123476 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:28.125080 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.121886 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.127676 21145 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:28.128258 21214 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:28.009819 24663 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:28.129863 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.126942 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:28.126065 21215 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:28.133046 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.133664 21138 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:28.083122 24615 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:28.042529 24445 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.019790 24545 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:28.135699 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.107360 24587 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:28.136392 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:28.137101 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.158514 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.017042 24654 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:28.141245 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:28.166545 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.166872 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.144083 21145 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:28.175697 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.134949 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.147081 21036 leader_election.cc:400] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:28.178037 21036 leader_election.cc:403] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:07:28.164319 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 1982) took 3205 ms. Trace:
W20260709 13:07:28.138501 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51371.5us
I20260709 13:07:28.178902 21070 rpcz_store.cc:276] 0709 13:07:24.958742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.495651 (+1536909us) service_pool.cc:224] Handling call
0709 13:07:26.549827 (+ 54176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.550353 (+   526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.164114 (+1613761us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b540041d050. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:28.164259 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123749632}
I20260709 13:07:28.169651 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.003777 24576 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.109654 24662 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:28.015899 24607 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:28.352150 21069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 1995) took 2328 ms (client timeout 1923 ms). Trace:
I20260709 13:07:28.190948 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:28.228905 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:28.352753 21069 rpcz_store.cc:269] 0709 13:07:25.860750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.971240 (+110490us) service_pool.cc:224] Handling call
0709 13:07:28.188447 (+2217207us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b540001aed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:28.188564 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4170955520}
I20260709 13:07:28.211073 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.350116 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.175127 21141 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:28.211971 23761 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:28.136993 24724 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:28.354130 21145 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:28.353330 21142 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:28.363075 21138 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:28.187879 24572 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:28.365891 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.099493 24588 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:28.375098 21801 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:07:27.997785 24690 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:28.377825 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:28.211522 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 286684us
I20260709 13:07:28.379263 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.379092 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.133880 24259 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.219s
I20260709 13:07:28.394025 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.398690 21145 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:28.399410 21219 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
W20260709 13:07:28.402474 24739 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.495s	user 0.001s	sys 0.000s
W20260709 13:07:28.404001 24742 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.458s	user 0.001s	sys 0.000s
I20260709 13:07:28.403752 24750 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:28.405130 23761 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:28.405551 23761 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:28.406383 23761 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:28.407193 24576 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:28.407771 24576 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.163486 24445 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:28.215523 24688 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:28.144466 24553 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.240s
I20260709 13:07:28.412118 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.372014 24724 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:28.423499 24724 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:28.374696 24247 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:28.427747 21146 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:28.420459 24622 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:28.421355 24688 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:28.351011 24589 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:07:28.432075 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.350323 24650 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:07:28.432111 24445 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:28.450850 21220 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:28.461220 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:28.459652 24706 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.647s	user 0.010s	sys 0.000s
I20260709 13:07:28.408221 24576 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.464967 24676 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:28.497504 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.498148 21146 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:28.505398 21801 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:28.536186 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:28.547145 21187 leader_election.cc:400] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:07:28.548725 21187 leader_election.cc:403] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:07:28.493279 24706 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
W20260709 13:07:28.554750 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 364155us
I20260709 13:07:28.465453 24688 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.415218 24545 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:28.556144 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:28.500334 24676 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:28.433260 24654 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.546770 24612 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:28.593737 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.596167 24611 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:28.596697 24611 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:28.597667 24611 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.588155 24478 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:28.599771 24478 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:28.601068 24478 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.599835 24310 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:28.605170 24676 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.605854 24310 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:28.608749 24662 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.609823 24662 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:28.612916 23761 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:28.613966 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.598834 24611 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:28.572043 24547 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.011s	sys 0.011s
I20260709 13:07:28.633038 21146 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:28.620461 21066 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:28.634807 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.635118 24318 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:07:28.635540 21147 raft_consensus.cc:2376] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10. Re-sending same reply.
I20260709 13:07:28.638119 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.639541 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.640815 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.643321 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.647614 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:28.652483 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.652217 21147 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:28.655529 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.654002 24453 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:28.655791 24594 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:28.654263 21801 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:07:28.662128 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.671828 24607 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:28.672099 24226 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:28.606981 24310 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.676553 24310 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:28.680487 24548 consensus_meta.cc:287] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.683s	user 0.007s	sys 0.002s
I20260709 13:07:28.681102 24548 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.685081 24588 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:28.688572 24576 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:28.689426 24654 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:28.696756 24553 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:28.697376 24553 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:28.702695 24612 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:28.704553 24612 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:28.619685 24545 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:28.708799 24688 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:28.712858 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:28.714766 24676 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:28.719622 24588 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:28.727344 24460 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.809s	user 0.012s	sys 0.000s
W20260709 13:07:28.650388 24741 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.098s	user 0.003s	sys 0.000s
I20260709 13:07:28.698381 24553 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:28.729142 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1071) took 1109 ms. Trace:
I20260709 13:07:28.729944 24553 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:28.729545 21067 rpcz_store.cc:276] 0709 13:07:27.620112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.626603 (+  6491us) service_pool.cc:224] Handling call
0709 13:07:27.656072 (+ 29469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.656402 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.656430 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.668240 (+ 11810us) log.cc:844] Serialized 26 byte log entry
0709 13:07:28.000555 (+332315us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:28.000601 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.000692 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.649514 (+648822us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:28.728862 (+ 79348us) raft_consensus.cc:1685] finished
0709 13:07:28.728886 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.729088 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":556,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":35018,"spinlock_wait_cycles":7025920,"thread_start_us":331920,"threads_started":1,"wal-append.queue_time_us":802865}
I20260709 13:07:28.737730 24548 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:28.740636 24729 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.741258 24729 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:28.744202 24729 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:28.754456 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.755540 21146 raft_consensus.cc:2376] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9. Re-sending same reply.
I20260709 13:07:28.756672 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.757543 21146 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 6: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:28.639266 24724 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:07:28.770704 24663 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.000s
W20260709 13:07:28.631536 24547 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.012s	sys 0.012s
I20260709 13:07:28.771947 21143 raft_consensus.cc:2410] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:28.774508 21142 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
W20260709 13:07:28.621994 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.792042 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.805335 21035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.027s	sys 0.031s
W20260709 13:07:28.805881 21035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.027s	sys 0.031s
I20260709 13:07:28.807001 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2018) took 1073 ms. Trace:
W20260709 13:07:28.807715 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53844.7us
I20260709 13:07:28.807516 21066 rpcz_store.cc:276] 0709 13:07:27.734033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.851042 (+117009us) service_pool.cc:224] Handling call
0709 13:07:28.806823 (+955781us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54003cb254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:28.806940 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1659777920}
I20260709 13:07:28.810871 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:28.814014 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:28.821694 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 630781us
I20260709 13:07:28.821810 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.822577 21143 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:28.837121 21142 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:07:28.837927 21142 pending_rounds.cc:85] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 6
I20260709 13:07:28.838399 21142 pending_rounds.cc:107] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:07:28.839015 21142 raft_consensus.cc:2889] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:28.836730 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 220815us
I20260709 13:07:28.845182 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:28.864626 21217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.038s	sys 0.058s
W20260709 13:07:28.858973 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 45039.3us
W20260709 13:07:28.865244 21217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.038s	sys 0.058s
I20260709 13:07:28.865749 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:28.866117 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:28.865075 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.867491 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.867343 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:28.868047 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.869650 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:28.863430 21143 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
W20260709 13:07:28.843196 24653 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.005s	sys 0.019s
I20260709 13:07:28.770731 24724 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:28.834985 24453 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:28.920272 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 78541.6us
I20260709 13:07:28.921638 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:28.906523 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:28.922580 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:28.922812 21066 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:28.924540 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:28.843436 24652 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.006s	sys 0.020s
W20260709 13:07:28.909976 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 90502.9us
W20260709 13:07:28.931921 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.835507 24667 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:28.807237 24588 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.101s
W20260709 13:07:28.850242 24549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.013s	sys 0.012s
I20260709 13:07:28.947860 21066 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:28.908380 24615 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:28.935608 21067 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:07:28.907547 24653 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.006s	sys 0.019s
W20260709 13:07:28.954396 20968 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 241) took 27617 ms (client timeout 15000 ms). Trace:
I20260709 13:07:28.948135 24606 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:28.948319 24445 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:28.957024 21064 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:28.957572 24691 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.958616 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:28.959669 24613 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:28.960000 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:28.986168 21796 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:29.002337 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:28.961011 24550 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:29.004932 21066 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:29.006656 21071 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:29.006975 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:28.961083 24588 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:28.963171 24318 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:28.811386 24460 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:28.920486 24453 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:28.935141 24652 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.006s	sys 0.020s
I20260709 13:07:28.864265 24607 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:29.008124 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:29.009040 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:28.986228 24709 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.849942 24576 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:07:29.012993 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:29.010479 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:29.044286 21066 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:29.015709 21072 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:29.019434 21067 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:29.020176 21143 raft_consensus.cc:2376] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9. Re-sending same reply.
I20260709 13:07:28.975823 20970 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:28.983947 24549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.013s	sys 0.013s
W20260709 13:07:28.977298 24687 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.890s	user 0.008s	sys 0.003s
I20260709 13:07:29.028957 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:29.029932 21138 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:07:29.055492 21038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.017s	sys 0.020s
I20260709 13:07:28.977362 24667 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.036938 21069 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:28.986560 24606 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:29.037398 21063 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:07:29.008193 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:29.063858 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:29.064594 21065 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:28.957088 24226 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:07:29.037434 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.047636 24588 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:29.019590 24445 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:07:28.985875 20968 rpcz_store.cc:269] 0709 13:07:01.336780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.355518 (+ 18738us) service_pool.cc:224] Handling call
0709 13:07:01.565839 (+210321us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b5400002490. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f8640 00007f7f482cfb76 00007f7f483d9c3e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc 00007f7f46899392 00007f7f4085748c 00007f7f408568c6
0709 13:07:28.954309 (+27388470us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:22.821542 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:22.926952 (+105410us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:22.927015 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:07:22.927035 (+    20us) tablet.cc:662] Decoding operations
0709 13:07:23.135771 (+208736us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:23.135856 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:23.135881 (+    25us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:07:23.137454 (+  1573us) tablet.cc:701] Row locks acquired
0709 13:07:23.137509 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:07:23.137668 (+   159us) write_op.cc:270] Start()
0709 13:07:23.231335 (+ 93667us) write_op.cc:276] Timestamp: P: 1783602443137606 usec, L: 0
0709 13:07:23.254574 (+ 23239us) op_driver.cc:348] REPLICATION: starting
0709 13:07:23.383482 (+128908us) log.cc:844] Serialized 8351 byte log entry
0709 13:07:25.327518 (+1944036us) op_driver.cc:464] REPLICATION: finished
0709 13:07:25.611747 (+284229us) write_op.cc:301] APPLY: starting
0709 13:07:25.611854 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:25.612253 (+   399us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:25.612392 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:25.907824 (+295432us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:26.044067 (+136243us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=21
0709 13:07:26.044122 (+    55us) write_op.cc:312] APPLY: finished
0709 13:07:26.044628 (+   506us) log.cc:844] Serialized 145 byte log entry
0709 13:07:26.044833 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:26.045036 (+   203us) write_op.cc:454] Released schema lock
0709 13:07:26.045113 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:26.045511 (+   398us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:26.045542 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":109015,"reported_tablets":36,"spinlock_wait_cycles":2646784,"child_traces":[["op",{"apply.queue_time_us":283902,"apply.run_cpu_time_us":9980,"apply.run_wall_time_us":433755,"num_ops":21,"prepare.queue_time_us":801197,"prepare.run_cpu_time_us":36258,"prepare.run_wall_time_us":1187814,"raft.queue_time_us":857746,"raft.run_cpu_time_us":19736,"raft.run_wall_time_us":244672,"replication_time_us":1977783,"thread_start_us":1484536,"threads_started":4,"wal-append.queue_time_us":924182,"wal-append.run_cpu_time_us":68298,"wal-append.run_wall_time_us":2753326}]]}
I20260709 13:07:28.931538 24684 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 13:07:29.019380 24318 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:29.053745 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:29.120627 21143 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:07:28.952883 24587 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.005s	sys 0.026s
I20260709 13:07:29.054922 21139 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:29.033102 24453 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:29.136585 21147 raft_consensus.cc:2410] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:07:29.043699 24607 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:29.057297 21038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.018s	sys 0.020s
W20260709 13:07:29.057668 21215 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.016s	sys 0.018s
W20260709 13:07:28.978901 24572 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.000s
I20260709 13:07:28.975210 24612 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:29.041090 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2763) took 1150 ms. Trace:
W20260709 13:07:29.141594 24756 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:07:29.141752 21144 rpcz_store.cc:276] 0709 13:07:27.890423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.892113 (+  1690us) service_pool.cc:224] Handling call
0709 13:07:29.013590 (+1121477us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b5400482720. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42493dc0 00007f7f4251fe18 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6
0709 13:07:29.040908 (+ 27318us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b540030b954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:29.041034 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305150976}
I20260709 13:07:29.083884 24588 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:29.085565 21065 raft_consensus.cc:2410] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:07:29.175843 21146 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:07:29.069701 21767 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:29.106506 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:29.108501 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:29.180423 21072 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:29.063120 24667 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:29.015164 24691 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:29.118373 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2604) took 1017 ms. Trace:
W20260709 13:07:29.187923 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.051s	sys 0.107s
I20260709 13:07:29.121652 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:29.188835 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:29.188107 21216 rpcz_store.cc:276] 0709 13:07:28.101672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.103389 (+  1717us) service_pool.cc:224] Handling call
0709 13:07:28.103818 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.104273 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.104301 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.104834 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 13:07:28.571709 (+466875us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:28.571758 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.571819 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.118094 (+546275us) raft_consensus.cc:1685] finished
0709 13:07:29.118125 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.118293 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2769,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":12729,"thread_start_us":466468,"threads_started":1,"wal-append.queue_time_us":953679}
W20260709 13:07:29.139688 24587 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.005s	sys 0.030s
I20260709 13:07:29.106156 24226 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:29.141247 24572 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:29.084878 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.208933 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:29.209707 21138 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:29.050506 24709 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:29.181895 23761 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:29.232391 23761 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:29.233470 23761 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:29.182314 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 112693us
W20260709 13:07:29.140311 21215 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.016s	sys 0.018s
I20260709 13:07:29.060983 24524 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:07:29.122465 24318 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:29.134960 24460 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:29.210700 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:29.211688 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:29.211000 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:29.216313 24747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.019s	sys 0.020s
W20260709 13:07:29.223719 21186 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.079s	user 0.035s	sys 0.047s
I20260709 13:07:29.240784 21141 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:29.234028 21145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.032s	sys 0.062s
W20260709 13:07:29.188485 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.051s	sys 0.107s
I20260709 13:07:29.122318 24445 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:07:29.237373 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:29.239464 20970 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.240934 21218 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:29.244585 21147 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:29.122488 24615 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:29.247409 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:29.247799 21216 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:29.248605 21220 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:29.249699 21214 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:07:29.250819 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 142396us
I20260709 13:07:29.252928 21068 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:07:29.253224 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.260363 21213 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:29.260599 21147 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:29.258270 21139 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:07:29.264951 21145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.032s	sys 0.063s
I20260709 13:07:29.279737 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:29.280543 21214 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:29.243211 24747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.019s	sys 0.020s
I20260709 13:07:29.285000 21138 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:29.285738 21220 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:29.286156 21213 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:29.278976 21216 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:29.288473 24612 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:29.214459 24587 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:29.289659 24587 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:29.292286 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:29.296494 21214 raft_consensus.cc:2410] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:07:29.296907 21141 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:29.305248 21217 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:29.309023 21219 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:29.304911 20970 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 6, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 6 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.311654 20970 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 6, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 6 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.313230 20970 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 7 to 9, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 9 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.314913 20970 catalog_manager.cc:5697] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 2 to -1, leader changed from <none> to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193), NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) evicted, now has a pending config: VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193), VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194), VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195), NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 3 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } } pending_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:07:29.224406 21212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.013s	sys 0.035s
W20260709 13:07:29.317111 21212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.014s	sys 0.035s
I20260709 13:07:29.318217 20970 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:29.319042 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1118) took 1299 ms. Trace:
I20260709 13:07:29.261492 21147 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:29.322528 21147 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:29.240768 21186 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.036s	sys 0.047s
W20260709 13:07:29.291735 24553 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.002s
I20260709 13:07:29.329244 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:29.330510 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 65780.1us
I20260709 13:07:29.261232 24684 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:29.320387 20970 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.336409 20970 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 7 to 8, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:29.323196 24662 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.713s	user 0.008s	sys 0.002s
I20260709 13:07:29.291473 24615 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:29.365064 21144 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:07:29.355556 21147 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:07:29.366765 21147 pending_rounds.cc:85] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 6
I20260709 13:07:29.367890 21147 pending_rounds.cc:107] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:07:29.365783 20970 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 4, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 4 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.371040 21147 raft_consensus.cc:2889] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:29.371730 21147 pending_rounds.cc:107] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:07:29.372097 21147 raft_consensus.cc:2889] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:29.309381 24549 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:29.294355 24653 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
W20260709 13:07:29.384547 21187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.017s	sys 0.051s
W20260709 13:07:29.384939 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55799.1us
W20260709 13:07:29.388785 21187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.017s	sys 0.052s
W20260709 13:07:29.390305 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59050.7us
I20260709 13:07:29.394390 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:29.391487 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 848249us
I20260709 13:07:29.398277 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:29.305373 24606 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:29.409116 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.428910 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:29.427356 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:29.430162 21801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.011s	sys 0.018s
W20260709 13:07:29.432585 21801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.012s	sys 0.018s
I20260709 13:07:29.431242 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:29.396855 24478 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.020s	sys 0.034s
I20260709 13:07:29.435636 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:29.389957 24767 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.397955 24247 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:29.438019 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:29.348461 24588 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:29.467711 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.481949 21138 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:07:29.380436 24226 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:29.444486 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:29.430253 24755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.006s	sys 0.038s
W20260709 13:07:29.455504 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.460717 21221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.052s	user 0.025s	sys 0.039s
W20260709 13:07:29.466216 21794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.002s	sys 0.020s
I20260709 13:07:29.403514 24684 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:29.431919 24760 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:07:29.536494 21221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.026s	sys 0.043s
W20260709 13:07:29.535833 21794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.002s	sys 0.021s
I20260709 13:07:29.319680 21212 rpcz_store.cc:276] 0709 13:07:28.020046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.020213 (+   167us) service_pool.cc:224] Handling call
0709 13:07:28.020518 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.020782 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:28.549408 (+528626us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.318264 (+768856us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:29.318347 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:29.318425 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.318755 (+   330us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.318961 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":533836,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":125,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":1221679,"threads_started":2}
I20260709 13:07:29.402909 24572 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:07:29.386952 24549 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:29.463054 24602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.778s	user 0.027s	sys 0.021s
W20260709 13:07:29.442636 24478 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.020s	sys 0.035s
W20260709 13:07:29.536327 24755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.006s	sys 0.039s
W20260709 13:07:29.491693 24690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.004s	sys 0.043s
I20260709 13:07:29.461974 24247 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:29.546859 21185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.035s	sys 0.044s
W20260709 13:07:29.552173 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 124964us
W20260709 13:07:29.552696 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 114752us
I20260709 13:07:29.497778 24649 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:29.442242 24663 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:07:29.435034 24548 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:07:29.546687 24684 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:29.558658 24662 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:29.497923 24553 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
W20260709 13:07:29.555642 24602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.870s	user 0.027s	sys 0.022s
I20260709 13:07:29.498147 24678 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:07:29.564569 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:29.564287 21036 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.003s	sys 0.014s
W20260709 13:07:29.583511 21036 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.004s	sys 0.014s
W20260709 13:07:29.584772 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 663391us
W20260709 13:07:29.564848 21185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.037s	sys 0.045s
I20260709 13:07:29.607815 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2210) took 1444 ms. Trace:
W20260709 13:07:29.608247 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 738604us
W20260709 13:07:29.609095 21218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1109) took 1677 ms (client timeout 1894 ms). Trace:
I20260709 13:07:29.557410 24259 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:07:29.560169 24690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.005s	sys 0.044s
W20260709 13:07:29.586117 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.614120 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.565995 24194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.007s	sys 0.007s
W20260709 13:07:29.587679 21765 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.005s	sys 0.013s
I20260709 13:07:29.568229 24607 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:29.587682 21070 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.004s	sys 0.012s
I20260709 13:07:29.565910 24247 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:29.586968 24652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.008s	sys 0.008s
I20260709 13:07:29.577003 24548 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:29.591248 21066 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:29.588536 21072 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:07:29.629653 21072 pending_rounds.cc:85] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 5
I20260709 13:07:29.629936 21072 pending_rounds.cc:107] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:07:29.630201 21072 raft_consensus.cc:2889] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:29.577821 24684 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:29.602998 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:29.568632 24761 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:07:29.604351 21064 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.005s	sys 0.013s
I20260709 13:07:29.609017 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:29.609483 21218 rpcz_store.cc:269] 0709 13:07:27.932138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.946045 (+ 13907us) service_pool.cc:224] Handling call
0709 13:07:29.608948 (+1662903us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540034c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:29.609041 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2861382912}
I20260709 13:07:29.462286 24767 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:29.641474 24767 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:29.610524 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:29.629832 24775 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:07:29.608201 21141 rpcz_store.cc:276] 0709 13:07:28.163916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.378791 (+214875us) service_pool.cc:224] Handling call
0709 13:07:29.607663 (+1228872us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b540036cc54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:29.607762 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957057408}
W20260709 13:07:29.600471 24774 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 13:07:29.616484 21765 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.005s	sys 0.013s
I20260709 13:07:29.562844 24766 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 13:07:29.616976 21070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.004s	sys 0.012s
W20260709 13:07:29.652293 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 814687us
I20260709 13:07:29.586664 24650 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:29.594835 24654 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 13:07:29.618336 24194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.007s	sys 0.007s
W20260709 13:07:29.632225 21142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.011s	sys 0.012s
W20260709 13:07:29.628194 24652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.008s	sys 0.008s
W20260709 13:07:29.637539 24777 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 13:07:29.636761 21064 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.005s	sys 0.014s
I20260709 13:07:29.644469 21185 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:29.614643 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.652995 21765 leader_election.cc:336] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.924s (SENT)
I20260709 13:07:29.667831 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:29.632805 24690 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:29.628865 24548 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:29.657963 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 422146us
W20260709 13:07:29.659525 21142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.011s	sys 0.012s
I20260709 13:07:29.646255 24778 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:29.670817 21800 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:07:29.682253 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:29.683139 21767 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:29.686108 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:29.686314 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:29.696594 24662 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:29.697219 24662 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:29.698016 24662 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:29.700099 24776 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:07:29.701316 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2194) took 1609 ms. Trace:
I20260709 13:07:29.701727 21145 rpcz_store.cc:276] 0709 13:07:28.091802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.399572 (+307770us) service_pool.cc:224] Handling call
0709 13:07:28.399839 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.463523 (+ 63684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.463554 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:28.464027 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.265958 (+801931us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:29.266026 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.266140 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.701041 (+434901us) raft_consensus.cc:1685] finished
0709 13:07:29.701061 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.701252 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63311,"raft.queue_time_us":67455,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":10218,"thread_start_us":769087,"threads_started":1,"wal-append.queue_time_us":1065792}
W20260709 13:07:29.709426 24764 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 13:07:29.711385 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.712538 24780 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:07:29.719632 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.720301 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:29.728766 23761 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.671914 24602 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:29.735713 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:29.736385 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:29.729526 23761 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:29.793777 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51250.9us
I20260709 13:07:29.794883 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:29.690909 24711 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:29.799472 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:29.800246 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56039.2us
I20260709 13:07:29.807055 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:29.807943 21071 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:29.809365 23761 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:29.684599 24194 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:07:29.818415 23761 raft_consensus.cc:2764] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:07:29.739929 24549 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:29.820737 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:29.806656 24711 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:29.822589 21069 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.003s	sys 0.034s
W20260709 13:07:29.828569 21069 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.003s	sys 0.034s
I20260709 13:07:29.842278 21794 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:29.840564 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 105265us
I20260709 13:07:29.845882 21036 leader_election.cc:400] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:29.846315 21036 leader_election.cc:403] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:29.854528 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:29.866690 24763 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:07:29.881680 24763 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:29.889540 21767 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:29.892820 24318 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.002s
I20260709 13:07:29.893312 24318 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:29.894568 24602 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:29.895133 24602 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.838137 24711 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:29.913349 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 468965us
I20260709 13:07:29.919595 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:29.841678 24607 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:29.856086 24768 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:29.804710 24548 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:29.889568 24676 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:29.944464 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:29.914436 24602 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:29.856770 24687 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:07:29.904309 24790 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:29.941838 24653 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.937485 24318 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:29.984969 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 166943us
I20260709 13:07:29.954644 24613 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:07:29.991801 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 283791us
I20260709 13:07:29.950752 24676 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:29.954767 24607 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.007040 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:30.019748 21214 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:29.983917 24548 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:30.008085 24453 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.839s	user 0.008s	sys 0.005s
W20260709 13:07:30.025950 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.024312 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1088) took 1009 ms. Trace:
I20260709 13:07:30.029175 21070 rpcz_store.cc:276] 0709 13:07:29.014476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.014596 (+   120us) service_pool.cc:224] Handling call
0709 13:07:29.014891 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.015237 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.015269 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.034193 (+ 18924us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.652703 (+618510us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:30.016088 (+363385us) log.cc:844] Serialized 14 byte log entry
0709 13:07:30.024002 (+  7914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.024074 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.024118 (+    44us) raft_consensus.cc:1685] finished
0709 13:07:30.024130 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.024254 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7509,"prepare.queue_time_us":222796,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":4673,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":5888,"thread_start_us":945465,"threads_started":2,"wal-append.queue_time_us":748808}
W20260709 13:07:30.036276 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 391883us
W20260709 13:07:30.040447 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 423538us
W20260709 13:07:30.041536 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.044929 21063 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:30.048887 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:30.053174 21071 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
W20260709 13:07:30.066073 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.070648 21070 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:30.017825 24767 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:30.040887 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1121) took 1671 ms. Trace:
I20260709 13:07:30.073910 21219 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:30.074448 21215 rpcz_store.cc:276] 0709 13:07:28.369278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.369389 (+   111us) service_pool.cc:224] Handling call
0709 13:07:28.398115 (+ 28726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.398341 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:29.235491 (+837150us) log.cc:844] Serialized 14 byte log entry
0709 13:07:30.040029 (+804538us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:30.040089 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:30.040612 (+   523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.040697 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.040823 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1533,"prepare.queue_time_us":209045,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":343621,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7344,"thread_start_us":1640444,"threads_started":3,"wal-append.queue_time_us":591812}
I20260709 13:07:30.022974 24613 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:30.089458 24613 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:30.090454 24613 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.005601 24653 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:30.083701 24797 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:07:30.085361 24790 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:30.085558 24549 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.093302 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:30.038762 24691 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:30.027576 24607 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:30.100845 24788 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.291s	user 0.001s	sys 0.001s
I20260709 13:07:30.028801 24478 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:30.101008 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:30.101455 24792 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:07:30.101905 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.102064 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:30.015578 24760 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:30.111519 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.184782 21141 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
W20260709 13:07:30.185428 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.130517 24690 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:30.023430 24587 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:30.181396 24478 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:30.099655 24524 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.160465 24652 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:07:30.153941 24549 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.033425 24453 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.127727 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:30.122164 24667 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.889s	user 0.013s	sys 0.000s
I20260709 13:07:30.187701 23761 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.183161 24584 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:30.182945 24769 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:07:30.136176 24259 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.002s	sys 0.036s
I20260709 13:07:30.197278 24766 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.110275 24663 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:30.215588 24524 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.155435 24676 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.202452 24587 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.217087 24478 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:30.234642 24794 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:07:30.178030 24550 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 13:07:30.254505 24654 consensus_queue.cc:1175] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "46c1676121b64b7d9bcf2227cc639dd3" responder_term: 11 status { last_received { term: 10 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:30.256759 24676 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:30.220013 23761 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.232568 24572 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:30.257376 24572 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:30.257629 24588 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:30.257820 24572 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:30.258082 24760 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.224560 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 532) took 1182 ms. Trace:
I20260709 13:07:30.238229 24709 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:30.246239 24803 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:07:30.245769 24766 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:30.260864 24801 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:07:30.246663 24663 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:30.262456 24806 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:30.263252 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 849) took 1060 ms. Trace:
I20260709 13:07:30.263742 21796 rpcz_store.cc:276] 0709 13:07:29.202923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.203128 (+   205us) service_pool.cc:224] Handling call
0709 13:07:29.203594 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.204061 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.204101 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.204656 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.738839 (+534183us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:29.738878 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.738967 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.262969 (+524002us) raft_consensus.cc:1685] finished
0709 13:07:30.262995 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.263185 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1444,"raft.queue_time_us":845,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":24318,"thread_start_us":478812,"threads_started":1,"wal-append.queue_time_us":698151}
I20260709 13:07:30.246479 24690 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.233865 24769 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:30.233767 24259 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:30.234095 24667 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:30.272962 21065 rpcz_store.cc:276] 0709 13:07:29.042316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.184537 (+142221us) service_pool.cc:224] Handling call
0709 13:07:29.184799 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.185010 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:29.646274 (+461264us) log.cc:844] Serialized 14 byte log entry
0709 13:07:30.224089 (+577815us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:30.224166 (+    77us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:30.224225 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.224313 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.224480 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48904,"prepare.queue_time_us":682380,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":74,"raft.queue_time_us":82067,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":6258,"thread_start_us":971582,"threads_started":3,"wal-append.queue_time_us":532544}
I20260709 13:07:30.286262 24769 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.287611 24769 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:30.286016 24259 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:30.309002 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2197) took 2216 ms (client timeout 2008 ms). Trace:
W20260709 13:07:30.309487 21140 rpcz_store.cc:269] 0709 13:07:28.092715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.211702 (+118987us) service_pool.cc:224] Handling call
0709 13:07:30.308692 (+2096990us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54002968d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:30.308882 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3714867968}
I20260709 13:07:30.325275 24823 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.325943 24823 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:30.329325 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.282927 24709 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:30.206970 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 202149us
I20260709 13:07:30.098541 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:30.391376 21072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.009s	sys 0.010s
W20260709 13:07:30.391940 21072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.009s	sys 0.010s
W20260709 13:07:30.399925 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 63678.7us
W20260709 13:07:30.399053 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 271437us
W20260709 13:07:30.403998 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 484531us
I20260709 13:07:30.405263 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:30.230383 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1120) took 1861 ms. Trace:
W20260709 13:07:30.404523 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 306080us
I20260709 13:07:30.414368 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:30.506721 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 529) took 1511 ms. Trace:
W20260709 13:07:30.508257 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 93828.5us
W20260709 13:07:30.486405 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.004s	sys 0.010s
W20260709 13:07:30.511598 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.005s	sys 0.010s
W20260709 13:07:30.513818 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 659360us
W20260709 13:07:30.501153 21069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.004s	sys 0.012s
W20260709 13:07:30.515648 21069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.005s	sys 0.012s
W20260709 13:07:30.503468 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 100290us
W20260709 13:07:30.503749 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 299874us
I20260709 13:07:30.505599 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 423) took 1689 ms. Trace:
W20260709 13:07:30.522658 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.516985 21142 rpcz_store.cc:276] 0709 13:07:28.815911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.848800 (+ 32889us) service_pool.cc:224] Handling call
0709 13:07:28.849272 (+   472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:28.923553 (+ 74281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.923584 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:29.014813 (+ 91229us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b2c00010828. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:07:29.026362 (+ 11549us) log.cc:844] Serialized 52 byte log entry
0709 13:07:29.672229 (+645867us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:29.740877 (+ 68648us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.741150 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.741263 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.504733 (+763470us) raft_consensus.cc:1685] finished
0709 13:07:30.504902 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.505261 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73792,"prepare.queue_time_us":55110,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":111710,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":46053,"spinlock_wait_cycles":26814336,"thread_start_us":610507,"threads_started":1,"wal-append.queue_time_us":782520}
I20260709 13:07:30.525695 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:30.510965 21064 rpcz_store.cc:276] 0709 13:07:28.994919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.995099 (+   180us) service_pool.cc:224] Handling call
0709 13:07:29.027063 (+ 31964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.027336 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:29.664438 (+637102us) log.cc:844] Serialized 14 byte log entry
0709 13:07:30.505594 (+841156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:30.505753 (+   159us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:30.505881 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.506032 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.506338 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7103,"prepare.queue_time_us":795353,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":62589,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":52,"thread_start_us":1455959,"threads_started":3,"wal-append.queue_time_us":535078}
I20260709 13:07:30.527230 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:30.528345 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.423125 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:30.540686 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 117504us
I20260709 13:07:30.542102 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:30.542948 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:30.408766 21221 rpcz_store.cc:276] 0709 13:07:28.368890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.369005 (+   115us) service_pool.cc:224] Handling call
0709 13:07:28.407715 (+ 38710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.407945 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:29.543166 (+1135221us) log.cc:844] Serialized 14 byte log entry
0709 13:07:30.229861 (+686695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:30.229957 (+    96us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:30.230028 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.230123 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.230301 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":199687,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":7187,"raft.queue_time_us":259430,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":76,"thread_start_us":1820672,"threads_started":3,"wal-append.queue_time_us":482144}
I20260709 13:07:30.552953 21142 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:30.553375 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:30.551270 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:30.560832 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:30.572255 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:30.418996 23761 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:30.398231 24226 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.005s	sys 0.009s
W20260709 13:07:30.424515 24247 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.792s	user 0.017s	sys 0.001s
I20260709 13:07:30.403090 24584 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.649385 24247 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.425001 24607 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:30.427927 24615 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.044s	user 0.026s	sys 0.000s
I20260709 13:07:30.410481 24662 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:30.651613 24691 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:30.651708 24812 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:30.652393 24607 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:30.406105 24663 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:30.664539 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 104042us
I20260709 13:07:30.665021 24808 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:30.427706 24667 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:30.665952 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:30.429255 24763 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.547s	user 0.015s	sys 0.000s
I20260709 13:07:30.668012 24763 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:30.669147 24763 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.670050 24763 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:30.441401 24761 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.009s	sys 0.020s
W20260709 13:07:30.671811 24761 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.009s	sys 0.020s
I20260709 13:07:30.672413 24761 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:30.675362 24831 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.676283 24831 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.434633 24798 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:30.422206 24818 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:30.426448 24823 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:30.398146 24828 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:30.712276 24587 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:30.713230 24615 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.419003 24829 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:07:30.715911 24829 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:30.716432 24829 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:30.627506 21766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.008s	sys 0.028s
W20260709 13:07:30.628116 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.010s	sys 0.026s
W20260709 13:07:30.628872 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 70701.7us
I20260709 13:07:30.638736 21141 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:30.432803 24453 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:30.647722 24684 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:30.647923 24553 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.648056 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:30.651707 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1092) took 1625 ms. Trace:
I20260709 13:07:30.664721 24612 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.380797 24709 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.636432 24817 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:30.735395 24226 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.005s	sys 0.010s
I20260709 13:07:30.674376 24761 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.704365 24667 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:30.736423 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:30.736860 24667 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:30.709465 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 43818.2us
I20260709 13:07:30.739641 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:30.742153 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:30.723380 21766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.009s	sys 0.028s
W20260709 13:07:30.728026 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.011s	sys 0.026s
I20260709 13:07:30.724567 21069 rpcz_store.cc:276] 0709 13:07:29.025967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.143881 (+117914us) service_pool.cc:224] Handling call
0709 13:07:29.184341 (+ 40460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.184558 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:29.829174 (+644616us) log.cc:844] Serialized 14 byte log entry
0709 13:07:30.626585 (+797411us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:30.633993 (+  7408us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:30.644703 (+ 10710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.651453 (+  6750us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.651648 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":671086,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":68,"raft.queue_time_us":135482,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":259,"thread_start_us":1393891,"threads_started":3,"wal-append.queue_time_us":456027}
I20260709 13:07:30.710145 24653 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:30.739681 24817 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:30.730192 24553 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.700657 24837 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:30.718324 24829 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.753470 24453 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:30.757787 24790 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:30.747898 24831 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:30.789745 24615 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:30.794574 24812 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:30.679186 24798 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.802057 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:30.790032 24612 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.759711 24828 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:07:30.811054 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 285621us
I20260709 13:07:30.799038 24684 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:30.815477 24825 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:30.816458 24825 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:30.816072 24684 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:30.822458 24841 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.266s	user 0.002s	sys 0.000s
I20260709 13:07:30.820566 24790 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:30.823952 24684 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:30.825423 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 63429.2us
I20260709 13:07:30.825814 24829 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:30.826059 24584 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:30.752386 24667 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:30.836295 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 264410us
W20260709 13:07:30.837826 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.780416 24818 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:30.826853 24584 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:07:30.815498 24587 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:30.892715 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:30.900728 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.901794 24684 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.902855 24684 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.908257 24790 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:30.916507 24818 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.859766 24615 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:07:30.923498 24802 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.924587 24802 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:30.929767 24549 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:30.931432 21069 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:30.933182 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:30.934080 21069 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:07:30.936503 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 194610us
I20260709 13:07:30.906153 24684 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:30.943081 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:30.954592 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:30.859381 24831 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:07:30.982592 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:30.883872 24691 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:31.011827 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:31.017541 21069 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:31.001466 24549 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.007707 24667 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.997465 24828 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:31.025771 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:30.991531 24587 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.039693 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:31.044670 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 71080.6us
I20260709 13:07:31.045755 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:31.047554 24606 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.047693 24854 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:31.048367 24711 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.950s	user 0.027s	sys 0.000s
I20260709 13:07:31.048887 24711 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.039593 24613 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:07:31.050549 24769 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.762s	user 0.017s	sys 0.006s
I20260709 13:07:31.051174 24613 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:31.051498 24602 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.070s	user 0.013s	sys 0.013s
I20260709 13:07:31.052048 24602 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.053041 24766 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.054220 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.054847 24602 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.039197 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.052611 24613 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:31.057240 24676 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.800s	user 0.014s	sys 0.014s
I20260709 13:07:31.055266 21066 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
W20260709 13:07:31.044725 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 68636.6us
W20260709 13:07:31.047686 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 169420us
I20260709 13:07:31.011950 24802 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:31.039600 24460 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:30.939776 24825 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:31.083412 24548 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.926s	user 0.002s	sys 0.027s
W20260709 13:07:31.087445 24846 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:07:31.075830 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:31.089640 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.090471 21064 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:31.091173 21064 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:07:31.091581 24587 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:31.093101 21063 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
W20260709 13:07:31.095403 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 214272us
I20260709 13:07:31.096438 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1103) took 1040 ms. Trace:
I20260709 13:07:31.096982 21067 rpcz_store.cc:276] 0709 13:07:30.055537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.055730 (+   193us) service_pool.cc:224] Handling call
0709 13:07:30.110885 (+ 55155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:30.496701 (+385816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:30.496752 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:30.497448 (+   696us) log.cc:844] Serialized 26 byte log entry
0709 13:07:30.637536 (+140088us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:30.637578 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.811944 (+174366us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b54002ac6d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:30.812008 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:31.095926 (+283918us) raft_consensus.cc:1685] finished
0709 13:07:31.095951 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:31.096212 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121614,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":61934,"spinlock_wait_cycles":208897920,"thread_start_us":139627,"threads_started":1,"wal-append.queue_time_us":387474}
W20260709 13:07:31.100476 24572 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.842s	user 0.014s	sys 0.008s
I20260709 13:07:31.101081 24572 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.103433 24549 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.048332 24606 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.051263 24711 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:31.090971 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:31.057835 24763 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.131796 21221 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:31.132723 21215 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: []
W20260709 13:07:31.129725 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 142124us
I20260709 13:07:31.089569 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:31.135232 24247 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.130154 24832 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:31.167883 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.168321 21795 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:31.168330 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.169028 21219 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:31.170192 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.143515 24709 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.170158 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.048316 24667 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.173822 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:31.184996 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60482.9us
I20260709 13:07:31.128813 24572 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:31.139544 24606 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:31.153687 24553 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.141685 24549 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:31.195739 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 148477us
I20260709 13:07:31.129302 24859 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:31.199019 21215 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:31.201645 24663 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.202328 24663 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.204857 24663 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.206542 21215 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:07:31.190567 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.212954 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:07:31.222294 24833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.001s	sys 0.016s
I20260709 13:07:31.231750 21216 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:31.233407 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:31.234475 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:31.233071 24691 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.233511 24667 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:31.231499 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46914.3us
I20260709 13:07:31.235451 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:31.231595 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:31.227162 24833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.002s	sys 0.016s
I20260709 13:07:31.227468 24549 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:31.228261 24848 raft_consensus.cc:2764] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:07:31.250177 21219 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:31.229162 24612 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.237143 24798 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.256487 24572 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:31.288822 24572 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.289604 24572 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:31.288909 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 63666.5us
W20260709 13:07:31.295925 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 175351us
I20260709 13:07:31.297540 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:31.299510 24460 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:31.300030 24460 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:31.300436 24460 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:31.303056 24607 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.005s
I20260709 13:07:31.299042 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:31.363838 24607 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:31.311913 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:31.358438 24662 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.706s	user 0.010s	sys 0.001s
I20260709 13:07:31.355005 24667 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:07:31.356041 24856 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:07:31.361817 24553 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:31.359889 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:31.361428 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.405907 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.410723 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:31.421317 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:31.421478 21216 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:31.422070 21219 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:31.422804 21220 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:31.424808 21071 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:31.429399 24690 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:07:31.430853 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 78119.7us
I20260709 13:07:31.432621 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:31.433550 24690 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:31.443070 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 189068us
I20260709 13:07:31.451321 24247 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.451961 24247 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.454449 24247 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:31.459169 24588 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.201s	user 0.011s	sys 0.018s
I20260709 13:07:31.448230 24549 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:31.420226 24667 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:31.470679 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:31.470790 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 210978us
I20260709 13:07:31.448469 24662 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:31.473428 21220 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:31.473776 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.470585 24763 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:31.483448 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.485713 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:31.512548 21064 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:31.489421 21216 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:31.509817 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46697.1us
I20260709 13:07:31.491196 21215 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:31.492775 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1135) took 1473 ms. Trace:
I20260709 13:07:31.493816 21213 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:31.498082 24524 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:31.502679 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:31.514220 21217 rpcz_store.cc:276] 0709 13:07:30.018709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.018829 (+   120us) service_pool.cc:224] Handling call
0709 13:07:30.036502 (+ 17673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:31.231714 (+1195212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:31.231758 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:31.231794 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:31.231826 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:31.492383 (+260557us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002ec2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:31.492428 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:31.492581 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2747864448}
I20260709 13:07:31.517403 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:31.522027 21219 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:31.524214 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.524560 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.525553 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.526141 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.526420 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.526764 21221 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:31.526175 21215 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:31.526996 21217 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:31.504237 24662 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.535764 21071 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:31.536410 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 237515us
I20260709 13:07:31.537463 24662 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:31.548178 21219 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:31.548867 21217 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:31.548887 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.550227 21217 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:31.552781 24478 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:31.556551 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:31.557039 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.557060 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:31.557946 21063 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:31.558030 21065 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:31.557979 24667 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.561414 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.563658 21215 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:31.565662 21220 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 13:07:31.565876 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.574508 21767 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:31.575004 24460 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.575982 24447 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:31.577255 24460 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.578657 24819 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.520093 24572 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.568655 21111 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:31.588239 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.590332 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:31.590346 21065 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:31.591753 21220 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:31.592932 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.599383 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.603569 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.602788 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:31.615173 21071 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:31.544832 24549 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:07:31.629415 21219 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 4. Current term is 6. Ops: [4.4-4.4]
W20260709 13:07:31.630331 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79119.7us
W20260709 13:07:31.632853 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 341769us
I20260709 13:07:31.634061 24524 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:31.634720 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.635655 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.635864 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.637269 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.638558 21767 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:31.639569 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:31.634229 21216 raft_consensus.cc:2368] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:31.602639 24851 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:31.648372 24851 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.648789 24851 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:31.649902 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:31.581173 24819 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.651005 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.651759 21070 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 4.
I20260709 13:07:31.663496 21071 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:31.662118 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:31.679615 21067 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:31.660336 24653 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.687657 21070 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:07:31.684515 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.689528 21071 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:31.670499 24818 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:31.700132 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60642.3us
I20260709 13:07:31.677047 24829 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:31.701514 21187 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:31.706481 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.707530 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.713121 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.718118 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:31.702478 24653 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.720606 24653 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.721455 24663 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.722194 24663 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.723502 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:31.723670 21071 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:31.724368 21212 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:31.724556 24663 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:31.724694 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.725301 24764 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:31.725855 24778 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:31.725920 24764 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.726502 24778 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:31.727593 24778 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:31.728359 24764 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.728909 24606 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.729750 21145 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:31.725349 21071 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:31.735972 21143 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:07:31.738619 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:31.741791 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:31.742558 21142 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:31.736423 21143 pending_rounds.cc:85] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 4
I20260709 13:07:31.744302 21064 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:31.746037 21767 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:31.699214 24818 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:31.748883 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49101.1us
I20260709 13:07:31.729432 24606 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.752197 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:31.752833 21140 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:31.754261 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:31.757109 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:31.757763 21144 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:31.743805 21143 pending_rounds.cc:107] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:07:31.759663 21143 raft_consensus.cc:2889] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:31.696919 24850 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:07:31.762576 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:31.750391 24818 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:31.742836 24819 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.761905 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:31.766726 21064 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:31.758644 24662 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.737210 24829 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:31.774470 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.768888 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:31.779182 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:31.785392 21145 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:31.796208 21143 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:31.780212 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:31.778697 24830 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:31.775202 24766 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:31.804704 21138 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:31.812508 21142 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:31.811100 21064 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:31.781418 24607 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:31.812098 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:31.796346 24663 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:31.773087 24667 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:07:31.826200 21066 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:31.830199 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:31.800458 24572 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:31.776168 24761 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:07:31.805429 24478 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:31.778307 24859 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:07:31.840917 21142 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:31.814952 24607 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.816104 24766 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.838133 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:31.827787 24663 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.819916 24830 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:31.839844 24478 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:31.844198 24607 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:31.858719 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:31.859746 24663 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:31.857532 24766 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:31.883869 21145 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:31.890817 21795 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:31.869496 24478 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:31.894398 21143 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:07:31.870646 24226 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:31.900084 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 137601us
I20260709 13:07:31.904023 21145 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:07:31.864006 24829 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:31.934983 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:31.935896 21142 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:31.939836 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 171054us
I20260709 13:07:31.941310 24247 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:31.946894 21145 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:07:31.893713 24687 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 13:07:31.944878 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:31.953225 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:31.908979 24755 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 13:07:31.959952 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:31.962917 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:31.909089 24226 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:31.916777 24615 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:07:31.960057 24247 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:31.977188 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 101938us
I20260709 13:07:31.978202 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:31.979521 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:31.964763 24662 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:31.963478 24572 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:31.953697 24602 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:07:31.968649 24606 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:32.005577 24889 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:07:32.007596 24884 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:07:31.996613 24572 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.011781 24572 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:32.012537 24818 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:32.014724 24818 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:31.937002 24830 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.018580 21144 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:31.991319 24247 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.023018 21140 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
W20260709 13:07:32.028345 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.021570 24654 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:07:32.033515 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 77117.5us
I20260709 13:07:32.046046 21139 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
W20260709 13:07:32.084595 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 59058.8us
I20260709 13:07:32.044407 24662 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.088165 24662 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.026671 24755 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:32.085879 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:32.098301 24895 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:32.041455 24900 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:32.114236 24900 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:32.114773 24900 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:32.115886 24893 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 13:07:32.116621 24881 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.565s	user 0.003s	sys 0.003s
I20260709 13:07:32.082377 24654 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:32.094733 24887 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
W20260709 13:07:32.111091 24896 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:07:32.026050 24653 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
W20260709 13:07:32.033284 24890 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.096s	user 0.001s	sys 0.001s
I20260709 13:07:32.091094 24662 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.095472 24755 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:32.124966 21142 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:32.034399 24247 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:32.124562 24687 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:32.124924 24460 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.152500 24654 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.155738 24755 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.157123 24755 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:32.158818 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.159154 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.164430 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.142402 24687 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.167349 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.150668 24663 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.170039 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:32.176757 24851 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 13:07:32.177197 24851 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.179556 24906 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:07:32.180416 24892 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:32.180769 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.181298 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.181633 24851 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:32.181696 24524 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:32.180346 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.180930 24892 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:32.175140 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.168867 24687 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:32.182258 24524 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:32.163033 24460 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.210270 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.211313 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.220228 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:32.228514 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:32.218703 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.039s (SENT) [suppressed 47 similar messages]
I20260709 13:07:32.233119 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.234107 21215 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 6.
I20260709 13:07:32.240123 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.241050 21142 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:32.244925 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.249565 21797 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:32.264186 21141 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:32.265125 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.267596 21800 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:32.272146 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55340.9us
I20260709 13:07:32.250344 24914 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.233079 24892 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.297513 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:32.299901 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 98441us
I20260709 13:07:32.300450 24892 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:32.320454 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:32.328204 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.328959 21793 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
W20260709 13:07:32.314348 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8698757629999818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32410000000000005s for a response
W20260709 13:07:32.332142 24918 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.112s	user 0.002s	sys 0.000s
I20260709 13:07:32.336498 24927 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.337323 24927 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.321028 24663 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.314975 24607 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:32.332144 21036 connection.cc:256] RPC call timeout handler was delayed by -1.7062083780000421s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26589999999999997s for a response
W20260709 13:07:32.342526 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5520339280000144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 13:07:32.342864 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4026339280000002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
W20260709 13:07:32.343182 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3216339279999829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
W20260709 13:07:32.343654 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1407339280000315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
I20260709 13:07:32.296639 24914 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.344067 24663 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.344615 24663 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.342010 24607 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.348217 21066 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:07:32.348685 21066 pending_rounds.cc:85] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 3
I20260709 13:07:32.348932 21066 pending_rounds.cc:107] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:07:32.349157 21066 raft_consensus.cc:2889] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:32.349102 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:32.349646 24927 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.349793 21064 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:32.350457 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:32.351049 21072 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:32.363132 24851 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:32.364912 24653 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:32.365337 24653 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:32.366209 24653 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.368317 24829 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:32.369414 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.370411 21141 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:32.372591 21140 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:32.346755 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:32.370419 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:32.344013 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0822339280000506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
I20260709 13:07:32.388037 21064 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:32.391462 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:32.377822 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:32.367152 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1132) took 2361 ms. Trace:
I20260709 13:07:32.408059 21071 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:32.311774 24654 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.409308 24654 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:32.417451 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49178.7us
I20260709 13:07:32.419623 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:32.419849 21139 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:32.420275 21067 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:32.424945 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:32.426512 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:32.415341 24478 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.001s
I20260709 13:07:32.433220 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.421545 24798 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:32.397876 21218 rpcz_store.cc:276] 0709 13:07:30.005496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.005655 (+   159us) service_pool.cc:224] Handling call
0709 13:07:30.022210 (+ 16555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:31.470904 (+1448694us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:31.657806 (+186902us) log.cc:844] Serialized 14 byte log entry
0709 13:07:31.869627 (+211821us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:31.869674 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:31.869722 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:32.366760 (+497038us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54004434d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:32.366811 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:32.366975 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10505,"prepare.queue_time_us":28684,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":6670,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":7828,"spinlock_wait_cycles":3308687232,"thread_start_us":356694,"threads_started":2,"wal-append.queue_time_us":263142}
I20260709 13:07:32.438114 24905 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:32.432004 24602 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:32.305112 24524 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.431836 24478 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.427610 24653 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:32.428027 24615 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:07:32.447448 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 95737.7us
W20260709 13:07:32.449126 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78828.8us
I20260709 13:07:32.450155 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.418880 24572 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.432212 24829 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:07:32.452904 24798 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:32.454109 24859 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:32.459254 21795 raft_consensus.cc:2410] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:07:32.442667 24905 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:32.449746 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:32.442818 24602 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.469177 21143 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:32.480160 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56977.9us
I20260709 13:07:32.481267 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.485908 24478 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:32.488237 24226 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:32.492089 24524 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:32.495591 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.499694 24914 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.502244 24687 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.472788 24825 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:32.503623 24460 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:32.504217 24897 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:32.504704 24602 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:32.508543 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.509049 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.510216 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.510651 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.512151 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.512758 21217 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:07:32.515784 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 66129us
I20260709 13:07:32.516608 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:32.517731 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:32.465227 24900 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:32.472788 24766 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.000s
I20260709 13:07:32.530539 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:32.527108 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 77028.1us
I20260709 13:07:32.528748 24808 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:32.529631 24917 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:32.530028 24929 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:32.530895 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.531926 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:32.529155 21219 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:32.536854 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:32.540689 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:32.524884 24914 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:32.526598 24654 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.537900 24808 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.565147 24911 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:07:32.548873 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 103266us
W20260709 13:07:32.546571 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 117357us
I20260709 13:07:32.543804 24478 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:32.537364 24606 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:07:32.548060 24917 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:32.559168 24808 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:32.537629 24766 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.562351 24654 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:32.585196 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.596066 24687 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.599668 21219 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:32.569588 24478 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.611605 21139 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:07:32.611150 24917 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.612561 24910 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:32.613224 24910 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.579180 24911 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:32.587800 24654 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.620906 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.623418 21138 raft_consensus.cc:2368] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:32.609642 24478 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:32.627401 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.622288 24654 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:32.636843 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.638084 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.639014 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.639802 21147 raft_consensus.cc:2368] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:32.640038 24766 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.642379 24900 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:32.642513 24932 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:32.643018 24932 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:32.646759 24932 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.648123 24932 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:32.651139 24911 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.669219 24602 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:32.671932 24247 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.002s
I20260709 13:07:32.672554 24247 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.675880 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.676779 21147 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:32.687605 24247 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.637300 24572 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:32.694269 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 162442us
I20260709 13:07:32.697999 24829 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:32.698902 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.700631 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.701432 21140 raft_consensus.cc:2368] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:32.705559 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.705837 21795 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:07:32.706270 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.705539 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.707139 21147 raft_consensus.cc:2368] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:32.710753 24910 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:32.710891 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.711731 24602 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.712144 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.712828 21142 raft_consensus.cc:2368] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10.
I20260709 13:07:32.711148 21797 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:07:32.715186 21797 pending_rounds.cc:85] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 8
I20260709 13:07:32.715453 21797 pending_rounds.cc:107] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:07:32.715695 21797 raft_consensus.cc:2889] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:32.699160 24761 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:32.705615 24825 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.702544 24588 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:07:32.717833 24818 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:32.718902 24814 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.738477 24814 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.738754 24615 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:32.723969 24825 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.741322 24825 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:32.726536 24663 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.744861 24814 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:32.722098 24654 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.745271 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.727357 24808 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.736531 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.748287 21138 raft_consensus.cc:2368] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:32.700302 24829 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.749277 24808 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.719434 24226 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:32.742693 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.751438 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.751559 24829 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.753228 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.753141 24905 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.754127 24524 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:32.754640 24905 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:32.755503 24663 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:32.754963 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.758798 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.714459 24859 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.787631 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:32.747068 24654 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:32.805848 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 85621.8us
I20260709 13:07:32.800590 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:32.808735 24940 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:32.809392 24940 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.812294 24940 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:32.812975 24940 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:07:32.814495 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83805.2us
I20260709 13:07:32.755985 24911 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:32.761751 24915 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:32.816160 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.817063 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.786931 24808 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:32.820058 21111 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:32.820235 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.808840 24818 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:32.821635 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.831645 21139 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:32.823439 21141 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:07:32.824611 21145 raft_consensus.cc:2368] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:07:32.803978 24892 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.834455 21142 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:07:32.801656 24898 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:32.822461 24663 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:32.843408 21147 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:32.843544 24818 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:32.844743 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:32.844969 24818 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:32.845373 24818 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:32.846529 24830 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:32.834803 24911 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:32.847370 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.804760 24572 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 13:07:32.848814 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 102429us
I20260709 13:07:32.849320 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:32.804152 24859 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.846287 24818 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.840132 24915 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:07:32.863659 21139 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:32.836182 24226 consensus_queue.cc:1243] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:07:32.895444 21147 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
W20260709 13:07:32.896934 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49866.5us
I20260709 13:07:32.899734 24818 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:32.901505 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.902127 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.902666 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.903195 24859 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:32.903393 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.903786 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.904335 21220 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:32.904469 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.905624 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.903800 24859 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:07:32.906147 21213 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:32.867710 24247 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:07:32.896981 24942 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:32.909041 24942 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:32.911556 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.912159 21217 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:32.913800 21213 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:32.914783 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.915354 21213 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:32.915923 21216 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:32.914814 21220 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:32.916961 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.917275 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.918031 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.918329 21220 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:32.918583 21216 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:32.919615 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:32.920188 21221 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:32.920938 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:32.922385 21213 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
W20260709 13:07:32.923447 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.923949 24226 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:32.924015 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 74805.9us
I20260709 13:07:32.925545 24588 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:32.925837 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:32.927474 24761 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:32.927990 24761 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:32.928833 24761 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.933503 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:32.934281 21140 raft_consensus.cc:2368] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:32.934666 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.935091 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.935608 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.935927 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.936233 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:32.936589 21146 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:32.936793 21145 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:32.937578 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:32.938476 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.938759 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:32.939136 21796 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:32.939673 21795 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:07:32.939837 21797 raft_consensus.cc:2368] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12.
I20260709 13:07:32.940060 21795 pending_rounds.cc:85] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 7
I20260709 13:07:32.940726 21795 pending_rounds.cc:107] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:07:32.940984 21795 raft_consensus.cc:2889] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:32.943869 24226 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:32.945277 24226 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:32.915257 21218 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:32.947875 21218 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:32.848958 24944 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 13:07:32.954037 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 437) took 1089 ms. Trace:
I20260709 13:07:32.954495 21143 rpcz_store.cc:276] 0709 13:07:31.865172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.910315 (+ 45143us) service_pool.cc:224] Handling call
0709 13:07:31.952269 (+ 41954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:32.477346 (+525077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:32.953793 (+476447us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b54001eb4d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:32.953975 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1188206848}
I20260709 13:07:32.896282 24898 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.956254 24898 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.938894 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:32.970954 21217 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:32.839151 24892 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:32.995640 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:32.911149 24942 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:32.987757 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48956.8us
I20260709 13:07:32.947427 24478 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:33.059553 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 761) took 1164 ms. Trace:
W20260709 13:07:33.089160 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:33.065419 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.191208 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:32.948536 21218 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:33.171217 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 52358.3us
I20260709 13:07:32.904541 24915 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:33.064431 24907 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:32.991533 24932 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:33.175607 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:33.180217 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 233591us
I20260709 13:07:33.203786 24830 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:33.205390 24711 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:33.205791 24711 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:33.196825 24826 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:33.206661 24711 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.207311 24549 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.207578 21793 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:33.207901 24549 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.207330 24826 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:33.209640 24826 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:33.207937 24711 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:33.219425 24942 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:33.216202 24549 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:33.220041 24942 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.220399 24826 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.222188 24942 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:33.222621 24826 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:33.215847 24851 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:33.223542 24947 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:33.224454 24947 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:33.188568 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 707353us
I20260709 13:07:33.228725 24818 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:33.229133 21212 raft_consensus.cc:2410] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:33.229300 24818 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.230043 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:33.230660 21064 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:33.231590 24911 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:33.231963 24911 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:33.232376 24892 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:32.979692 21216 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:07:33.194237 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:33.232765 24911 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.233632 24814 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:33.233800 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:33.234040 24814 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:33.234856 21110 leader_election.cc:400] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12.
I20260709 13:07:33.235553 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:33.235791 24549 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:33.236207 21070 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:33.236456 24549 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:33.236861 24549 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:33.237216 24549 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:33.240439 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:33.241303 21069 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:07:33.241731 21791 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b6dd6862dd14baba4a82c5a837bc802 with delete_type TABLET_DATA_TOMBSTONED (TS b70fc04de37e4239a8336534f0c9c0fb not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:48444
I20260709 13:07:33.233981 24911 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:33.197724 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:33.235132 21110 leader_election.cc:403] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:07:33.243593 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45941.3us
I20260709 13:07:33.235131 24814 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:33.243798 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:33.244275 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:33.244602 24814 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.244634 24662 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:33.188335 24944 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:07:33.245777 24572 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12.
I20260709 13:07:33.246439 24572 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:33.246961 24572 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:33.248159 24572 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:33.244906 24907 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.202693 21797 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEARNER]: Advancing to term 8
I20260709 13:07:33.198992 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:33.225059 24851 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:33.196259 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 533671us
W20260709 13:07:33.252770 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 53852.5us
I20260709 13:07:33.245281 24662 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:33.203605 24478 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:33.200217 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1130) took 1439 ms. Trace:
I20260709 13:07:33.190831 21800 rpcz_store.cc:276] 0709 13:07:31.863049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.864884 (+  1835us) service_pool.cc:224] Handling call
0709 13:07:31.887298 (+ 22414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:32.269476 (+382178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.027400 (+757924us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54004b7dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:33.027606 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":873991424}
I20260709 13:07:33.253517 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:33.246464 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:33.253643 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:33.260413 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:33.261019 21071 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.258785 21219 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:33.263355 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:33.261946 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.266480 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:33.267122 21147 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:33.267412 21064 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:07:33.268148 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:33.272092 21072 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:33.276580 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:33.270527 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:33.279525 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:33.256729 21068 rpcz_store.cc:276] 0709 13:07:31.760817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.805933 (+ 45116us) service_pool.cc:224] Handling call
0709 13:07:31.806206 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:31.806388 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:31.819024 (+ 12636us) log.cc:844] Serialized 14 byte log entry
0709 13:07:31.819516 (+   492us) log.cc:844] Serialized 14 byte log entry
0709 13:07:32.506413 (+686897us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b18003a30c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f420d2d70 00007f7f420c6293 00007f7f420bbcea 00007f7f420bc23a 00007f7f42526a2a 00007f7f425360bf 00007f7f42536038
0709 13:07:33.188730 (+682317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:33.188818 (+    88us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:33.188880 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.188975 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:33.200143 (+ 11168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":694247,"prepare.queue_time_us":86541,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":2598,"raft.queue_time_us":682555,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":43433,"spinlock_wait_cycles":11108096}
I20260709 13:07:33.200839 24915 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:33.295893 24915 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:33.296984 21214 pending_rounds.cc:85] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 6
I20260709 13:07:33.297330 21214 pending_rounds.cc:107] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:07:33.297801 21214 raft_consensus.cc:2889] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:33.298430 20970 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 164) took 22287 ms (client timeout 15000 ms). Trace:
I20260709 13:07:33.289811 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:33.299304 21068 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:07:33.298473 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:33.294245 24851 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.291054 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:33.218778 24898 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.297259 24915 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:33.302176 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:33.304445 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:33.263574 24663 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:33.263317 24900 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:33.260188 24553 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:33.301373 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:33.306370 21138 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:33.306764 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:33.311998 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:33.324901 21212 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:33.325335 21139 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:33.314316 21147 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:33.316638 21141 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:33.317328 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:33.335572 21067 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:33.264492 24947 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:33.276237 24818 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:33.344149 21069 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:33.355877 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:33.357159 21064 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:33.305218 24588 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:33.370285 21146 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 15. Ops: [13.11-13.11]
I20260709 13:07:33.374464 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:33.371570 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:33.376966 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:33.378368 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 80010.2us
I20260709 13:07:33.277179 24606 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:07:33.370894 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:33.382778 21067 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:33.378191 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:33.387074 21147 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:33.390821 21139 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:07:33.386092 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:33.339161 24553 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.397967 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:33.398980 21069 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 5.
I20260709 13:07:33.399715 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:33.412189 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:33.402693 21064 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:33.412875 21796 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:33.373468 24663 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:33.414381 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:33.417073 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:33.365274 24900 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:33.372565 24588 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.385403 24763 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:33.429311 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:33.435823 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:33.400334 24553 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:33.420768 24892 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:33.477276 21071 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:33.480669 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:33.439546 21800 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:33.486099 21796 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
W20260709 13:07:33.298817 20970 rpcz_store.cc:269] 0709 13:07:11.103486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.120389 (+ 16903us) service_pool.cc:224] Handling call
0709 13:07:11.965686 (+845297us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b540039d5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42486090 00007f7f42524f3a 00007f7f482db1cc 00007f7f483d937e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc 00007f7f46899392 00007f7f4085748c 00007f7f408568c6
0709 13:07:33.298150 (+21332464us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:30.207849 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:30.218913 (+ 11064us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:30.218958 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:07:30.218980 (+    22us) tablet.cc:662] Decoding operations
0709 13:07:30.246366 (+ 27386us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:30.246454 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:30.246509 (+    55us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:07:30.247334 (+   825us) tablet.cc:701] Row locks acquired
0709 13:07:30.247352 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:07:30.247553 (+   201us) write_op.cc:270] Start()
0709 13:07:30.247708 (+   155us) write_op.cc:276] Timestamp: P: 1783602450247494 usec, L: 0
0709 13:07:30.247729 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:07:30.262527 (+ 14798us) log.cc:844] Serialized 4270 byte log entry
0709 13:07:31.080509 (+817982us) op_driver.cc:464] REPLICATION: finished
0709 13:07:31.436016 (+355507us) write_op.cc:301] APPLY: starting
0709 13:07:31.436127 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:31.436449 (+   322us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:31.436578 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:31.438032 (+  1454us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:31.438395 (+   363us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:07:31.438445 (+    50us) write_op.cc:312] APPLY: finished
0709 13:07:31.542934 (+104489us) log.cc:844] Serialized 77 byte log entry
0709 13:07:31.587904 (+ 44970us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:31.603061 (+ 15157us) write_op.cc:454] Released schema lock
0709 13:07:31.603127 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:31.652803 (+ 49676us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:31.656017 (+  3214us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":768247,"reported_tablets":20,"spinlock_wait_cycles":404041472,"child_traces":[["op",{"apply.queue_time_us":355125,"apply.run_cpu_time_us":4536,"apply.run_wall_time_us":169153,"num_ops":10,"prepare.queue_time_us":659984,"prepare.run_cpu_time_us":21431,"prepare.run_wall_time_us":238372,"raft.queue_time_us":521143,"raft.run_cpu_time_us":26420,"raft.run_wall_time_us":179751,"replication_time_us":783294,"thread_start_us":900030,"threads_started":4,"wal-append.queue_time_us":192873,"wal-append.run_cpu_time_us":32319,"wal-append.run_wall_time_us":2106603}]]}
I20260709 13:07:33.431017 24447 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:33.439393 24588 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:33.436551 24942 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:33.485837 24826 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:33.507880 24932 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.512784 24932 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.514863 24932 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:33.516816 21797 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:33.519487 24194 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:33.512212 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:33.438292 24906 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:07:33.482749 24663 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:33.482848 24892 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:33.578847 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 174831us
I20260709 13:07:33.496454 24907 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:33.535686 21185 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:33.584173 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48611.3us
I20260709 13:07:33.405014 24662 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:33.531437 21146 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:33.537967 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:33.496981 24945 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:33.617260 21064 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:07:33.629827 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 91966.1us
I20260709 13:07:33.509050 24942 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:33.560381 24194 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:33.650074 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.582845 24892 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:33.635687 24934 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:07:33.601529 24947 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:33.602982 24945 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:33.686600 24945 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:33.688160 24945 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:33.692041 24950 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:07:33.654081 24942 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:33.676632 21064 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:33.640868 24606 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:33.678172 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:33.708406 24606 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.708987 24606 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:33.712267 24663 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:33.683867 24194 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.601267 24941 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:07:33.716948 24941 consensus_queue.cc:1243] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:33.717594 24941 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:33.717973 24941 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:33.685207 24947 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.719767 24947 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:33.628437 24226 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:07:33.725484 24553 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:33.718963 24941 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:33.740731 21065 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:33.742647 24549 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.000s
I20260709 13:07:33.642153 24900 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:33.745162 24953 tablet_replica.cc:333] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:07:33.663691 24761 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:07:33.708443 21064 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:33.793061 21065 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:07:33.802762 21142 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:07:33.768296 21072 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:33.773464 21067 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:33.776186 21212 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:33.777329 21068 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
W20260709 13:07:33.787142 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 109076us
W20260709 13:07:33.789952 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.765399 24927 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:33.766449 24917 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:33.923779 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.946321 21072 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:33.955335 21068 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:33.903760 24941 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:33.994884 24958 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:07:33.794620 24549 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:33.944105 24478 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:34.009608 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45154.1us
W20260709 13:07:34.001935 24915 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.664s	user 0.005s	sys 0.007s
I20260709 13:07:33.994053 24814 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:07:34.010814 24911 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.761s	user 0.007s	sys 0.007s
W20260709 13:07:34.221113 24662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.038s
W20260709 13:07:34.113555 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.002s	sys 0.041s
W20260709 13:07:34.389163 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.009s	sys 0.041s
W20260709 13:07:34.058660 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:34.307523 21063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.047s	sys 0.039s
W20260709 13:07:34.390559 21063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.048s	sys 0.040s
W20260709 13:07:34.390666 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:34.391232 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.392534 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1152) took 1121 ms. Trace:
W20260709 13:07:34.221994 24930 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.005s	sys 0.033s
I20260709 13:07:34.350363 21036 leader_election.cc:400] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
W20260709 13:07:34.357910 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.007s	sys 0.023s
W20260709 13:07:34.337378 24952 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.327s	user 0.005s	sys 0.004s
W20260709 13:07:34.344676 24942 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.648s	user 0.024s	sys 0.003s
I20260709 13:07:34.345400 24818 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.987249 24917 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:34.375808 21800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.009s	sys 0.015s
W20260709 13:07:34.417672 21800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.009s	sys 0.015s
I20260709 13:07:34.018786 24478 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:34.375542 21145 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.010s	sys 0.016s
W20260709 13:07:34.403635 24947 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.683s	user 0.013s	sys 0.000s
I20260709 13:07:34.351581 24763 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:34.397583 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.008s	sys 0.023s
W20260709 13:07:34.434506 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 1.13312e+06us
I20260709 13:07:34.396977 24831 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:07:34.396607 21138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.002s	sys 0.066s
W20260709 13:07:34.438388 21138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.002s	sys 0.066s
I20260709 13:07:34.398658 21036 leader_election.cc:403] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 13:07:34.388914 24662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.000s	sys 0.040s
W20260709 13:07:34.424728 21145 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.010s	sys 0.016s
W20260709 13:07:34.373929 24851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.012s	sys 0.032s
I20260709 13:07:34.374997 24814 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:34.406414 24194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.023s	sys 0.008s
W20260709 13:07:34.397265 24930 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.005s	sys 0.034s
W20260709 13:07:34.439760 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 126579us
W20260709 13:07:34.440526 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.443758 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2269) took 1093 ms. Trace:
W20260709 13:07:34.442665 24851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.013s	sys 0.033s
W20260709 13:07:34.447067 24194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.024s	sys 0.008s
W20260709 13:07:34.457183 24966 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:07:34.457026 24549 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:34.439636 24553 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:34.425860 24818 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:34.440089 24947 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:34.466153 24814 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:34.471664 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2039) took 1175 ms. Trace:
I20260709 13:07:34.471889 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2271) took 1101 ms. Trace:
I20260709 13:07:34.474454 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2047) took 1073 ms. Trace:
I20260709 13:07:34.475467 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2809) took 1540 ms. Trace:
I20260709 13:07:34.417778 24942 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:34.492326 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.479569 21065 rpcz_store.cc:276] 0709 13:07:33.401083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.802430 (+401347us) service_pool.cc:224] Handling call
0709 13:07:33.802741 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:33.802931 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:33.946276 (+143345us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.473900 (+527624us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:34.473981 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:34.474082 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.474200 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.474371 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":146434,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":44,"raft.queue_time_us":20580,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5200,"thread_start_us":525200,"threads_started":2}
I20260709 13:07:34.511552 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2268) took 1203 ms. Trace:
I20260709 13:07:34.447551 24927 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:34.478874 21066 rpcz_store.cc:276] 0709 13:07:33.296694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.436374 (+139680us) service_pool.cc:224] Handling call
0709 13:07:33.485323 (+ 48949us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.495204 (+  9881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.495249 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:33.496104 (+   855us) log.cc:844] Serialized 26 byte log entry
0709 13:07:33.794787 (+298683us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:33.794843 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.794931 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.471385 (+676454us) raft_consensus.cc:1685] finished
0709 13:07:34.471405 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.471570 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":284,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":58942,"thread_start_us":278012,"threads_started":1,"wal-append.queue_time_us":917537}
I20260709 13:07:34.459501 24662 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.477155 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:34.479449 21147 rpcz_store.cc:276] 0709 13:07:33.370979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.397833 (+ 26854us) service_pool.cc:224] Handling call
0709 13:07:33.425194 (+ 27361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.902972 (+477778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.903003 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:33.954663 (+ 51660us) log.cc:844] Serialized 26 byte log entry
0709 13:07:34.368046 (+413383us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:34.368095 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.368182 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.471518 (+103336us) raft_consensus.cc:1685] finished
0709 13:07:34.471570 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.471799 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36697,"raft.queue_time_us":69673,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":678753,"threads_started":2,"wal-append.queue_time_us":465812}
I20260709 13:07:34.461416 24911 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:34.507808 21145 rpcz_store.cc:276] 0709 13:07:32.935272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.950324 (+ 15052us) service_pool.cc:224] Handling call
0709 13:07:33.188034 (+237710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.188560 (+   526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.188589 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:33.204132 (+ 15543us) log.cc:844] Serialized 26 byte log entry
0709 13:07:33.490494 (+286362us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:34.439741 (+949247us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.449090 (+  9349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.449161 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.475128 (+ 25967us) raft_consensus.cc:1685] finished
0709 13:07:34.475153 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.475371 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9081,"prepare.queue_time_us":587088,"prepare.run_cpu_time_us":9149,"prepare.run_wall_time_us":238286,"raft.queue_time_us":10943,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":7075,"spinlock_wait_cycles":10269440,"thread_start_us":1143681,"threads_started":2,"wal-append.queue_time_us":727495}
W20260709 13:07:34.498822 24606 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.770s	user 0.009s	sys 0.004s
W20260709 13:07:34.540899 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.443460 24763 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:34.512008 21144 rpcz_store.cc:276] 0709 13:07:33.307989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.341957 (+ 33968us) service_pool.cc:224] Handling call
0709 13:07:33.376322 (+ 34365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.394862 (+ 18540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.394893 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:33.415359 (+ 20466us) log.cc:844] Serialized 26 byte log entry
0709 13:07:33.602271 (+186912us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:33.602310 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.602375 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.511233 (+908858us) raft_consensus.cc:1685] finished
0709 13:07:34.511259 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.511475 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":61835,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":2286,"thread_start_us":161671,"threads_started":1,"wal-append.queue_time_us":572209}
W20260709 13:07:34.587882 21143 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.047s	sys 0.079s
W20260709 13:07:34.608932 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 150105us
W20260709 13:07:34.759361 21143 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.048s	sys 0.079s
W20260709 13:07:34.828514 24981 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.240s	user 0.000s	sys 0.000s
I20260709 13:07:34.829465 24763 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:34.830085 24763 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:34.830166 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 412) took 1290 ms. Trace:
I20260709 13:07:34.830850 21800 rpcz_store.cc:276] 0709 13:07:33.539443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.539604 (+   161us) service_pool.cc:224] Handling call
0709 13:07:33.594909 (+ 55305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.595319 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.595364 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:33.603457 (+  8093us) log.cc:844] Serialized 26 byte log entry
0709 13:07:34.418167 (+814710us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:34.418207 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.418273 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.829896 (+411623us) raft_consensus.cc:1685] finished
0709 13:07:34.829927 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.830094 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2885,"raft.queue_time_us":4620,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":34450,"thread_start_us":740841,"threads_started":1,"wal-append.queue_time_us":825468}
I20260709 13:07:34.830953 24763 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:34.833499 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:34.837713 24949 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.354s	user 0.000s	sys 0.018s
I20260709 13:07:34.841405 24898 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:34.834200 24763 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:34.856488 24763 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:34.863812 24917 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:34.865778 24961 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.386s	user 0.001s	sys 0.000s
I20260709 13:07:34.867233 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1149) took 1612 ms. Trace:
I20260709 13:07:34.870401 21145 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:07:34.834390 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 456) took 1201 ms. Trace:
I20260709 13:07:34.876812 21144 rpcz_store.cc:276] 0709 13:07:33.633470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.817862 (+1184392us) service_pool.cc:224] Handling call
0709 13:07:34.834323 (+ 16461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:34.853322 21069 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:07:34.809309 24892 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.130s	user 0.009s	sys 0.003s
I20260709 13:07:34.882190 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2041) took 1584 ms. Trace:
I20260709 13:07:34.920235 21069 rpcz_store.cc:276] 0709 13:07:33.297282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.852518 (+1555236us) service_pool.cc:224] Handling call
0709 13:07:34.852942 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:34.881759 (+ 28817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.882107 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:34.924152 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2808) took 1989 ms. Trace:
I20260709 13:07:34.758420 24851 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:34.937840 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2276) took 1386 ms. Trace:
I20260709 13:07:34.938383 21145 rpcz_store.cc:276] 0709 13:07:33.533362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.821765 (+1288403us) service_pool.cc:224] Handling call
0709 13:07:34.870112 (+ 48347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:34.919058 (+ 48946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.919434 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:34.867745 21070 rpcz_store.cc:276] 0709 13:07:33.254781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.255737 (+   956us) service_pool.cc:224] Handling call
0709 13:07:33.269165 (+ 13428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.282929 (+ 13764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.282963 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:33.304481 (+ 21518us) log.cc:844] Serialized 26 byte log entry
0709 13:07:33.770109 (+465628us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:33.770168 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.770242 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.866938 (+1096696us) raft_consensus.cc:1685] finished
0709 13:07:34.866964 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.867170 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34025,"raft.queue_time_us":29748,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":15517,"thread_start_us":391200,"threads_started":1,"wal-append.queue_time_us":1091368}
I20260709 13:07:34.809290 24553 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:34.835011 24968 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.383s	user 0.000s	sys 0.001s
I20260709 13:07:34.924757 21140 rpcz_store.cc:276] 0709 13:07:32.934975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.940751 (+  5776us) service_pool.cc:224] Handling call
0709 13:07:32.941073 (+   322us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:32.941421 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:32.941450 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:32.942008 (+   558us) log.cc:844] Serialized 52 byte log entry
0709 13:07:33.304463 (+362455us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:33.561334 (+256871us) log.cc:844] Serialized 14 byte log entry
0709 13:07:33.581968 (+ 20634us) log.cc:844] Serialized 14 byte log entry
0709 13:07:33.582194 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.582492 (+   298us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.837056 (+1254564us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:34.923807 (+ 86751us) raft_consensus.cc:1685] finished
0709 13:07:34.923851 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.924070 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28215,"prepare.queue_time_us":1379830,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":100,"raft.queue_time_us":25305,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":18048,"spinlock_wait_cycles":47820416,"thread_start_us":570659,"threads_started":2,"wal-append.queue_time_us":459775}
I20260709 13:07:34.824575 24814 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:34.948127 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 366852us
I20260709 13:07:34.952930 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 455) took 1321 ms. Trace:
I20260709 13:07:34.953377 21145 rpcz_store.cc:276] 0709 13:07:33.631246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.939441 (+1308195us) service_pool.cc:224] Handling call
0709 13:07:34.952394 (+ 12953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:34.952592 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.952620 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:34.952663 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:34.952701 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.952759 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.952873 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:34.607483 24662 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:34.950616 24851 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:34.807933 24606 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:34.958218 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:34.959736 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 482645us
I20260709 13:07:34.758931 24911 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:34.961426 24991 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.962445 24662 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:34.967962 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2277) took 1324 ms. Trace:
I20260709 13:07:34.983362 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:34.957463 24588 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.424s	user 0.010s	sys 0.003s
I20260709 13:07:34.954983 24971 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:34.955102 24982 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:07:34.983830 21140 rpcz_store.cc:276] 0709 13:07:33.643323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.952109 (+1308786us) service_pool.cc:224] Handling call
0709 13:07:34.967447 (+ 15338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:34.967633 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.967666 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:34.967712 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:34.967752 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.967805 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.967906 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:34.988771 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:34.985957 24763 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:34.991523 24942 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:34.992141 24942 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.992568 24942 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:34.992735 24606 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:34.987969 24588 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:34.993448 24971 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:34.994694 24942 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:34.995609 24927 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:34.995779 24971 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:34.996040 24927 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:34.996484 24927 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:35.001561 24983 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:07:34.937176 24892 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:34.969051 24945 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.280s	user 0.013s	sys 0.002s
I20260709 13:07:35.010456 24930 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.011044 24930 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.011463 24662 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:35.012131 24662 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.013441 24930 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.014127 24930 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.014617 24662 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:34.995227 24942 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:35.023649 24942 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.007944 24763 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:35.025018 24763 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.025931 24942 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.007669 24971 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.026804 24971 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:35.018193 24962 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.071s	user 0.010s	sys 0.009s
I20260709 13:07:35.028724 24588 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.029126 24971 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:34.955401 24553 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.040179 21140 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:35.043953 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2279) took 1285 ms. Trace:
I20260709 13:07:35.044425 21140 rpcz_store.cc:276] 0709 13:07:33.758198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.989029 (+1230831us) service_pool.cc:224] Handling call
0709 13:07:35.039733 (+ 50704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:35.043592 (+  3859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.043876 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:35.034463 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 454) took 1403 ms. Trace:
I20260709 13:07:35.030818 24763 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.046566 21144 rpcz_store.cc:276] 0709 13:07:33.630886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.877963 (+1247077us) service_pool.cc:224] Handling call
0709 13:07:34.940683 (+ 62720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:35.031618 (+ 90935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:35.031648 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:35.031706 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:35.031736 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.034144 (+  2408us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b540032c8d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:35.034213 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.034381 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208601472}
W20260709 13:07:35.035214 24988 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:07:35.003257 24478 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.105182 24478 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:35.105826 24478 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.108018 24478 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:35.109508 24941 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.102s	user 0.011s	sys 0.000s
I20260709 13:07:35.111887 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2272) took 1726 ms. Trace:
I20260709 13:07:35.112290 21139 rpcz_store.cc:276] 0709 13:07:33.385631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.391705 (+  6074us) service_pool.cc:224] Handling call
0709 13:07:33.395587 (+  3882us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:33.446464 (+ 50877us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.446498 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:33.479119 (+ 32621us) log.cc:844] Serialized 52 byte log entry
0709 13:07:33.786317 (+307198us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:34.455222 (+668905us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.583732 (+128510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.583807 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:35.001455 (+417648us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b48001a01d8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:07:35.111473 (+110018us) raft_consensus.cc:1685] finished
0709 13:07:35.111678 (+   205us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.111824 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":253867,"prepare.queue_time_us":637759,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":132364,"spinlock_wait_cycles":14684928,"thread_start_us":887346,"threads_started":3,"wal-append.queue_time_us":881012}
I20260709 13:07:35.007321 24991 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.114513 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2264) took 1812 ms. Trace:
I20260709 13:07:35.114928 21142 rpcz_store.cc:276] 0709 13:07:33.302585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.341915 (+ 39330us) service_pool.cc:224] Handling call
0709 13:07:33.376607 (+ 34692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.803219 (+426612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.114266 (+1311047us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54003be554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:35.114456 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979544448}
I20260709 13:07:35.115923 24991 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.036585 24892 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.032864 24911 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.034123 24996 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:34.393041 21063 rpcz_store.cc:276] 0709 13:07:33.270620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.270884 (+   264us) service_pool.cc:224] Handling call
0709 13:07:33.287914 (+ 17030us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:33.288123 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:33.444273 (+156150us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.391050 (+946777us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:34.391111 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:34.391166 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.391270 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.391423 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":387260,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":2582,"thread_start_us":1096685,"threads_started":2}
I20260709 13:07:35.122839 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:34.448282 21138 rpcz_store.cc:276] 0709 13:07:33.350368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.350687 (+   319us) service_pool.cc:224] Handling call
0709 13:07:33.351022 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:33.351232 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:33.795322 (+444090us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.442916 (+647594us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.443206 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.443394 (+   188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:34.443441 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:34.443493 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.443561 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.443696 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3714,"prepare.queue_time_us":652571,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":1275,"raft.queue_time_us":15438,"thread_start_us":1050632,"threads_started":2}
I20260709 13:07:35.124832 21070 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:35.124714 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.125552 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.130824 21064 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:35.126245 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 455) took 1877 ms. Trace:
I20260709 13:07:35.038429 24553 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.125555 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.033789 24851 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.133620 21067 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:35.135073 21066 raft_consensus.cc:2410] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:07:35.135447 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.131522 21213 rpcz_store.cc:276] 0709 13:07:33.248339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.248450 (+   111us) service_pool.cc:224] Handling call
0709 13:07:33.248709 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:33.250948 (+  2239us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:33.372299 (+121351us) log.cc:844] Serialized 14 byte log entry
0709 13:07:33.531597 (+159298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:33.531725 (+   128us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:33.531806 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.017630 (+1485824us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:35.126016 (+108386us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b540013c254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:35.126067 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.126191 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5232,"prepare.queue_time_us":264344,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":137,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":9869,"spinlock_wait_cycles":107154560,"thread_start_us":279771,"threads_started":2,"wal-append.queue_time_us":405349}
I20260709 13:07:35.100860 24947 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.146039 21069 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:35.050268 24930 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.145064 24892 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.137866 24996 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.174576 24953 raft_consensus.cc:2243] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEARNER]: Raft consensus shutting down.
I20260709 13:07:35.177789 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.179702 21066 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
W20260709 13:07:35.182443 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8136973070000977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:07:35.165736 24947 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:35.176157 24941 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:35.191027 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.180641 24851 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:07:35.188728 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:35.189687 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.175338 24892 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.185752 24766 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:35.212574 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.223992 25009 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.224475 21064 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:07:35.183053 21036 connection.cc:256] RPC call timeout handler was delayed by -1.7623973070000147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 13:07:35.232944 21036 connection.cc:256] RPC call timeout handler was delayed by -1.7227973070000644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
I20260709 13:07:35.233085 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:35.233292 21036 connection.cc:256] RPC call timeout handler was delayed by -1.6984973070000251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
W20260709 13:07:35.233620 21036 connection.cc:256] RPC call timeout handler was delayed by -1.6480973070000573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 13:07:35.233912 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5760973070000546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
I20260709 13:07:35.224763 21065 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:35.225035 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:35.240453 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5643973070000357s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
I20260709 13:07:35.205798 24953 pending_rounds.cc:70] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Trying to abort 1 pending ops.
I20260709 13:07:35.226652 25010 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.226890 21070 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:35.228214 25011 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:35.228494 21063 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:35.230311 25012 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:35.229179 25013 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:35.229799 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:35.233912 21064 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:07:35.243975 24814 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:35.234508 25009 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.235949 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2274) took 1790 ms. Trace:
I20260709 13:07:35.241007 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 452) took 1978 ms. Trace:
W20260709 13:07:35.240979 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5418973069999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1834s for a response
I20260709 13:07:35.225298 25002 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.205267 24995 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:35.244946 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:35.262926 21065 raft_consensus.cc:2376] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6. Re-sending same reply.
I20260709 13:07:35.206372 24941 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.228427 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.268605 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.252522 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:35.269755 21142 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:35.224628 25007 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:35.259636 21143 rpcz_store.cc:276] 0709 13:07:33.265640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.265778 (+   138us) service_pool.cc:224] Handling call
0709 13:07:33.288277 (+ 22499us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 13:07:33.341744 (+ 53467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.341793 (+    49us) raft_consensus.cc:1539] Triggering prepare for 5 ops
0709 13:07:33.416845 (+ 75052us) log.cc:844] Serialized 130 byte log entry
0709 13:07:33.801414 (+384569us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:34.759976 (+958562us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.817920 (+ 57944us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.818264 (+   344us) log.cc:844] Serialized 14 byte log entry
0709 13:07:35.166634 (+348370us) log.cc:844] Serialized 14 byte log entry
0709 13:07:35.184064 (+ 17430us) log.cc:844] Serialized 14 byte log entry
0709 13:07:35.240513 (+ 56449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.240665 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:35.240726 (+    61us) raft_consensus.cc:1685] finished
0709 13:07:35.240745 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.240935 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102701,"prepare.queue_time_us":625056,"prepare.run_cpu_time_us":209,"prepare.run_wall_time_us":1014,"raft.queue_time_us":99419,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":39511,"spinlock_wait_cycles":10227328,"thread_start_us":1531722,"threads_started":3,"wal-append.queue_time_us":1390105}
W20260709 13:07:35.259833 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4725973070000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
I20260709 13:07:35.264200 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.246207 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:35.233209 24945 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:35.242398 24588 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.239599 25000 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:35.203735 24947 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.269277 21064 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
W20260709 13:07:35.272115 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4293973070000447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
I20260709 13:07:35.208416 24851 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.257728 24994 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:35.275584 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.283552 21070 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:35.277405 25014 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.279904 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.306042 21072 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:35.306285 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.255409 24998 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:35.275143 21065 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:35.241856 24766 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:35.280608 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3825973070000828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
W20260709 13:07:35.310894 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3249973070001033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
W20260709 13:07:35.311323 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3114973070000815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
I20260709 13:07:35.311091 21070 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:35.280936 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.273708 24995 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.258060 21146 rpcz_store.cc:276] 0709 13:07:33.445927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.638081 (+192154us) service_pool.cc:224] Handling call
0709 13:07:33.638514 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:33.659819 (+ 21305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.659865 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:33.705327 (+ 45462us) log.cc:844] Serialized 26 byte log entry
0709 13:07:35.103688 (+1398361us) spinlock_profiling.cc:243] Waited 51.5 ms on lock 0x7b1800420b20. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:07:35.223576 (+119888us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:35.223629 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.223714 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:35.235693 (+ 11979us) raft_consensus.cc:1685] finished
0709 13:07:35.235724 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.235905 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5242,"raft.queue_time_us":20947,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":13612,"spinlock_wait_cycles":118349184,"thread_start_us":119664,"threads_started":1,"wal-append.queue_time_us":1528583}
I20260709 13:07:35.282714 25000 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.312419 21064 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:35.284956 24989 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:35.312435 25014 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.278914 24945 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.289599 24851 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.257843 25012 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.293123 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.263262 24814 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:35.279121 24976 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.176s	user 0.002s	sys 0.000s
I20260709 13:07:35.311388 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.318372 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:35.311900 21036 connection.cc:256] RPC call timeout handler was delayed by -1.280897307000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21240000000000003s for a response
W20260709 13:07:35.319142 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2637973070000044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 13:07:35.283131 24930 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.315868 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:35.320947 21070 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:07:35.242866 24953 pending_rounds.cc:77] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7305635428880179200 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } } new_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } } }
W20260709 13:07:35.319657 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2565973070001064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 13:07:35.253508 25010 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.318693 21065 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
W20260709 13:07:35.321964 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2385973070000773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
W20260709 13:07:35.322394 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2007973070000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 13:07:35.322702 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1890973069999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
W20260709 13:07:35.322892 21036 connection.cc:256] RPC call timeout handler was delayed by -1.044197307000104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23870000000000002s for a response
W20260709 13:07:35.323060 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0324973070000851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24s for a response
I20260709 13:07:35.304704 24816 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.257323 25011 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:35.428113 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51683.4us
I20260709 13:07:35.429697 24930 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:35.531069 21800 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:35.298955 24994 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:35.570119 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 339592us
I20260709 13:07:35.315189 24998 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.295264 24927 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.271255 25002 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.281215 24588 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.323946 24953 raft_consensus.cc:2917] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEARNER]: Aborting config change with OpId 1.2: Aborted: Op aborted
I20260709 13:07:35.566890 21070 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:35.482095 24850 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:35.276149 25007 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.619154 21036 leader_election.cc:400] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:35.627655 21036 leader_election.cc:403] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:35.627976 21071 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:07:35.620885 21218 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:35.467218 25010 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.313416 24989 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.377084 25013 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:35.469609 24996 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.628929 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2040) took 2332 ms. Trace:
I20260709 13:07:35.469106 24945 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.457944 24947 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.456135 24892 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.465111 24814 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.454480 24995 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.474009 25011 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.634651 21071 rpcz_store.cc:276] 0709 13:07:33.296982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.627141 (+2330159us) service_pool.cc:224] Handling call
0709 13:07:35.627663 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:35.628560 (+   897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.628851 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:35.613107 24763 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:35.649005 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.623118 25000 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:35.662994 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:35.664045 21187 connection.cc:256] RPC call timeout handler was delayed by -1.4339478179999787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 13:07:35.664533 21187 connection.cc:256] RPC call timeout handler was delayed by -1.4254978459999847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 13:07:35.427186 24816 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.634073 24953 raft_consensus.cc:2889] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 8 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:07:35.635805 25013 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:35.648732 24945 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.649251 24996 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:35.652959 25016 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.676614 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:35.652655 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:35.678887 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.648238 25009 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.628400 25012 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.672787 24953 raft_consensus.cc:2272] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 8 NON_PARTICIPANT]: Raft consensus is shut down!
W20260709 13:07:35.665156 21187 connection.cc:256] RPC call timeout handler was delayed by -1.1477478179999707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
W20260709 13:07:35.651659 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 376959us
I20260709 13:07:35.681872 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.690646 21071 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:35.692644 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:35.665988 24763 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.657639 24892 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:07:35.675465 25003 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.633234 24850 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.679324 24947 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:35.715386 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.716831 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.717718 21219 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:35.717764 21068 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:35.719873 21217 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:35.722497 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.723215 21071 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:35.724133 25002 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.724392 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.725062 21068 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:35.732097 21218 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:35.732671 21072 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:35.733076 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 448) took 2837 ms. Trace:
I20260709 13:07:35.733510 21218 rpcz_store.cc:276] 0709 13:07:32.895401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.895508 (+   107us) service_pool.cc:224] Handling call
0709 13:07:32.915131 (+ 19623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:33.241547 (+326416us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b180017d0c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42557646 00007f7f4249158e 00007f7f42512f9e 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4
0709 13:07:35.620404 (+2378857us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b58008c0fd0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f424780aa 00007f7f424fa447 00007f7f42512fe7 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4
0709 13:07:35.732733 (+112329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.733014 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4718481408}
I20260709 13:07:35.678146 24945 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.707326 24941 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.678653 24996 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.745292 25023 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.729715 24947 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.752223 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:35.680763 25016 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.756100 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 457) took 1991 ms. Trace:
I20260709 13:07:35.877924 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.854686 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:35.880301 21065 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
W20260709 13:07:35.754789 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 144230us
I20260709 13:07:35.878196 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:35.882751 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:35.877981 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.883641 21801 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:35.883699 21072 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:35.712769 25003 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.753057 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:35.879487 21140 rpcz_store.cc:276] 0709 13:07:33.764307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.045440 (+1281133us) service_pool.cc:224] Handling call
0709 13:07:35.083019 (+ 37579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:35.114923 (+ 31904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:35.114951 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:35.115429 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:07:35.254567 (+139138us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:35.254629 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.755541 (+500912us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b54002963d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:35.755614 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:35.755682 (+    68us) raft_consensus.cc:1685] finished
0709 13:07:35.755702 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.756021 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":669382,"raft.queue_time_us":1188,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":695933,"spinlock_wait_cycles":72467712,"thread_start_us":138646,"threads_started":1,"wal-append.queue_time_us":161637}
W20260709 13:07:35.886644 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 194133us
I20260709 13:07:35.891975 21795 raft_consensus.cc:2410] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:07:35.754045 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:35.734236 25015 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.720420 25013 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.898097 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:35.901388 21219 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:35.900736 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:35.878098 24763 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.844710 24995 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:35.785665 24941 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.903599 25025 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.903646 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:35.901818 25015 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.904572 21220 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:07:35.901510 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 284551us
I20260709 13:07:35.905375 24945 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.866371 24927 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:35.903945 24941 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.906607 25013 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:35.907363 25009 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:35.907981 24996 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.909235 24850 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.910972 21217 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:35.748627 25022 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.843657 24930 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:36.183779 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.094838 24850 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:35.914534 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:35.975775 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64215.2us
I20260709 13:07:36.218152 24850 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:36.217267 25022 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:35.915544 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:36.218914 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:35.976075 24996 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.244s
I20260709 13:07:36.057567 25016 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:36.055204 24818 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.921110 25025 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.959983 25007 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:35.941605 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58923.1us
I20260709 13:07:36.223584 25025 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:36.225135 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:36.225391 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:36.225893 24953 ts_tablet_manager.cc:1916] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:36.226001 25014 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:36.226035 21214 raft_consensus.cc:2376] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8. Re-sending same reply.
I20260709 13:07:35.919750 24998 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:35.960455 25026 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:36.231225 24977 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.229s	user 0.000s	sys 0.010s
I20260709 13:07:36.238240 25031 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:36.234192 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2278) took 2498 ms. Trace:
I20260709 13:07:36.240229 24945 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:36.244020 24999 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:36.236588 24994 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:36.246035 25033 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:07:36.237535 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 989461us
W20260709 13:07:36.237833 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.238058 25019 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.248853 25019 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:35.974061 25003 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:36.076910 21793 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:35.975303 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:36.058893 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 101114us
W20260709 13:07:36.087042 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:36.277279 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:36.278199 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.965014 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:35.912842 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:36.287987 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:36.239601 21145 rpcz_store.cc:276] 0709 13:07:33.735452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.954173 (+1218721us) service_pool.cc:224] Handling call
0709 13:07:34.957103 (+  2930us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:34.987727 (+ 30624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.987811 (+    84us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:35.006813 (+ 19002us) log.cc:844] Serialized 52 byte log entry
0709 13:07:35.209220 (+202407us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:35.244578 (+ 35358us) log.cc:844] Serialized 14 byte log entry
0709 13:07:36.233554 (+988976us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b1800091cc0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287 00007f7f42537208
0709 13:07:36.233733 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:36.233825 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:36.233895 (+    70us) raft_consensus.cc:1685] finished
0709 13:07:36.233983 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:36.234131 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":741326,"prepare.queue_time_us":39611,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":4358,"raft.queue_time_us":1028943,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":772276,"spinlock_wait_cycles":3783424,"thread_start_us":112270,"threads_started":1,"wal-append.queue_time_us":182896}
I20260709 13:07:36.290601 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:36.225571 21036 leader_election.cc:400] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:36.294158 21036 leader_election.cc:403] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:07:36.295007 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:35.786758 24766 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:36.222116 25022 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:36.295922 21219 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:36.288002 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:36.305984 21072 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:36.251091 25019 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:36.235106 25000 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:36.081172 24941 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:35.900682 25027 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:36.298851 24663 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.335639 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.312820 24763 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.920924 24907 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.399s
I20260709 13:07:36.344272 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:36.348738 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 109131us
I20260709 13:07:36.349486 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:36.313657 24814 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:35.886467 25023 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:36.313470 25000 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:36.297088 24930 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:36.376086 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.376756 24930 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:36.350869 24763 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:36.355643 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.356133 20969 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 4 to 6, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 6 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.378661 25023 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:36.379060 24763 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:36.351158 24927 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:36.379540 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:36.356954 21221 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:36.380311 21064 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:36.359907 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2265) took 3057 ms. Trace:
I20260709 13:07:36.381131 21141 rpcz_store.cc:276] 0709 13:07:33.302907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.337474 (+ 34567us) service_pool.cc:224] Handling call
0709 13:07:33.379333 (+ 41859us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:33.786675 (+407342us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:07:35.036914 (+1250239us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b180043a0e0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f45f143c7 00007f7f425269a7 00007f7f425360bf
0709 13:07:35.037112 (+   198us) log.cc:844] Serialized 14 byte log entry
0709 13:07:35.181078 (+143966us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:35.181855 (+   777us) log.cc:844] Serialized 52 byte log entry
0709 13:07:35.181994 (+   139us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:35.184104 (+  2110us) log.cc:844] Serialized 14 byte log entry
0709 13:07:35.186077 (+  1973us) log.cc:844] Serialized 14 byte log entry
0709 13:07:35.186252 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.187760 (+  1508us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:36.054810 (+867050us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b180043a0e0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:36.359650 (+304840us) raft_consensus.cc:1685] finished
0709 13:07:36.359673 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:36.359818 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1923646,"prepare.queue_time_us":640856,"prepare.run_cpu_time_us":178,"prepare.run_wall_time_us":7078,"raft.queue_time_us":175333,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":660544,"spinlock_wait_cycles":70778880,"thread_start_us":902024,"threads_started":3,"wal-append.queue_time_us":167568}
I20260709 13:07:36.379361 20969 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 4 to 6, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 6 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.383529 24606 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.384124 24606 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:36.386312 24606 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
W20260709 13:07:36.364693 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 117284us
I20260709 13:07:36.387204 25037 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:36.388010 25037 consensus_queue.cc:1243] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:07:36.388613 24588 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:36.392746 25015 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:36.394711 25028 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.396914 24850 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:36.397558 24816 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:36.397006 20969 catalog_manager.cc:5697] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 2, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added, no longer has a pending config: VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193), VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194), VOTER 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195), NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 3 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:07:36.346824 24663 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:36.402822 24663 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:36.367792 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:36.372622 24973 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:36.374423 25000 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:36.334470 25016 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:07:36.420531 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:36.416918 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67515.4us
I20260709 13:07:36.353428 24814 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:36.441917 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.421569 20969 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 4 to 5, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 5 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.442245 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.442442 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:36.420017 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:36.446025 21138 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:36.447731 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.448395 21143 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:36.449389 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.450657 25026 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:36.451398 24831 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:36.452069 24831 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:36.452385 24524 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:36.452929 24524 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:36.454972 24524 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:36.455477 25036 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:36.456348 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:36.456775 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 89102.7us
I20260709 13:07:36.457145 21216 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:36.443555 21145 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:36.458529 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.459530 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.462348 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.450742 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.475909 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:36.476775 21212 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:36.476909 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.477535 21142 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:36.479774 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:36.403419 24851 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:36.484063 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.484422 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.484803 21797 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:36.485664 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:36.486248 21140 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:36.412613 24910 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.387072 24766 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:07:36.487917 21069 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:36.489735 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:36.424052 24549 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.441035 24663 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:36.456399 24814 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:36.576858 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.599898 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.718715 21797 raft_consensus.cc:2410] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:07:36.480669 24572 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:36.554091 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.463591 24816 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:07:36.568852 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 108481us
I20260709 13:07:36.610397 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.623889 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.480013 25019 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.139s
I20260709 13:07:36.480139 25022 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.139s
I20260709 13:07:36.480372 24898 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.139s
I20260709 13:07:36.405362 25028 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:36.415704 24973 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:36.484997 21794 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:36.420106 24907 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.492014 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.565850 24553 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.149s
I20260709 13:07:36.486845 24588 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.139s
I20260709 13:07:36.626888 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:36.801851 20969 catalog_manager.cc:5697] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 4 to 5, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 5 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.659789 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:36.641697 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.669238 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2275) took 3186 ms. Trace:
I20260709 13:07:36.676806 21138 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:36.677623 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.628368 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.568384 24818 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:36.572714 24910 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:36.682523 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:36.456033 25036 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:36.688341 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.690446 21140 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:36.692713 24995 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:36.694785 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:36.695070 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.696624 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.827173 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.827596 21142 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:36.570605 24851 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:36.662475 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.683156 24947 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:36.703413 24989 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:36.708662 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:36.833779 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.788476 25043 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:36.711313 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 268944us
I20260709 13:07:36.716858 24997 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:36.757121 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:36.722908 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.790140 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:36.793004 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 567544us
I20260709 13:07:36.803524 21147 rpcz_store.cc:276] 0709 13:07:33.482752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.472953 (+1990201us) service_pool.cc:224] Handling call
0709 13:07:35.473493 (+   540us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:36.304448 (+830955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:36.304477 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:36.305063 (+   586us) log.cc:844] Serialized 52 byte log entry
0709 13:07:36.447297 (+142234us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:36.483667 (+ 36370us) log.cc:844] Serialized 14 byte log entry
0709 13:07:36.571296 (+ 87629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:36.571383 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:36.669024 (+ 97641us) raft_consensus.cc:1685] finished
0709 13:07:36.669041 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:36.669177 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1340178,"prepare.queue_time_us":182042,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":873273,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":516852,"thread_start_us":214671,"threads_started":3,"wal-append.queue_time_us":278477}
I20260709 13:07:36.839157 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:36.807344 21070 raft_consensus.cc:2410] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:07:36.804239 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:36.804313 21799 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:36.810124 21216 raft_consensus.cc:2410] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:36.757956 24816 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:36.803261 21795 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:36.828576 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:36.846367 21794 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:36.718122 24549 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:36.830891 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 136200us
I20260709 13:07:36.793478 25000 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:36.833050 21146 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:07:36.710397 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 220746us
I20260709 13:07:36.803659 20969 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 4 to 5, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 5 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.798305 25026 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:36.838893 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 81855.5us
I20260709 13:07:36.854136 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:36.854821 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:36.831125 24995 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:36.843675 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:36.753293 24663 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:36.836551 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:36.847777 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 63107.6us
I20260709 13:07:36.798305 24524 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:07:36.857393 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:36.909799 21141 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:36.888597 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:36.837817 24662 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:07:36.905443 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61861.9us
I20260709 13:07:36.842728 21215 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:07:36.826308 24831 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:36.849249 24816 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:36.917713 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:36.920310 21215 pending_rounds.cc:85] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 5
I20260709 13:07:36.930028 21215 pending_rounds.cc:107] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:07:36.930421 21215 raft_consensus.cc:2889] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:36.845602 25037 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:36.922613 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:37.088220 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 165699us
I20260709 13:07:37.089071 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.090152 25028 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:36.837965 24978 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:07:36.804443 24907 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:36.977461 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 59828.5us
I20260709 13:07:36.979795 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:37.092041 21146 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:36.981065 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 141484us
I20260709 13:07:37.092487 25036 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.093019 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:37.094424 21036 leader_election.cc:400] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:36.911634 24947 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:37.095429 24947 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:37.096251 24947 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.097455 24947 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:36.906361 24965 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:37.099485 24965 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.099852 24965 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:37.101230 24572 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.101673 24572 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:36.910701 24990 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:36.916991 24942 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:37.010152 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 83261.4us
W20260709 13:07:37.010046 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 121541us
I20260709 13:07:37.107735 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:36.927769 24927 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:36.848212 25043 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:36.884874 25026 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.067238 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.117620 21138 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:37.064477 24898 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:37.119117 24898 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:37.119383 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:37.119625 24898 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:37.072152 25051 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:07:37.072324 24831 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:37.071990 24995 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.073379 24549 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:37.073609 25048 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:07:37.074352 25054 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
W20260709 13:07:37.075001 25046 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:07:36.853505 24997 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:37.006064 25031 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:36.923681 24911 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:07:37.009927 24662 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:37.107496 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.112107 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:37.113858 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:37.113648 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:37.116652 21036 leader_election.cc:400] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:37.118785 25036 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:37.123787 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.145496 21138 raft_consensus.cc:2410] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:07:37.112202 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:37.146586 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:37.126207 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 263323us
I20260709 13:07:37.124073 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.115469 24818 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:37.148721 24818 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:37.149196 24818 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:37.110694 24989 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:37.115640 24945 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.131356 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1199) took 1419 ms. Trace:
I20260709 13:07:37.151908 21215 rpcz_store.cc:276] 0709 13:07:35.711720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.715775 (+  4055us) service_pool.cc:224] Handling call
0709 13:07:36.842227 (+1126452us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:07:36.842406 (+   179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.127323 (+284917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.131269 (+  3946us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8310656}
W20260709 13:07:37.153035 21184 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.111052 24927 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:37.153940 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.153815 24927 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:37.159838 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.160768 21215 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:37.133972 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.134160 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.162451 21139 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:37.137209 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.128652 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:37.163586 21143 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:37.164816 21145 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
W20260709 13:07:37.170099 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56522.7us
I20260709 13:07:37.171075 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:37.172384 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:37.180819 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52232.6us
I20260709 13:07:37.181145 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.183614 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.183833 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:37.184295 21145 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:37.189576 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.189374 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:37.194125 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.195609 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:37.195058 21219 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
W20260709 13:07:37.208547 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 84560.8us
I20260709 13:07:37.209892 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:37.213665 21143 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:07:37.144807 25031 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:37.206480 21147 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:37.226135 21218 pending_rounds.cc:85] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 3
I20260709 13:07:37.169801 25036 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:37.234139 21218 pending_rounds.cc:107] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:07:37.241478 21218 raft_consensus.cc:2889] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:37.214953 21069 raft_consensus.cc:2376] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8. Re-sending same reply.
I20260709 13:07:37.175444 24588 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:37.188697 24917 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:37.181807 24995 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:37.248265 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.197499 24549 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:37.197464 24971 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:37.252381 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:37.252130 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.208325 24606 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:07:37.247455 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 152141us
I20260709 13:07:37.254753 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:37.265550 21069 raft_consensus.cc:2376] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9. Re-sending same reply.
I20260709 13:07:37.265697 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.192474 25038 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:37.257854 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 48070.8us
I20260709 13:07:37.277925 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.278568 21067 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
W20260709 13:07:37.260915 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.192929 24942 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:37.196420 25067 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:37.257567 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.267359 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.284267 21212 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:37.285017 21212 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:07:37.267912 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:37.285641 24907 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.268407 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:37.282363 24663 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:37.287672 24606 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:37.286924 21070 raft_consensus.cc:2376] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8. Re-sending same reply.
I20260709 13:07:37.288640 24606 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.289600 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.290308 21070 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:37.243157 24973 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.292066 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:37.292650 21067 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:37.291816 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.269135 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:37.269388 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:37.293934 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:37.245272 25036 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:37.294474 21064 raft_consensus.cc:2376] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13. Re-sending same reply.
I20260709 13:07:37.252869 24917 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:37.272042 25066 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:07:37.272778 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.273593 25039 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:37.247828 24588 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.327638 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:37.241112 25025 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:37.331267 24898 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.283177 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:37.338697 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60321.2us
I20260709 13:07:37.339440 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.343057 24907 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:37.343550 24907 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.343904 24907 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:37.333086 24588 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:37.351452 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.294492 21144 raft_consensus.cc:606] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:37.295253 21070 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:37.295082 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:37.353178 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58166.4us
I20260709 13:07:37.264142 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.270514 21036 leader_election.cc:400] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10.
W20260709 13:07:37.355193 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 91145.3us
I20260709 13:07:37.355427 21036 leader_election.cc:403] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:07:37.352896 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:37.300640 24662 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.356535 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.324566 24942 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:37.357450 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.358141 21146 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:37.356560 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:37.301478 24572 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.359539 21070 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:07:37.247545 24997 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:37.275804 25045 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
I20260709 13:07:37.307590 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:37.364934 21797 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:37.283308 24831 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:37.308296 25043 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:37.309180 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.309994 25013 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:37.310364 25007 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:37.313412 25049 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:07:37.313577 24953 ts_tablet_manager.cc:1929] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260709 13:07:37.321573 25026 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:37.272783 25057 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:07:37.292639 25067 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.323467 21794 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:37.324084 21071 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
W20260709 13:07:37.325946 21185 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.507s (SENT) [suppressed 16 similar messages]
I20260709 13:07:37.326499 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.274358 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.335542 24818 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.331875 25025 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:37.242462 25031 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:37.332293 25037 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:37.265158 24971 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:37.322952 24917 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:37.293674 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:37.409308 21065 raft_consensus.cc:2376] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12. Re-sending same reply.
W20260709 13:07:37.409355 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 139000us
I20260709 13:07:37.259574 24549 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.325279 25039 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:37.360466 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.354223 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:37.414237 21070 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
W20260709 13:07:37.414356 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 60262.5us
I20260709 13:07:37.360976 25022 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:37.363536 24761 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:37.363343 24808 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:07:37.402472 25067 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:37.369393 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 593) took 1020 ms. Trace:
W20260709 13:07:37.418146 24814 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.002s
I20260709 13:07:37.382488 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.364724 24690 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:37.428210 21072 rpcz_store.cc:276] 0709 13:07:36.349355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.349496 (+   141us) service_pool.cc:224] Handling call
0709 13:07:36.364617 (+ 15121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:36.381337 (+ 16720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:36.381376 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:36.381829 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:07:36.713585 (+331756us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:36.713631 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:36.713695 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:37.369147 (+655452us) raft_consensus.cc:1685] finished
0709 13:07:37.369171 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.369318 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17463,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":331357,"threads_started":1,"wal-append.queue_time_us":428100}
I20260709 13:07:37.417083 24549 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:37.360921 24900 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:37.329185 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:37.354081 24927 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.375489 25073 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10.
I20260709 13:07:37.378803 25013 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:37.417927 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:37.418537 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:37.419610 21212 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:37.423076 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.425556 21065 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:37.447926 21071 raft_consensus.cc:2410] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:37.407527 25025 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:37.426317 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 45094us
I20260709 13:07:37.413460 24953 log.cc:1199] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-3-root/wals/8b6dd6862dd14baba4a82c5a837bc802
I20260709 13:07:37.410535 25037 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.410400 24971 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:37.365321 25032 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:37.361560 24973 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:37.422580 24761 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:37.416957 25039 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:37.419054 25022 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:37.418866 25023 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:07:37.441972 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 112836us
I20260709 13:07:37.368073 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.460284 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:37.463194 25077 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:07:37.450278 25013 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:37.452492 25026 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:37.455757 25032 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:07:37.468994 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58570.9us
I20260709 13:07:37.434284 24690 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:37.450043 24606 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:37.476671 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:37.477711 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.493824 24973 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:37.494241 24973 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:37.494628 24973 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:37.494457 25001 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:37.494904 21113 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:37.495250 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.495585 25001 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:37.495813 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:37.496348 24911 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:37.496447 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:37.496831 24911 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:37.497422 24970 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:37.497597 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:37.497771 24911 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.498525 24915 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:37.499233 24915 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:37.499626 24915 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:37.499017 24911 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:37.500018 24915 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:37.500078 24766 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:37.500471 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:37.500679 24766 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:37.501065 24766 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:37.501542 25019 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:37.501962 25019 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.502306 25019 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:37.478511 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 590) took 1170 ms. Trace:
I20260709 13:07:37.455518 25003 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:37.504212 25003 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:37.480415 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 112430us
I20260709 13:07:37.504045 21068 rpcz_store.cc:276] 0709 13:07:36.308550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.308703 (+   153us) service_pool.cc:224] Handling call
0709 13:07:36.320679 (+ 11976us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:36.431317 (+110638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:36.431362 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:36.474741 (+ 43379us) log.cc:844] Serialized 52 byte log entry
0709 13:07:36.841630 (+366889us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:36.841686 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:36.841775 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:37.312786 (+471011us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:37.478238 (+165452us) raft_consensus.cc:1685] finished
0709 13:07:37.478256 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.478456 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33647,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5683,"spinlock_wait_cycles":13326080,"thread_start_us":351634,"threads_started":2,"wal-append.queue_time_us":576398}
I20260709 13:07:37.506021 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:37.506438 21068 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:37.506808 21214 raft_consensus.cc:2410] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:07:37.507690 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.465431 24953 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48444 (request call id 9) took 4225 ms. Trace:
I20260709 13:07:37.508499 24953 rpcz_store.cc:276] 0709 13:07:33.240210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.241564 (+  1354us) service_pool.cc:224] Handling call
0709 13:07:33.635577 (+394013us) ts_tablet_manager.cc:1151] Deleting tablet 8b6dd6862dd14baba4a82c5a837bc802
0709 13:07:33.668503 (+ 32926us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:07:36.401096 (+2732593us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:37.313185 (+912089us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:37.465326 (+152141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1392792,"tablet-delete.queue_time_us":332714,"thread_start_us":151313,"threads_started":1}
I20260709 13:07:37.449383 25038 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:37.483942 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:37.504778 25003 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:37.455442 25037 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:37.476034 24761 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:37.480077 25068 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:37.437017 24814 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.515533 25068 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.516034 25068 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:37.520790 24814 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.521396 25070 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:37.522567 25070 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:37.522961 25070 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:37.524148 25070 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.525568 25070 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:37.526700 24606 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:37.524190 24761 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.531948 24606 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.534003 25019 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.507546 24915 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.509109 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.477159 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:37.492187 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:37.537449 24915 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:37.537748 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.536702 24761 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:37.491179 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 165245us
I20260709 13:07:37.538748 25003 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.539954 25037 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:37.542129 25037 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:07:37.543083 25003 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:37.539143 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.545359 21214 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:37.546214 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.547235 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.548170 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.549122 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.548940 24766 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.480288 25022 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:37.449431 24900 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:37.551952 21139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:37.481007 24991 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:37.567071 25074 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:07:37.554139 21146 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:37.466980 24907 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.603899 21185 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:37.456439 25018 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:07:37.557942 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:37.651707 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 93843.7us
W20260709 13:07:37.653517 24947 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
W20260709 13:07:37.604560 24816 consensus_meta.cc:287] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.003s
I20260709 13:07:37.432751 24942 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.654358 24816 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:37.602144 24965 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.000s
I20260709 13:07:37.542181 24973 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.490204 25032 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:37.561939 25056 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:07:37.565798 25082 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:37.566040 25075 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:07:37.563105 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:37.567826 25019 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.491956 24690 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:37.574098 24910 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.575016 24917 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.576507 25007 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:37.581023 24900 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:37.480535 25023 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.592113 24927 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:37.592939 25080 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.053s	user 0.000s	sys 0.002s
I20260709 13:07:37.593221 24818 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:37.593569 24588 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.594265 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:37.598318 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 109338us
I20260709 13:07:37.553469 24898 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.530963 25068 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.556195 24995 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:37.630059 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.630254 24549 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.630730 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:37.631469 24766 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:37.638779 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2280) took 1223 ms. Trace:
I20260709 13:07:37.694114 21141 rpcz_store.cc:276] 0709 13:07:36.415636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.419547 (+  3911us) service_pool.cc:224] Handling call
0709 13:07:37.638516 (+1218969us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54001eb4d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:37.638662 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986775424}
I20260709 13:07:37.694141 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:37.639588 25081 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:37.639869 24851 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:37.568962 25026 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:37.703341 21214 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:37.640173 25022 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.646452 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.648847 24907 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:37.655586 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:37.551173 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:37.708206 21138 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:37.629084 25000 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.771s	user 0.012s	sys 0.000s
I20260709 13:07:37.663693 21216 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:37.664247 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 207266us
I20260709 13:07:37.666913 21218 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:37.646960 25039 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.661684 24990 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:37.677846 21139 raft_consensus.cc:606] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:37.662827 24663 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:37.683166 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:37.679935 25023 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:37.677865 25007 consensus_queue.cc:1243] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:07:37.750236 25088 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:07:37.691274 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 133395us
I20260709 13:07:37.663153 24965 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:37.700389 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 73127us
I20260709 13:07:37.753618 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:37.754070 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.754418 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:37.754930 21220 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 13. Ops: []
I20260709 13:07:37.706164 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.756189 25095 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:37.756829 25095 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.757256 25095 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:37.743187 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:37.743932 21063 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:07:37.746091 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:37.746399 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:37.665639 25082 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.692312 24995 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:37.703080 24818 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:37.756265 21218 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
W20260709 13:07:37.747012 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.758488 21213 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:37.753664 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.762570 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:37.763756 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:37.765355 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:37.765899 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:37.770798 21796 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:37.754442 25039 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:37.762992 21185 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:37.769717 21217 raft_consensus.cc:2410] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:07:37.771883 21220 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:37.774541 21765 leader_election.cc:336] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.672s (SENT)
I20260709 13:07:37.775162 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:37.775241 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.777112 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:37.778496 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 92333us
I20260709 13:07:37.780097 24761 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:37.782680 24549 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.783125 25084 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:37.783656 25084 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.784099 25084 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:37.784318 24917 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.786890 21138 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:07:37.748610 24766 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:37.793980 21036 leader_election.cc:400] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:37.798444 21036 leader_election.cc:403] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:37.799670 25026 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8.
I20260709 13:07:37.779562 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.768411 24995 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:37.809742 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:37.812283 25007 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:37.813143 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.810920 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.815052 21215 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:37.801409 21036 leader_election.cc:400] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
W20260709 13:07:37.815358 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.815637 21036 leader_election.cc:403] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:37.784971 24945 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:37.792476 24761 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:37.821750 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:37.823408 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.835295 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.836880 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.837754 21069 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:37.838235 25036 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260709 13:07:37.838747 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.838675 25036 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.840451 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.840955 25036 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:37.841144 21066 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 5.
I20260709 13:07:37.836058 21072 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 6.
I20260709 13:07:37.842072 25025 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.842845 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.843538 21065 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:37.844410 25025 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.808911 24766 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.845757 25065 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:07:37.846629 21138 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:37.846786 25065 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:37.810633 24907 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:37.847188 25065 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:37.847675 24766 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:37.847730 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.848477 21064 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:37.848706 25022 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.849646 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.851418 25026 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 5. Current term is 6.
I20260709 13:07:37.854214 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.848281 25065 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:37.839092 25093 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:07:37.857167 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:37.857709 21072 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:07:37.866219 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.867478 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:37.846335 24971 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.868119 21215 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:37.870339 24971 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.871134 21143 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:37.835455 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.859807 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:37.875627 21070 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:37.861701 21796 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:07:37.847144 24816 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:37.864586 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.877427 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:37.877733 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:37.848774 24588 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:37.878350 21794 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:37.877772 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 76519us
I20260709 13:07:37.880465 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:37.820367 25007 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:07:37.835721 24945 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:37.895556 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.911656 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:37.885222 24927 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:37.888202 25036 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:37.926692 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.927462 21072 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:37.916134 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:37.929419 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:37.903656 25065 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:37.864003 25023 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:37.931739 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:37.934690 20954 catalog_manager.cc:5028] TS b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): tablet 8b6dd6862dd14baba4a82c5a837bc802 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:07:37.935076 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:37.950069 21794 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:37.942942 21215 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:37.929263 25000 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:37.933847 25036 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:37.977937 25097 ts_tablet_manager.cc:933] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:37.978322 25031 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.010s
I20260709 13:07:37.983028 25091 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:37.984040 25090 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:07:37.930929 24761 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:37.985378 21792 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:38.092955 21213 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
W20260709 13:07:38.057026 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 255777us
I20260709 13:07:38.125401 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:38.090519 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:38.071538 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:38.102928 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:38.102216 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:38.119755 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:38.083592 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:38.065369 21139 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 13:07:38.049141 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 137594us
I20260709 13:07:38.133934 21036 leader_election.cc:400] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:38.062276 21215 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:07:38.101924 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:38.136116 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:38.137331 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:38.125249 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:38.143836 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:38.144200 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:38.132310 21144 raft_consensus.cc:606] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:38.148339 21213 pending_rounds.cc:85] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 7
I20260709 13:07:38.148685 21213 pending_rounds.cc:107] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:07:38.148914 21213 raft_consensus.cc:2889] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:38.149109 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:38.151886 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:38.090719 25067 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.652s	user 0.009s	sys 0.000s
I20260709 13:07:38.152786 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:37.984551 25018 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:38.131294 24572 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:38.156924 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:38.164057 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:37.947523 25013 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.163215 21215 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:38.170194 21063 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:07:38.170712 21794 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:38.161486 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:38.087958 25023 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:38.175777 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:38.176884 21141 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:38.179338 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:38.154079 25039 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:38.193962 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:38.221937 21800 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
W20260709 13:07:38.222766 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.159212 25018 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:38.229796 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 95940.2us
I20260709 13:07:38.136297 25031 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.032308 24945 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.013293 25036 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.195768 20959 maintenance_manager.cc:419] P b068e036d3f142f4b255651ac6a0e101: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033372
I20260709 13:07:38.197465 25039 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:38.253662 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.258131 24663 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:38.258661 24663 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:38.260797 21141 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:38.261816 21221 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:38.264498 21794 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
W20260709 13:07:38.323675 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.222873 25091 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:38.161839 25067 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:38.352630 25095 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.595s	user 0.008s	sys 0.005s
I20260709 13:07:38.260229 25039 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:38.368229 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:38.367748 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:38.377413 21064 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:38.382033 21793 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:38.390444 21141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:38.391306 21141 raft_consensus.cc:606] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:38.391743 21141 raft_consensus.cc:614] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:38.396518 25067 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:38.414956 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 312140us
I20260709 13:07:38.415882 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:38.417747 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 441217us
I20260709 13:07:38.384472 24927 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:38.422886 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:38.376667 25095 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:38.433452 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65308.1us
I20260709 13:07:38.430218 25067 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:38.442426 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:38.438978 25084 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.654s	user 0.007s	sys 0.007s
I20260709 13:07:38.366351 25018 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.441345 25089 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:38.468740 25100 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.470831 24927 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:07:38.528975 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 86627.6us
I20260709 13:07:38.530043 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:38.531634 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:38.523134 21063 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:38.482218 25023 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:07:38.534363 21141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:38.488314 25018 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:38.545470 25013 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:38.562821 21141 raft_consensus.cc:606] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:38.546808 25056 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:38.565387 25056 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:38.566329 25056 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.445870 25091 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:07:38.567916 25091 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:38.568301 25091 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:38.569173 25091 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.570533 25091 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:38.476418 25084 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:38.572436 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:38.573251 25084 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:38.574936 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:07:38.587046 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55522.2us
I20260709 13:07:38.532912 25100 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:38.659929 21145 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.011s	sys 0.025s
W20260709 13:07:38.661505 21145 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.011s	sys 0.026s
W20260709 13:07:38.560443 25101 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.009s	sys 0.005s
I20260709 13:07:38.601114 24927 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:38.660894 25031 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:38.691834 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 874) took 1021 ms. Trace:
I20260709 13:07:38.693500 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:38.664608 25084 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.695665 21800 rpcz_store.cc:276] 0709 13:07:37.670804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.670999 (+   195us) service_pool.cc:224] Handling call
0709 13:07:37.683203 (+ 12204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.222447 (+539244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.691568 (+469121us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400682254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:38.691769 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238563328}
W20260709 13:07:38.701375 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.705927 21144 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:38.694526 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 209587us
I20260709 13:07:38.709563 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:38.707540 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:38.712303 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.680946 24927 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:38.718444 21140 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:38.715447 21144 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:38.719659 21141 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:38.722663 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:38.706470 25095 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:38.726572 25065 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.786s	user 0.008s	sys 0.004s
W20260709 13:07:38.727253 24690 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.048s	user 0.013s	sys 0.000s
I20260709 13:07:38.727739 24690 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:38.729923 24690 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:38.507866 25007 consensus_queue.cc:1175] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" responder_term: 15 status { last_received { term: 11 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: [14.11-14.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:38.739493 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:38.711199 24990 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:38.738323 25013 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:07:38.726589 25115 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:38.752631 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:38.723948 24927 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:38.744500 25031 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:07:38.760658 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:38.766381 21140 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:38.703387 25084 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:38.755112 24947 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:07:38.754118 25013 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:38.770946 21141 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:38.772192 25013 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.773629 25013 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:38.810878 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:38.811820 21063 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:38.812855 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.814965 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:38.818964 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1244) took 1024 ms. Trace:
I20260709 13:07:38.819576 21215 rpcz_store.cc:276] 0709 13:07:37.794600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.172021 (+377421us) service_pool.cc:224] Handling call
0709 13:07:38.360127 (+188106us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:38.368441 (+  8314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:38.368485 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:38.369179 (+   694us) log.cc:844] Serialized 52 byte log entry
0709 13:07:38.681688 (+312509us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:38.681733 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.681807 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:38.818602 (+136795us) raft_consensus.cc:1685] finished
0709 13:07:38.818636 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:38.818882 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":793,"raft.queue_time_us":7887,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":9094,"thread_start_us":259949,"threads_started":1,"wal-append.queue_time_us":424818}
I20260709 13:07:38.828460 25111 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.828857 24965 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:38.829051 25111 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:38.830842 24995 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.996s	user 0.007s	sys 0.006s
I20260709 13:07:38.831310 24995 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.832597 25082 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:38.833439 24995 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:38.833657 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:38.834117 21216 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:38.831302 24766 raft_consensus.cc:993] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:38.835526 24766 raft_consensus.cc:1081] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:38.835772 25068 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:38.836113 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:38.837180 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 472) took 1588 ms. Trace:
I20260709 13:07:38.837560 21143 rpcz_store.cc:276] 0709 13:07:37.248412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.251367 (+  2955us) service_pool.cc:224] Handling call
0709 13:07:37.251671 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.871708 (+620037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.836947 (+965239us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54002963d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:38.837127 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1424201600}
W20260709 13:07:38.838707 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.840862 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:38.775906 24997 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:38.841526 21064 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:38.841017 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 461) took 1332 ms. Trace:
I20260709 13:07:38.848639 21765 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:38.857014 21220 rpcz_store.cc:276] 0709 13:07:37.508233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.755638 (+247405us) service_pool.cc:224] Handling call
0709 13:07:37.755874 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.774745 (+ 18871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.819875 (+1045130us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b54000bc2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:38.840959 (+ 21084us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27034496}
I20260709 13:07:38.858153 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:38.853539 25111 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:38.804579 25033 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:38.760895 24818 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:07:38.750787 25097 tablet_copy_client.cc:323] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
W20260709 13:07:38.873240 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:38.846323 25032 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.176s	user 0.014s	sys 0.000s
W20260709 13:07:38.866644 25112 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:07:38.882212 24851 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:38.915077 21793 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:38.884503 25084 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:38.887089 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:38.895769 24690 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:38.897100 24973 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:38.902547 25107 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.671s	user 0.002s	sys 0.002s
I20260709 13:07:38.880163 25033 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:38.869397 25065 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:38.940979 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2086) took 1058 ms. Trace:
I20260709 13:07:38.941505 21070 rpcz_store.cc:276] 0709 13:07:37.882572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.882961 (+   389us) service_pool.cc:224] Handling call
0709 13:07:37.883393 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:37.896689 (+ 13296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:37.896774 (+    85us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:37.905330 (+  8556us) log.cc:844] Serialized 26 byte log entry
0709 13:07:38.132960 (+227630us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:38.133018 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.133108 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:38.901780 (+768672us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:38.940577 (+ 38797us) raft_consensus.cc:1685] finished
0709 13:07:38.940607 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:38.940901 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1751,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":8812,"spinlock_wait_cycles":40573312,"thread_start_us":116792,"threads_started":1,"wal-append.queue_time_us":223206}
I20260709 13:07:38.948156 25033 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:38.949955 25033 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:38.953272 21144 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:38.966507 21063 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:38.980055 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1240) took 1209 ms. Trace:
I20260709 13:07:38.980626 21218 rpcz_store.cc:276] 0709 13:07:37.770214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.147892 (+377678us) service_pool.cc:224] Handling call
0709 13:07:38.148226 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:38.155142 (+  6916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:38.155170 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:38.155677 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:07:38.438085 (+282408us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:38.438131 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.438210 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:38.979786 (+541576us) raft_consensus.cc:1685] finished
0709 13:07:38.979812 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:38.979981 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15,"raft.queue_time_us":14188,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":14351,"spinlock_wait_cycles":2816,"thread_start_us":282095,"threads_started":1,"wal-append.queue_time_us":419372}
I20260709 13:07:38.929807 24690 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:38.985687 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:38.986742 21145 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:38.992606 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:38.930018 25032 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:38.975057 25065 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:39.000376 21214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 466) took 1490 ms (client timeout 1774 ms). Trace:
W20260709 13:07:39.000844 21214 rpcz_store.cc:269] 0709 13:07:37.509601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.548903 (+ 39302us) service_pool.cc:224] Handling call
0709 13:07:39.000122 (+1451219us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540015c054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:39.000281 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348750080}
I20260709 13:07:39.001937 21799 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:39.002498 25032 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:39.004510 21799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 873) took 1334 ms. Trace:
I20260709 13:07:39.004908 21799 rpcz_store.cc:276] 0709 13:07:37.670298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.670442 (+   144us) service_pool.cc:224] Handling call
0709 13:07:37.679511 (+  9069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.002346 (+1322835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.004282 (+  1936us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400701a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:39.004453 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3040898816}
I20260709 13:07:38.987171 25100 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:38.996651 25040 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:39.023260 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 182730us
I20260709 13:07:39.025669 21767 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:39.018719 24965 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.014575 24978 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:39.042688 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:39.043648 24990 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:39.023515 25040 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.028709 25032 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:39.052897 25040 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:39.049113 24978 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.106597 24978 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:39.060582 25032 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:39.111408 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.058s	sys 0.151s
W20260709 13:07:39.112460 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.058s	sys 0.151s
I20260709 13:07:39.024904 24690 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:39.119884 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 531013us
I20260709 13:07:39.049532 24965 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.044905 24990 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.110253 25007 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:07:39.151392 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.153257 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 101828us
I20260709 13:07:39.114339 25032 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:39.060042 25065 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:39.162441 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 137987us
I20260709 13:07:39.153134 25007 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:39.158334 24690 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:39.175812 24766 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.159982 25084 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:39.161741 24995 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:39.176357 24766 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:39.183712 25091 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.613s	user 0.011s	sys 0.004s
W20260709 13:07:39.184245 21219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1226) took 1932 ms (client timeout 1828 ms). Trace:
W20260709 13:07:39.184682 21219 rpcz_store.cc:269] 0709 13:07:37.251671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.251823 (+   152us) service_pool.cc:224] Handling call
0709 13:07:39.184046 (+1932223us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b54004439d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:39.184173 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239714048}
I20260709 13:07:39.189944 21794 raft_consensus.cc:2410] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:07:39.196415 21221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2624) took 1689 ms (client timeout 1985 ms). Trace:
W20260709 13:07:39.196972 21221 rpcz_store.cc:269] 0709 13:07:37.506648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.693872 (+187224us) service_pool.cc:224] Handling call
0709 13:07:39.196109 (+1502237us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b5400104354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:39.196333 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1303074560}
I20260709 13:07:39.198764 21212 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:39.178303 25068 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:07:39.204625 25036 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:39.205070 25036 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:39.204876 25040 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.205928 25036 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.208354 24973 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.208865 24973 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.210304 25082 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:39.210711 25082 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:39.210970 24973 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:39.211535 25082 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.213055 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2080) took 1368 ms. Trace:
W20260709 13:07:39.213891 25129 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:07:39.214411 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:39.252305 25128 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:07:39.254876 24663 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.993s	user 0.010s	sys 0.002s
I20260709 13:07:39.214514 25131 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:39.255333 24663 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.255662 25131 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.214995 25000 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:39.257747 24663 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:39.190305 25124 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:07:39.259904 24965 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:39.260401 21792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 793) took 1500 ms. Trace:
I20260709 13:07:39.180410 25084 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.260810 21792 rpcz_store.cc:276] 0709 13:07:37.759576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.759678 (+   102us) service_pool.cc:224] Handling call
0709 13:07:37.764994 (+  5316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.124337 (+359343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.260171 (+1135834us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54002c3654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:39.260345 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485695360}
I20260709 13:07:39.262238 25082 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:39.262621 25084 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:39.195497 24995 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:39.221812 25039 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.838s	user 0.005s	sys 0.005s
I20260709 13:07:39.278733 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:39.279116 25039 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.281313 25039 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:39.229374 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:39.282840 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:39.283442 21070 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:07:39.286486 21145 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:39.288492 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:39.290355 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.229673 24927 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.230139 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:39.299784 21064 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:07:39.230810 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:39.231495 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:39.235405 24851 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.302714 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:39.303508 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:39.240805 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 463) took 1732 ms. Trace:
I20260709 13:07:39.241889 21799 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:39.245029 21144 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:39.245586 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:39.307716 21143 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:39.246483 21141 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:39.309334 21793 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:39.247253 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:39.310205 21146 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:39.248534 25001 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:07:39.251999 25138 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:39.278036 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:39.251720 21066 rpcz_store.cc:276] 0709 13:07:37.844973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.851796 (+  6823us) service_pool.cc:224] Handling call
0709 13:07:37.852292 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:37.852639 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:37.852669 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:37.853169 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:07:39.096978 (+1243809us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b18003a30c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:39.212467 (+115489us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b18003a30c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1
0709 13:07:39.212619 (+   152us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:39.212669 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.212787 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:39.212848 (+    61us) raft_consensus.cc:1685] finished
0709 13:07:39.212870 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.213004 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":665,"raft.queue_time_us":9206,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":23112,"spinlock_wait_cycles":10443904,"thread_start_us":40600,"threads_started":1,"wal-append.queue_time_us":40964}
I20260709 13:07:39.329830 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:39.330349 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 52410.5us
I20260709 13:07:39.331167 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:39.331975 25023 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:39.332389 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:39.333148 25023 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:39.334218 25023 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:39.259908 25000 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:39.334853 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 212207us
I20260709 13:07:39.334992 25000 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:39.335536 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:39.340267 25023 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:39.340847 25023 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:39.342183 25023 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.207127 25036 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:39.313337 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.257759 25131 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:39.305439 21212 rpcz_store.cc:276] 0709 13:07:37.508781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.757165 (+248384us) service_pool.cc:224] Handling call
0709 13:07:39.198507 (+1441342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.200445 (+  1938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.240606 (+ 40161us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400104350. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:39.240744 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3314077952}
I20260709 13:07:39.328457 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:39.350592 21138 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:39.328683 21801 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:07:39.329129 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2091) took 1284 ms. Trace:
I20260709 13:07:39.351704 21067 rpcz_store.cc:276] 0709 13:07:38.044905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.045263 (+   358us) service_pool.cc:224] Handling call
0709 13:07:38.069285 (+ 24022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.069564 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:38.070201 (+   637us) log.cc:844] Serialized 14 byte log entry
0709 13:07:38.887843 (+817642us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b180035ff80. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:07:39.328731 (+440888us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:39.328794 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:39.328850 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.328919 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.329065 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":233,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":1459,"spinlock_wait_cycles":15386368,"thread_start_us":440540,"threads_started":2,"wal-append.queue_time_us":1102343}
I20260709 13:07:39.354242 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:39.301896 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.334345 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:39.355189 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.356758 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:39.357910 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:39.355126 21067 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:07:39.359448 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:39.362727 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.363196 25031 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.363533 24818 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:39.363715 25031 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.364231 24818 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.365756 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:39.365969 25031 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:39.366492 21215 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:39.366775 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:39.367920 21215 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:39.368249 21214 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:39.369504 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.371191 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:39.371331 21213 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:39.371891 21213 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:39.372859 21213 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.374035 21213 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:39.376425 21214 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:39.378147 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:39.378888 21221 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:39.379469 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.367058 24818 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:39.384011 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:39.285128 25084 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:39.389081 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:39.294097 25040 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:39.394518 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:39.390829 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 98288.8us
I20260709 13:07:39.392678 21138 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:39.393848 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:39.395526 21141 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:39.395115 21066 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:39.362247 25018 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.779s	user 0.009s	sys 0.003s
I20260709 13:07:39.318944 24965 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.414387 21213 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:07:39.438374 21213 pending_rounds.cc:85] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 10
I20260709 13:07:39.438813 21213 pending_rounds.cc:107] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:07:39.439183 21213 raft_consensus.cc:2889] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:39.440338 21072 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:39.361953 25013 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:39.387964 25091 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:39.445248 21185 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:39.386204 25125 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:39.395617 24818 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:39.474284 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.477988 21067 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
W20260709 13:07:39.481084 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56271us
I20260709 13:07:39.401551 25023 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:39.484840 25033 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:39.488229 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:39.491719 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1203) took 1344 ms. Trace:
I20260709 13:07:39.458410 25031 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:07:39.488443 24927 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:39.492621 25031 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:39.493057 25031 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:39.493412 25031 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:39.494048 25032 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.386590 24942 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:07:39.495390 21800 raft_consensus.cc:2368] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12.
I20260709 13:07:39.495777 25144 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:39.496004 24942 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:39.496217 25144 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:39.496498 24942 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.496711 25144 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:39.496948 24942 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:39.492174 21071 rpcz_store.cc:276] 0709 13:07:38.147513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.734405 (+586892us) service_pool.cc:224] Handling call
0709 13:07:38.734967 (+   562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:39.107624 (+372657us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:39.107667 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:39.109777 (+  2110us) log.cc:844] Serialized 26 byte log entry
0709 13:07:39.162744 (+ 52967us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:39.162782 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.162850 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:39.491484 (+328634us) raft_consensus.cc:1685] finished
0709 13:07:39.491505 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.491661 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":433819,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":36081,"thread_start_us":415832,"threads_started":2,"wal-append.queue_time_us":280206}
W20260709 13:07:39.475428 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 429501us
I20260709 13:07:39.527920 21138 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:39.528977 21070 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:39.437000 25084 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:39.329414 24851 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.535897 21219 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:39.536654 21064 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:39.588649 25141 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:39.589666 25141 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:39.590145 25141 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.387022 25007 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.591818 25007 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:39.534544 25040 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:39.595523 25040 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:39.596414 25032 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:39.596339 25040 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.597577 25040 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:39.527158 24818 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.601958 24818 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:39.605085 24995 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:39.606278 25084 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.611660 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:39.612463 21064 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:39.457856 24965 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.640632 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:39.641559 21065 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:39.646770 21066 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
W20260709 13:07:39.658603 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.661525 21113 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.006s	sys 0.009s
W20260709 13:07:39.663533 21113 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.006s	sys 0.009s
W20260709 13:07:39.663959 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 503369us
I20260709 13:07:39.586328 25139 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
W20260709 13:07:39.687095 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 241119us
I20260709 13:07:39.695122 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:39.688974 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 353477us
W20260709 13:07:39.708738 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.671485 24851 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:39.721856 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.727317 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 366424us
W20260709 13:07:39.716291 25065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.036s	sys 0.086s
I20260709 13:07:39.705142 25147 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:39.698802 25039 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:39.718597 21064 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:39.670634 25001 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 13:07:39.720165 24524 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.075s	sys 0.077s
I20260709 13:07:39.781281 21143 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:07:39.750860 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 89894.1us
I20260709 13:07:39.770084 21066 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:39.771483 21146 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
W20260709 13:07:39.777468 25143 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:07:39.719702 25128 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:39.752043 25065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.036s	sys 0.087s
I20260709 13:07:39.792182 21066 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:39.753755 21065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39510
I20260709 13:07:39.804767 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.808075 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.809031 21219 raft_consensus.cc:2368] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:39.810645 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.761169 25147 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:39.782124 24524 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.075s	sys 0.077s
W20260709 13:07:39.794749 24990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.006s	sys 0.011s
W20260709 13:07:39.784490 24761 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.011s	sys 0.007s
I20260709 13:07:39.822376 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:39.823263 21068 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.018s	sys 0.029s
I20260709 13:07:39.790926 25136 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:39.768123 25000 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:39.796636 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67317.8us
I20260709 13:07:39.798199 25141 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:39.829617 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 609) took 1082 ms. Trace:
I20260709 13:07:39.828390 21066 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
W20260709 13:07:39.836979 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.836819 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:39.844930 21801 raft_consensus.cc:2410] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:07:39.847190 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.851365 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:39.857075 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 48257.4us
I20260709 13:07:39.857924 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:39.841902 21068 rpcz_store.cc:276] 0709 13:07:38.747270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.747700 (+   430us) service_pool.cc:224] Handling call
0709 13:07:38.762653 (+ 14953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.762888 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:39.049025 (+286137us) log.cc:844] Serialized 14 byte log entry
0709 13:07:39.829219 (+780194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:39.829280 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:39.829345 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.829417 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.829560 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44325,"prepare.queue_time_us":404666,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":176783,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":960390,"threads_started":3,"wal-append.queue_time_us":530004}
I20260709 13:07:39.860127 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:39.861146 21066 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:39.867839 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.872740 21213 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:39.873296 21215 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:39.873847 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:39.802762 25128 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:39.833981 24761 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.011s	sys 0.007s
I20260709 13:07:39.876950 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.831203 25136 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:39.882547 21801 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:07:39.827033 25065 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:07:39.887270 21213 raft_consensus.cc:2410] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:39.868333 24990 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:39.911063 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.863524 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 8b6dd6862dd14baba4a82c5a837bc802 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:07:39.921454 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.868120 24524 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:07:39.936883 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:39.926326 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.928949 21221 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:39.941650 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.945220 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.946036 21221 raft_consensus.cc:2368] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:39.913100 24965 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:39.944025 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.942803 21213 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:07:39.956594 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.958505 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:39.959828 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.963747 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:39.943997 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 8b6dd6862dd14baba4a82c5a837bc802 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-8b6dd6862dd14baba4a82c5a837bc802
W20260709 13:07:39.968667 25150 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:07:39.970428 25152 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:07:39.971194 25161 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:39.980041 25161 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:39.981491 25161 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:39.984304 25032 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.942790 24524 consensus_queue.cc:1243] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:07:39.985316 25032 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:39.988972 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.989483 25144 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.991671 25172 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:39.991593 25136 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.993530 25136 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:39.994827 25164 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:07:39.924451 24990 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:39.998971 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:39.999931 21213 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:40.001022 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.010619 21221 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:40.012238 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51742.6us
I20260709 13:07:40.015113 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:40.034794 21801 raft_consensus.cc:2410] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:39.934448 25139 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:07:40.027876 25032 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.028751 25144 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.027078 24663 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:40.032299 21231 tablet_copy_source_session.cc:215] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:40.013597 24990 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.016980 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:40.063099 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46246.7us
I20260709 13:07:40.064107 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:39.877635 25147 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.035008 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:40.056000 21221 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:40.065284 21799 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:40.071273 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:40.060107 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:40.058261 25036 consensus_meta.cc:287] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.712s	user 0.013s	sys 0.000s
I20260709 13:07:40.081784 21220 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:40.075870 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:40.077536 25084 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:40.069739 25000 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.084306 25022 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:40.085971 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:40.064939 25162 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:07:40.069875 25147 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:40.122726 24970 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:40.123183 24970 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:40.129669 25031 consensus_meta.cc:287] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.636s	user 0.005s	sys 0.006s
I20260709 13:07:40.130127 25031 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.131322 21214 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:40.137593 21792 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:39.723558 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:40.059111 24663 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:40.109030 24942 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.002s
W20260709 13:07:40.112147 25082 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.849s	user 0.005s	sys 0.006s
I20260709 13:07:40.092350 25084 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:40.122011 25040 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.000s
W20260709 13:07:40.143117 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 57241.1us
I20260709 13:07:40.097110 24973 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:40.160480 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:40.152172 25013 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.695s	user 0.001s	sys 0.010s
W20260709 13:07:40.170253 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.171909 21220 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:40.175995 24927 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:40.176631 24927 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:40.177781 24927 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.177860 24663 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.179774 24663 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:40.152671 24690 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:40.153956 24818 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:40.158524 25084 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.164801 25082 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:40.155413 24942 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.210687 25178 raft_consensus.cc:993] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:40.211926 24942 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:40.160565 24761 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:07:40.213043 24927 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:40.213676 24761 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:40.214212 24761 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:40.186472 25013 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:40.191606 25128 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.226558 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.227473 21220 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:40.222355 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:40.196195 24995 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:40.225353 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.217067 24973 raft_consensus.cc:1081] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:40.231004 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:40.232966 21220 raft_consensus.cc:2368] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:07:40.234226 25097 ts_tablet_manager.cc:1916] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260709 13:07:40.234386 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.234488 24766 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.235064 21212 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:07:40.232146 25089 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:40.235946 25089 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:40.237118 21218 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:40.238389 21212 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:40.237113 25089 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:40.239758 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.229034 25082 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:40.241572 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 474) took 2714 ms. Trace:
I20260709 13:07:40.242023 25082 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.242197 21216 rpcz_store.cc:276] 0709 13:07:37.526990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.139059 (+612069us) service_pool.cc:224] Handling call
0709 13:07:38.139466 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.834467 (+695001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.241249 (+1406782us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b5400102f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:40.241494 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597047680}
I20260709 13:07:40.244485 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:40.245427 25018 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:40.245645 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:40.245858 25018 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.246366 25018 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:40.246812 25082 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.247923 25170 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:40.248513 25170 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:40.253894 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 244193us
I20260709 13:07:40.255003 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.255883 24973 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:40.257324 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:40.258827 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:40.264454 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:40.265111 25183 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:40.266396 25009 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:40.266810 25009 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:40.267243 25192 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:40.267764 25009 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.269047 25009 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:40.256502 25153 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.423s	user 0.005s	sys 0.000s
I20260709 13:07:40.271214 24973 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:40.273471 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.274083 24973 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:40.275060 24973 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.247928 25111 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.276831 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.249689 25170 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.277292 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.259810 24990 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:40.277398 21143 raft_consensus.cc:2368] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:40.261763 25178 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:07:40.277678 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.285413 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.284076 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.284425 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:40.284730 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.287569 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:40.288576 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.216660 24761 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.282560 24766 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:40.291451 24766 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:40.291718 24761 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.291878 24766 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.289780 25111 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.304490 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:40.306497 25183 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:40.307148 25183 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:40.307943 21799 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:40.308084 25183 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.311694 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1208) took 1087 ms. Trace:
W20260709 13:07:40.311928 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 435495us
I20260709 13:07:40.312129 21063 rpcz_store.cc:276] 0709 13:07:39.224440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.227386 (+  2946us) service_pool.cc:224] Handling call
0709 13:07:39.227726 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:39.228091 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:39.228120 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:39.228611 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:07:39.661263 (+432652us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:39.661322 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.661419 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:40.206913 (+545494us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:40.311380 (+104467us) raft_consensus.cc:1685] finished
0709 13:07:40.311400 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:40.311619 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":265068,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"spinlock_wait_cycles":54587136,"thread_start_us":432235,"threads_started":1,"wal-append.queue_time_us":562880}
I20260709 13:07:40.315048 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:40.315867 21217 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 7: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:40.316816 25031 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:40.286965 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.324364 24990 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:40.325134 24978 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:40.325615 24990 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.325817 24978 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.327457 25186 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:40.328081 25186 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.294335 25111 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.328925 24978 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.330420 25186 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.330047 24970 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.331482 24970 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.331717 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:40.350499 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.352567 21147 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:40.351197 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:40.355710 25136 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:07:40.361591 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.362308 21217 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:40.357203 25186 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:40.358937 21799 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:07:40.363528 25186 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:40.360440 25025 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:40.364481 25025 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:40.364437 25186 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.365140 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 436) took 1507 ms. Trace:
I20260709 13:07:40.365526 21794 rpcz_store.cc:276] 0709 13:07:38.857614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.857795 (+   181us) service_pool.cc:224] Handling call
0709 13:07:40.364928 (+1507133us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b5400662bd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:40.365052 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760127232}
I20260709 13:07:40.365753 25025 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.366816 25025 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.353721 24766 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.370198 21146 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:40.361120 25018 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:40.363121 25165 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.432s	user 0.001s	sys 0.003s
I20260709 13:07:40.372788 24766 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.373308 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1222) took 1015 ms. Trace:
I20260709 13:07:40.378346 21072 rpcz_store.cc:276] 0709 13:07:39.358122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.441361 (+ 83239us) service_pool.cc:224] Handling call
0709 13:07:39.441764 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:39.442040 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:39.442077 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:39.452443 (+ 10366us) log.cc:844] Serialized 26 byte log entry
0709 13:07:39.778728 (+326285us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:39.778764 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.778830 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:40.362520 (+583690us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:40.373041 (+ 10521us) raft_consensus.cc:1685] finished
0709 13:07:40.373062 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:40.373254 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16610048,"thread_start_us":199004,"threads_started":1,"wal-append.queue_time_us":380954}
W20260709 13:07:40.379974 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.374150 25084 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.382308 25018 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.384562 21143 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:40.386031 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:40.387876 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 54985.8us
W20260709 13:07:40.385955 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53806.1us
I20260709 13:07:40.393527 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:40.396086 21147 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:40.394080 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:40.394140 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.397164 21144 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:40.398227 21143 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:40.404825 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.410148 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.413668 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.390549 24761 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.424585 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.425498 21143 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:40.403246 25018 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.405681 25126 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:40.429747 21143 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:40.434738 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.435012 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:40.433707 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.435734 21216 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:40.431005 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.427302 25191 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:40.438697 21147 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:40.439083 21212 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:40.410995 25178 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:40.420068 24978 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:40.446346 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.446798 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.450139 21220 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:07:40.438314 25184 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:07:40.454859 21138 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:40.441267 25126 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.411684 24997 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:07:40.432034 25139 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:40.426939 25192 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:07:40.463387 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:40.456094 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:40.429227 24761 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.482614 24761 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:40.460248 25126 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:40.483860 21138 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:07:40.484962 25192 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:40.485380 25192 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:40.486383 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1220) took 1155 ms. Trace:
I20260709 13:07:40.486223 25192 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.486714 25163 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:40.456638 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:40.462158 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.457310 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:40.464095 21213 raft_consensus.cc:2376] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10. Re-sending same reply.
I20260709 13:07:40.470139 21801 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:40.474184 21799 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:40.491331 21799 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:07:40.439704 25018 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.464285 25139 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.495146 25163 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.504357 25097 ts_tablet_manager.cc:1929] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20260709 13:07:40.505137 25163 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.488229 21065 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:40.505756 21065 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:40.507213 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.508419 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:40.508600 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.509349 21216 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:40.510314 25023 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.489382 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.510842 25023 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.512174 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.512460 25145 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:40.512794 21213 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:40.513034 25157 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.513319 21213 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:40.513639 21036 connection.cc:256] RPC call timeout handler was delayed by -3.1740005589999782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
I20260709 13:07:40.513583 25157 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.514423 21221 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:40.514539 25000 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:40.496479 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.514994 25000 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.492048 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.511112 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:40.516599 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.486799 21069 rpcz_store.cc:276] 0709 13:07:39.331443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.359358 (+ 27915us) service_pool.cc:224] Handling call
0709 13:07:39.437313 (+ 77955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.437630 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:39.498517 (+ 60887us) log.cc:844] Serialized 14 byte log entry
0709 13:07:39.800493 (+301976us) log.cc:844] Serialized 14 byte log entry
0709 13:07:40.238096 (+437603us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18003d2ee0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:40.485840 (+247744us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b18003d2ee0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287 00007f7f42537208
0709 13:07:40.486007 (+   167us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:40.486068 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:40.486113 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.486182 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:40.486322 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":718752,"prepare.queue_time_us":62849,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":1678,"raft.queue_time_us":687447,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":13484,"spinlock_wait_cycles":52077696,"thread_start_us":301271,"threads_started":1,"wal-append.queue_time_us":384512}
W20260709 13:07:40.514159 21036 connection.cc:256] RPC call timeout handler was delayed by -2.9202005589999089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260709 13:07:40.517894 21036 connection.cc:256] RPC call timeout handler was delayed by -2.871600558999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
I20260709 13:07:40.518038 21214 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:40.480525 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 801) took 1204 ms. Trace:
W20260709 13:07:40.518369 21036 connection.cc:256] RPC call timeout handler was delayed by -2.8347005589999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
I20260709 13:07:40.496765 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1224) took 1043 ms. Trace:
I20260709 13:07:40.505306 25097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6dd6862dd14baba4a82c5a837bc802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.519063 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.530043 21142 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:40.530470 21066 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:40.517921 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.517052 25000 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.533953 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.531062 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:40.537703 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1250) took 1172 ms. Trace:
I20260709 13:07:40.538239 21218 rpcz_store.cc:276] 0709 13:07:39.364810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.366558 (+  1748us) service_pool.cc:224] Handling call
0709 13:07:40.537517 (+1170959us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54003b4cd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:40.537638 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1818926336}
I20260709 13:07:40.535081 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1255) took 1165 ms. Trace:
I20260709 13:07:40.547680 21215 rpcz_store.cc:276] 0709 13:07:39.369708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.370193 (+   485us) service_pool.cc:224] Handling call
0709 13:07:39.370563 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.873864 (+503301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.534863 (+660999us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b540015c054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:40.535025 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1155668352}
I20260709 13:07:40.517832 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.518410 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.550277 21063 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:40.494618 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.518621 21800 rpcz_store.cc:276] 0709 13:07:39.276086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.276194 (+   108us) service_pool.cc:224] Handling call
0709 13:07:40.480310 (+1204116us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54005429d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:40.480463 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496419328}
I20260709 13:07:40.519160 21067 rpcz_store.cc:276] 0709 13:07:39.453686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.480807 (+ 27121us) service_pool.cc:224] Handling call
0709 13:07:39.579800 (+ 98993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.580048 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:39.863959 (+283911us) log.cc:844] Serialized 14 byte log entry
0709 13:07:40.496418 (+632459us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:40.496467 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:40.496510 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.496611 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:40.496711 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":496154,"prepare.queue_time_us":72544,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":455660,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":388203,"threads_started":2,"wal-append.queue_time_us":307919}
W20260709 13:07:40.518826 21036 connection.cc:256] RPC call timeout handler was delayed by -2.13900055900001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28530000000000005s for a response
W20260709 13:07:40.551906 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8375005589999773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31880000000000003s for a response
W20260709 13:07:40.552421 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5738005590000057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3481s for a response
I20260709 13:07:40.553263 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:40.519850 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.532289 21065 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.555631 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.557319 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:40.531697 21069 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:40.554641 21064 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:40.531441 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:40.564075 21143 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:40.498559 25192 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.552435 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:40.565034 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.565826 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.568403 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:40.569156 21140 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:40.567706 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:40.568562 21071 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.571166 21068 raft_consensus.cc:2368] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12.
I20260709 13:07:40.505137 25139 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.571831 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.608337 21792 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:40.609757 25139 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:40.573625 25100 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:40.579592 21145 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:40.581118 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.612872 21184 leader_election.cc:400] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12.
I20260709 13:07:40.613170 21184 leader_election.cc:403] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:07:40.536444 25036 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:07:40.572479 24662 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:07:40.513075 25145 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.620848 25100 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:40.621837 25100 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.622430 25145 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.556931 24761 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.623378 24761 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:40.623916 25082 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:40.617851 25134 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:07:40.625828 25097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:54634 (request call id 794) took 2856 ms. Trace:
I20260709 13:07:40.626340 25134 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:40.626788 25134 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:40.584388 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:40.584625 25018 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:40.585098 25056 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:40.585578 25195 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:07:40.532644 25182 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:40.586594 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.587459 25000 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:40.591239 21071 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:40.592092 25023 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.593757 24995 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:40.598785 21065 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.532358 25147 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:40.600713 25157 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.565132 25136 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:40.603242 24997 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.612632 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.619840 25031 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12.
I20260709 13:07:40.610549 25139 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:07:40.620366 25036 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:40.630510 21143 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:40.570253 25163 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:40.633899 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:40.634531 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:40.670140 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 57600.9us
I20260709 13:07:40.637285 21066 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:40.672760 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.673465 21066 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:40.674420 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.636497 21063 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:40.644954 25182 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.675880 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.676429 21063 raft_consensus.cc:2376] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11. Re-sending same reply.
I20260709 13:07:40.677078 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.677733 21063 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:40.678248 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.679011 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.644052 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:40.646204 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 59694.9us
I20260709 13:07:40.680861 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:40.626277 25097 rpcz_store.cc:276] 0709 13:07:37.776520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.781121 (+  4601us) service_pool.cc:224] Handling call
0709 13:07:38.146194 (+365073us) ts_tablet_manager.cc:934] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
0709 13:07:40.220083 (+2073889us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:07:40.275493 (+ 55410us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:40.504215 (+228722us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:40.504973 (+   758us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 13:07:40.624958 (+119985us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:07:40.625284 (+   326us) tablet_replica.cc:206] Creating consensus instance
0709 13:07:40.625660 (+   376us) ts_tablet_manager.cc:989] Replica registered.
0709 13:07:40.625731 (+    71us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":42168,"tablet-copy.queue_time_us":159055,"thread_start_us":349305,"threads_started":2}
I20260709 13:07:40.617446 24818 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:40.656111 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:40.625828 24761 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:40.665817 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.683845 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.699707 21066 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:40.653242 24995 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:40.700941 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:40.684849 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:40.686928 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.702720 21063 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:40.647341 25145 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:40.691128 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.692198 21794 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:40.706014 21217 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:40.706060 21065 raft_consensus.cc:2376] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9. Re-sending same reply.
I20260709 13:07:40.706209 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.707860 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.708060 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.708814 21065 raft_consensus.cc:2376] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9. Re-sending same reply.
I20260709 13:07:40.709975 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.695715 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.708276 21220 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:40.703893 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.712502 21145 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:40.691628 25000 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:40.672895 25036 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:40.716738 21110 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.717976 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.688207 25162 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:40.689391 25182 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:40.673918 25082 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:40.703897 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:40.721282 25157 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:40.719164 25163 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:40.659770 25147 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.722373 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.721820 25126 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.722998 21146 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:40.723992 25097 tablet_copy_client.cc:806] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:40.724619 25097 tablet_copy_client.cc:670] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:40.724934 25126 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:40.701442 24997 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.720240 21036 leader_election.cc:400] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:40.728161 21036 leader_election.cc:403] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:40.714895 21063 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:40.729936 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.730577 21063 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:40.730577 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.731448 21064 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:40.732239 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.732769 21064 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:40.733525 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.734037 21064 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:40.734746 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.736722 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.732784 21066 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:40.737805 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.738281 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.738881 21140 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:07:40.741114 21064 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:40.741751 21066 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:40.742507 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.743351 21066 raft_consensus.cc:2410] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:07:40.741861 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.745589 21064 raft_consensus.cc:2376] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10. Re-sending same reply.
I20260709 13:07:40.704766 25013 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.691876 25186 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:40.754678 21065 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
W20260709 13:07:40.747098 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.715078 25039 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:40.755023 21217 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:07:40.749713 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.730064 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:40.759694 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:40.760370 21146 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:40.760535 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:40.761173 21142 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:40.759150 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:40.719240 24978 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.753532 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.768024 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.755772 21767 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:40.747516 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.756765 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:40.769614 21066 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:40.769760 21063 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:40.692945 25136 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:40.770787 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.771636 21066 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:40.773398 21064 raft_consensus.cc:2410] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:07:40.747339 25178 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:40.751186 25184 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:40.781247 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.798105 21064 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:40.757318 25186 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.800606 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.812222 21063 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:40.715178 25025 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:07:40.802675 21036 leader_election.cc:400] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:40.822883 21036 leader_election.cc:403] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:40.756842 25145 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
W20260709 13:07:40.793934 25200 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:07:40.775009 25000 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:40.774641 24851 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:40.806695 21185 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:40.799980 25184 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.814528 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:40.848839 21064 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.787144 25039 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:40.843271 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:40.795077 24970 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.846184 21066 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:40.838883 21142 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
W20260709 13:07:40.828665 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 100953us
I20260709 13:07:40.799257 25023 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:40.824208 25186 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:40.823956 25163 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:40.757411 25013 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.774466 25033 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:07:40.797421 25009 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:07:40.794426 24978 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.848274 24927 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 6. Current term is 7.
I20260709 13:07:40.850850 25184 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:40.850344 25025 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:40.860297 25023 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.872795 24978 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.900115 25018 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.900882 25018 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.901489 25185 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:40.901942 25185 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:40.903677 25018 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:40.874511 25203 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:07:40.874910 24997 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.876598 25013 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.861402 25000 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:40.840159 25036 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.878715 24995 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:40.881132 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 119483us
I20260709 13:07:40.868021 25147 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:40.885566 25134 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.865115 24851 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:40.906422 25084 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:40.917981 24995 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:40.918744 25036 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.896189 25025 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.862144 24970 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.907588 25023 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:40.941613 24818 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:40.942049 24818 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:40.943427 25025 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:40.944372 21215 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:40.945639 25134 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:40.942821 24818 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.950093 25170 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.863876 25136 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:40.923202 21794 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:40.924901 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.925119 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:40.952278 21218 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
W20260709 13:07:40.912670 25204 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:07:40.896869 25202 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:07:40.927647 21800 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:40.930429 24851 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:40.933248 25206 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:07:40.939955 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:40.940187 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:40.940258 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:40.951128 24818 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.925868 25039 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.954692 25161 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.958410 25161 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.959227 24970 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.965559 25161 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.966010 25023 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.968155 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:40.902734 25185 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.930341 25182 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.955113 25186 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:40.969440 25056 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:40.972079 25161 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:40.978562 25097 tablet_copy_client.cc:538] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:40.950085 25184 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.979807 25056 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.981714 25178 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:40.982184 25185 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:40.983605 25161 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:40.994220 25161 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:40.998523 25182 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.000530 25186 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:40.858331 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:41.180095 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 321735us
I20260709 13:07:40.853264 21063 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:07:40.879490 21147 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:07:41.182991 25163 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:41.183559 21147 pending_rounds.cc:85] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 8
I20260709 13:07:41.183835 25163 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.184142 21147 pending_rounds.cc:107] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:07:41.184502 25163 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:41.184585 21147 raft_consensus.cc:2889] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:41.001439 25184 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.064519 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.188896 21066 raft_consensus.cc:2410] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:07:41.189116 21138 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:41.190291 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.191449 21066 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:40.992041 25056 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:41.096011 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.097741 25082 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.110509 24927 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.114593 25147 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:41.114867 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.103763 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.102303 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.116359 20968 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.118273 25157 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:41.198834 25082 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.199885 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:41.199797 25157 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.202407 24927 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:41.199024 21072 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:41.127770 25084 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:41.128211 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:41.106181 25000 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.105083 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.158088 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.160054 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.137686 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:41.095230 25023 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:41.161299 25018 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:07:41.203258 25157 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.172097 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:41.176061 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:41.167459 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:41.206966 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 93719.7us
I20260709 13:07:41.208357 21070 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:41.204650 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.208835 21071 raft_consensus.cc:2368] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12.
I20260709 13:07:41.209645 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.209164 21067 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:41.207449 25145 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:41.234368 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.234807 25145 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:41.235368 25145 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:41.239581 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:41.239740 21187 leader_election.cc:336] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.509s (SENT)
I20260709 13:07:41.240425 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:41.240523 21069 raft_consensus.cc:2376] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9. Re-sending same reply.
W20260709 13:07:41.242197 21187 leader_election.cc:336] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.598s (SENT)
I20260709 13:07:41.242707 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:41.243148 24818 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:07:41.244115 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.245747 21215 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:41.232836 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.248668 21066 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:41.233659 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:41.259996 21072 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:41.266127 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 72270.5us
I20260709 13:07:41.205729 24927 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:41.273355 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:41.274309 21147 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:41.275409 21801 raft_consensus.cc:2368] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:07:41.275872 21215 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:07:41.278919 21217 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:41.207791 25000 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:41.283823 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.288319 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:41.284009 21138 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:41.301064 21067 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:41.301755 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59651.9us
W20260709 13:07:41.303089 21187 leader_election.cc:336] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.626s (SENT)
I20260709 13:07:41.303586 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:41.301913 21067 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:07:41.304843 21065 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:41.209486 25147 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.283228 25000 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:41.314296 21218 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:41.315176 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:41.315863 21147 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:41.320060 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.308988 24818 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:07:41.325481 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 72143.3us
I20260709 13:07:41.326795 21063 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:41.328907 21069 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:41.331899 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 43787.5us
I20260709 13:07:41.294081 25084 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:41.336069 21071 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:41.336607 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 63790.8us
I20260709 13:07:41.337172 21063 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:41.347245 21069 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:41.349330 25082 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:41.352160 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:41.352444 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:41.352983 21219 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:41.353186 21212 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:41.354710 21220 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:41.355225 21215 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:41.356639 21072 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:41.307519 25145 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:41.361040 21138 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
W20260709 13:07:41.363127 21187 leader_election.cc:336] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.636s (SENT)
I20260709 13:07:41.367375 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.359216 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:41.403770 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:41.406842 21067 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:07:41.408953 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53248.1us
I20260709 13:07:41.409138 21218 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:41.423990 21072 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:41.425750 21072 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:41.428542 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69346.2us
W20260709 13:07:41.430778 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 104454us
I20260709 13:07:41.431957 25018 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:41.432385 25018 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.432719 25018 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:41.434242 21064 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.008s
I20260709 13:07:41.434710 21064 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:41.440459 25147 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.437608 25145 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:41.437369 21067 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:41.439874 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.447971 21068 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:41.449129 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 86070.4us
W20260709 13:07:41.451514 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.363675 25000 raft_consensus.cc:2764] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:07:41.437943 25056 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:41.469043 25222 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:07:41.470435 21217 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:41.460930 25082 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.476235 25082 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.478065 21065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39510
I20260709 13:07:41.478681 25082 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:41.478919 21065 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:41.479352 21065 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:41.479475 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:41.482146 24965 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:41.482373 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:41.484920 24978 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:41.483520 25036 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:41.517305 21217 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:41.518685 21219 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:41.515647 24978 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:41.521103 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 62059.1us
I20260709 13:07:41.523918 25023 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:41.527998 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44719.6us
I20260709 13:07:41.532575 21071 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:41.536123 25036 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:41.536568 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.536741 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:41.536969 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:41.536844 25036 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.531476 25023 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:41.541172 25023 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.541272 25036 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.524251 24978 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:41.546092 25184 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:41.553857 25184 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:41.555119 25184 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:41.551231 21065 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.563234 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:41.563859 25025 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.620s	user 0.014s	sys 0.011s
I20260709 13:07:41.563014 25136 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:07:41.572474 21187 leader_election.cc:336] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.684s (SENT)
I20260709 13:07:41.550761 25134 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:41.575770 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:41.578233 24997 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:07:41.580758 21036 leader_election.cc:400] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:07:41.581168 21036 leader_election.cc:403] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:07:41.582069 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:41.582602 25185 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.589s	user 0.018s	sys 0.003s
I20260709 13:07:41.585915 25025 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.582763 25163 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.585076 25036 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:41.610282 24997 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:41.616869 24997 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:41.617870 24997 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.619544 21066 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:41.619819 21064 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:41.601773 21065 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.610900 21070 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
W20260709 13:07:41.612129 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.613977 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:41.615350 25163 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:41.621289 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 70226.7us
I20260709 13:07:41.622478 25000 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:07:41.625473 21143 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:41.626106 21071 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:07:41.601608 25134 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:41.619238 24997 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.620903 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2127) took 1094 ms. Trace:
I20260709 13:07:41.627806 25084 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:41.639223 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:41.642745 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:41.645561 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 98578.2us
I20260709 13:07:41.646822 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:41.647657 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.652036 25182 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:41.652256 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 626) took 1133 ms. Trace:
I20260709 13:07:41.655268 21145 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:41.658047 25161 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:41.658533 25161 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.658938 25161 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:41.660883 25097 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:07:41.636245 21070 rpcz_store.cc:276] 0709 13:07:40.526305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.157473 (+631168us) service_pool.cc:224] Handling call
0709 13:07:41.620851 (+463378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:41.653743 21069 rpcz_store.cc:276] 0709 13:07:40.518814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.644393 (+1125579us) service_pool.cc:224] Handling call
0709 13:07:41.644730 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.651863 (+  7133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:41.651904 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:41.651957 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:41.651998 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.652061 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.652195 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:41.679750 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2128) took 1153 ms. Trace:
I20260709 13:07:41.681111 21072 rpcz_store.cc:276] 0709 13:07:40.526596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.426671 (+900075us) service_pool.cc:224] Handling call
0709 13:07:41.427192 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.467400 (+ 40208us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:41.646616 (+179216us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.679401 (+ 32785us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:41.679451 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:41.679492 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.679556 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.679692 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":231,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":107,"thread_start_us":95139,"threads_started":2,"wal-append.queue_time_us":21021}
I20260709 13:07:41.682956 21065 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:41.687412 21072 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:41.690382 25018 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:41.690902 21072 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:41.717664 24978 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.719348 21219 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:41.719884 25025 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:41.719980 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:41.702173 25056 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.726020 21142 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:41.728609 21069 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:41.729388 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1254) took 1012 ms. Trace:
I20260709 13:07:41.729794 21069 rpcz_store.cc:276] 0709 13:07:40.716616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.699524 (+982908us) service_pool.cc:224] Handling call
0709 13:07:41.728383 (+ 28859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.729106 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.729328 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:41.732059 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:41.733132 21215 raft_consensus.cc:2410] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:07:41.733656 24978 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.736009 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1243) took 1084 ms. Trace:
I20260709 13:07:41.736389 21068 rpcz_store.cc:276] 0709 13:07:40.651214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.104518 (+453304us) service_pool.cc:224] Handling call
0709 13:07:41.735838 (+631320us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540041c654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:41.735952 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458839424}
I20260709 13:07:41.737584 25018 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.741843 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:41.742522 21212 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:41.768877 21212 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
W20260709 13:07:41.778908 21187 leader_election.cc:336] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.728s (SENT)
I20260709 13:07:41.794217 25033 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.794868 25033 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.795819 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:41.796522 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2130) took 1021 ms. Trace:
I20260709 13:07:41.771945 25161 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:41.796882 21063 rpcz_store.cc:276] 0709 13:07:40.774701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.182109 (+407408us) service_pool.cc:224] Handling call
0709 13:07:41.796338 (+614229us) spinlock_profiling.cc:243] Waited 91.6 ms on lock 0x7b5400214b54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:41.796462 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210731264}
I20260709 13:07:41.797760 25033 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:41.798907 25161 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.757458 25018 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:07:41.799985 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.800191 25018 consensus_queue.cc:1243] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:41.800664 21072 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:41.805384 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2126) took 1279 ms. Trace:
I20260709 13:07:41.805771 21067 rpcz_store.cc:276] 0709 13:07:40.525995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.426281 (+900286us) service_pool.cc:224] Handling call
0709 13:07:41.435353 (+  9072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.443404 (+  8051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.805163 (+361759us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b54003bf450. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:41.805296 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6176000}
I20260709 13:07:41.807572 21070 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:41.808564 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.809212 21069 raft_consensus.cc:2376] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14. Re-sending same reply.
I20260709 13:07:41.810175 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.810761 21069 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:41.810982 21072 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:41.811945 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1248) took 1154 ms. Trace:
I20260709 13:07:41.812011 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.812855 21072 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:07:41.814018 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:41.815376 20968 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:41.816862 20968 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 5 to 8, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.761061 25134 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.820199 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:41.823704 21142 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:07:41.827661 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.828470 21063 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:41.830888 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.834012 21063 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 7: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:41.835003 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.807022 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.836416 24970 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:41.837081 25182 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:41.835059 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 201009us
I20260709 13:07:41.849753 21067 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:41.850205 25182 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.850735 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.851526 21067 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:41.831956 25223 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.280s	user 0.003s	sys 0.000s
I20260709 13:07:41.837551 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.818495 20968 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 7 to 11, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 11 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:41.841228 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:41.843139 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:41.843179 21068 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:41.845325 24851 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:41.845376 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1256) took 1098 ms. Trace:
I20260709 13:07:41.854911 21070 rpcz_store.cc:276] 0709 13:07:40.747267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.686515 (+939248us) service_pool.cc:224] Handling call
0709 13:07:41.807388 (+120873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.808042 (+   654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.845145 (+ 37103us) spinlock_profiling.cc:243] Waited 93.3 ms on lock 0x7b5400214b50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:41.845310 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214555904}
I20260709 13:07:41.854249 21147 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:41.846863 25033 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:41.812511 21071 rpcz_store.cc:276] 0709 13:07:40.657705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.551410 (+893705us) service_pool.cc:224] Handling call
0709 13:07:41.573625 (+ 22215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:41.626557 (+ 52932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.811716 (+185159us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b5400232dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:41.811894 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106834816}
W20260709 13:07:41.856994 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 205876us
I20260709 13:07:41.849324 21072 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:41.863145 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:41.872352 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.884071 21069 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:41.887955 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:41.888890 21070 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:41.894178 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.897261 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:41.898074 21138 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:41.894927 21063 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:41.900022 21072 raft_consensus.cc:2376] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10. Re-sending same reply.
I20260709 13:07:41.901633 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.849777 24970 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:41.905506 21142 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:07:41.906267 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 139074us
I20260709 13:07:41.906461 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:41.903450 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.907097 21218 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:41.907460 21070 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:07:41.910110 21065 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:41.887477 25056 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:41.915899 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 72878.8us
I20260709 13:07:41.914463 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:41.896339 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5294801649999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:07:41.917646 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:41.920236 21068 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:41.917768 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:41.921300 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:41.921628 21213 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:41.859997 25018 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 13:07:41.921326 20969 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 257) took 45160 ms (client timeout 15000 ms). Trace:
I20260709 13:07:41.923688 21070 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:41.923759 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:41.893646 20968 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 4 to 9, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 9 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.919735 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1249) took 1249 ms. Trace:
I20260709 13:07:41.927101 21035 leader_election.cc:400] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:07:41.927439 21035 leader_election.cc:403] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:41.934713 21071 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.936254 24997 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:41.936718 24997 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.936820 25056 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:41.937134 24997 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:41.937563 25056 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:41.940716 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:41.940150 20968 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 11, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 11 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.941473 21147 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:41.942308 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:41.942054 20968 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 4 to 8, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.943893 25126 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:07:41.944425 25178 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.944933 25178 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.945940 24970 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:41.947578 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1250) took 1265 ms. Trace:
I20260709 13:07:41.947927 21064 rpcz_store.cc:276] 0709 13:07:40.682597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.623851 (+941254us) service_pool.cc:224] Handling call
0709 13:07:41.657201 (+ 33350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:41.702430 (+ 45229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:41.702473 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:41.703054 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:07:41.773921 (+ 70867us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:41.773957 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.774030 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:41.947378 (+173348us) raft_consensus.cc:1685] finished
0709 13:07:41.947397 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.947525 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12181,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":2018,"thread_start_us":66280,"threads_started":1,"wal-append.queue_time_us":209114}
I20260709 13:07:41.949393 25000 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:07:41.950505 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:41.951251 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 63569.7us
I20260709 13:07:41.953081 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:41.953426 25084 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:41.958915 21068 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:41.946997 25178 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:41.969547 21138 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:41.919076 25163 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:41.976438 20968 catalog_manager.cc:5697] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:41.938732 25056 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:41.983022 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:41.985971 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:41.987493 21792 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:41.983381 24851 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.991523 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:41.996470 21218 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:41.962878 25134 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.994689 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:42.016438 25084 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:42.025748 21220 raft_consensus.cc:2368] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:42.018002 24851 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.029027 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:42.029376 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.030604 21138 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:42.031041 21138 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:42.029204 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:41.933375 25023 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:07:42.051508 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:42.052623 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.041307 20968 catalog_manager.cc:5697] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 8, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.000862 25184 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:42.056135 20968 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 6 to 9, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 9 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.068977 21213 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:42.069177 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.081532 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:42.082878 21218 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:42.084650 21063 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:42.085817 21067 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:42.060835 21138 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.082404 21216 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:42.088248 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.052253 24965 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.057077 25033 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.098861 21145 raft_consensus.cc:2410] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:07:42.098661 25033 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:41.925384 21066 rpcz_store.cc:276] 0709 13:07:40.670762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.623005 (+952243us) service_pool.cc:224] Handling call
0709 13:07:41.623607 (+   602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.637054 (+ 13447us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:41.726226 (+ 89172us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.907910 (+181684us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.919317 (+ 11407us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:41.919368 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:41.919419 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.919569 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.919683 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11074,"prepare.queue_time_us":45766,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":25366,"spinlock_wait_cycles":11037568,"thread_start_us":161111,"threads_started":2,"wal-append.queue_time_us":145526}
I20260709 13:07:42.101526 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.103019 25242 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:42.027211 25163 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.087759 25039 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:42.096657 24965 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.097790 21146 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:42.103274 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:42.095703 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:42.112268 21147 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:42.044133 25134 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.129884 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:42.121306 21138 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:42.128656 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.143476 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:42.143347 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 149333us
I20260709 13:07:42.103849 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:42.104895 25240 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:07:42.165122 24851 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.171054 24851 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.171667 24851 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.171877 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.173486 21141 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:42.173920 24851 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.165658 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.165333 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:42.175760 21147 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:42.176067 21146 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:42.176362 21146 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:42.167095 25182 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:42.176949 25182 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:42.177193 21146 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.177879 25182 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.170939 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:42.179461 21140 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:07:42.179890 25025 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:42.180027 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:42.182209 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:42.183365 21185 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:42.095836 25184 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:42.155706 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:42.184769 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.185328 25184 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.168488 21138 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
W20260709 13:07:42.104897 25236 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:07:42.166690 21071 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:42.186888 21146 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:42.187832 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:42.103849 25170 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.157748 21145 raft_consensus.cc:2368] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:07:42.200562 21142 raft_consensus.cc:2410] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:07:42.202617 21216 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:42.212755 21213 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:42.104251 25162 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:42.216078 21220 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
W20260709 13:07:42.198719 21113 leader_election.cc:336] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.507s (SENT)
I20260709 13:07:42.214977 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:42.231817 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 63754.8us
I20260709 13:07:42.235119 21140 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:42.235512 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:42.192728 25136 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.204751 25000 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:42.243398 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.243451 21214 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:42.242686 20968 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 6 to 8, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 8 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:42.250680 24851 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:42.251616 24851 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.254429 24851 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.259253 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.259333 21138 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:42.220290 25162 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:41.923960 20969 rpcz_store.cc:269] 0709 13:06:56.760941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.853710 (+ 92769us) service_pool.cc:224] Handling call
0709 13:06:59.323439 (+2469729us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540039d5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42486090 00007f7f42524f3a 00007f7f482db1cc 00007f7f483d937e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc 00007f7f46899392 00007f7f4085748c 00007f7f408568c6
0709 13:07:41.921034 (+42597595us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:37.340175 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:37.341029 (+   854us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:37.341101 (+    72us) write_op.cc:435] Acquired schema lock
0709 13:07:37.341130 (+    29us) tablet.cc:662] Decoding operations
0709 13:07:37.346968 (+  5838us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:37.347078 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:37.347131 (+    53us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:07:37.347940 (+   809us) tablet.cc:701] Row locks acquired
0709 13:07:37.348000 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:07:37.348233 (+   233us) write_op.cc:270] Start()
0709 13:07:37.348420 (+   187us) write_op.cc:276] Timestamp: P: 1783602457348152 usec, L: 0
0709 13:07:37.348450 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:07:37.349338 (+   888us) log.cc:844] Serialized 2559 byte log entry
0709 13:07:37.642159 (+292821us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b18001873c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:38.043954 (+401795us) op_driver.cc:464] REPLICATION: finished
0709 13:07:38.770079 (+726125us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b18001873c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:07:38.770233 (+   154us) write_op.cc:301] APPLY: starting
0709 13:07:38.770311 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:38.771024 (+   713us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:38.771051 (+    27us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:38.771664 (+   613us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:38.771875 (+   211us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:07:38.771921 (+    46us) write_op.cc:312] APPLY: finished
0709 13:07:38.772169 (+   248us) log.cc:844] Serialized 53 byte log entry
0709 13:07:38.772351 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:38.772459 (+   108us) write_op.cc:454] Released schema lock
0709 13:07:38.772501 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:38.816142 (+ 43641us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:38.816177 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1009217,"reported_tablets":45,"spinlock_wait_cycles":433654016,"child_traces":[["op",{"apply.queue_time_us":629225,"apply.run_cpu_time_us":2718,"apply.run_wall_time_us":11602,"num_ops":6,"prepare.queue_time_us":251461,"prepare.run_cpu_time_us":8501,"prepare.run_wall_time_us":70588,"raft.queue_time_us":245749,"raft.run_cpu_time_us":15081,"raft.run_wall_time_us":629143,"replication_time_us":695320,"spinlock_wait_cycles":27111424,"thread_start_us":955790,"threads_started":4,"wal-append.queue_time_us":68800,"wal-append.run_cpu_time_us":32126,"wal-append.run_wall_time_us":2358082}]]}
I20260709 13:07:42.264410 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:42.187247 25018 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:07:42.281989 21218 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:42.241000 25242 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:42.151949 25039 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.285367 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:42.283905 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 292859us
I20260709 13:07:42.285558 20968 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.291033 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.292114 21214 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:42.295195 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:42.296736 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 270925us
W20260709 13:07:42.297567 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 98887.4us
I20260709 13:07:42.298382 21767 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:42.298641 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.264026 25162 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:42.186641 25084 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.302698 20968 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.239972 25170 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.261281 25249 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:42.259593 25000 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.260135 25136 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.366806 21036 leader_election.cc:400] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:42.365269 21218 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
W20260709 13:07:42.370370 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 61840.8us
I20260709 13:07:42.371837 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.361609 21219 pending_rounds.cc:85] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 10
I20260709 13:07:42.378469 21219 pending_rounds.cc:107] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:07:42.378882 21219 raft_consensus.cc:2889] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:42.304296 25033 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:42.381863 21113 leader_election.cc:336] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.631s (SENT)
I20260709 13:07:42.381042 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:42.386929 21213 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:42.389087 20968 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.356647 25249 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.300225 24851 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:07:42.413142 25244 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:42.413805 25244 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:42.414373 25247 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 13:07:42.415966 25186 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:42.416487 25186 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.416882 25253 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:42.417459 25253 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:42.418412 25253 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.431161 25192 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:42.416917 25186 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:42.431608 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:42.431984 25134 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.434347 25250 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:42.435894 25170 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.436228 20968 catalog_manager.cc:5697] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:42.437588 24965 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:42.442711 24997 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
I20260709 13:07:42.414867 25244 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.444902 24851 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:42.445611 25264 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:42.448155 25163 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.408485 25249 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:42.505103 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 255413us
I20260709 13:07:42.500829 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.506912 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:42.486768 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.454980 25018 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:42.512814 25018 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:42.513899 25018 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:42.462091 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 163565us
I20260709 13:07:42.515760 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:42.508478 21140 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
W20260709 13:07:42.510866 25248 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:07:42.512256 21218 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:42.467712 21214 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:42.506206 20968 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 3 to 7, term changed from 4 to 7, VOTER 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) evicted. New cstate: current_term: 7 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.456768 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:42.519161 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 163905us
I20260709 13:07:42.516280 24851 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:07:42.526263 24851 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:42.526760 24851 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:42.526932 21113 leader_election.cc:336] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.636s (SENT)
I20260709 13:07:42.527441 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:42.530030 25163 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:42.530550 21113 leader_election.cc:336] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.637s (SENT)
I20260709 13:07:42.517352 25247 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.481762 25192 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.531908 25036 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:42.490016 25136 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.531767 25247 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.507022 24997 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.531138 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:42.533466 20968 catalog_manager.cc:5697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 8, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.537652 20968 catalog_manager.cc:5697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 1 to 5, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 5 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.547122 20968 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 10, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 10 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.521329 25242 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:42.554973 21213 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
W20260709 13:07:42.559204 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 260876us
I20260709 13:07:42.528333 25170 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:07:42.556871 25136 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:42.516744 25250 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:42.585021 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 125820us
I20260709 13:07:42.589910 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.594682 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.599416 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.600486 25264 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:42.601315 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.602267 25264 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:42.595521 21069 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
W20260709 13:07:42.607021 25246 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:07:42.609665 25267 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:42.609539 25242 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.610462 25267 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.627480 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:42.628396 25267 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:42.629877 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.630028 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:42.630511 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.630713 21070 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
W20260709 13:07:42.635957 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 105476us
W20260709 13:07:42.640494 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.641287 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.642448 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:42.638568 25134 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:42.644076 25139 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.644835 25268 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.644672 25139 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.645290 25250 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.645485 25268 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.647811 25145 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:42.581153 24851 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.648650 25163 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:42.649255 25163 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:42.650625 21187 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.930s (SENT)
I20260709 13:07:42.651136 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:42.650337 25163 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.656481 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:42.661007 25268 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.647034 25139 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:42.664117 21187 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 2.135s (SENT) [suppressed 42 similar messages]
I20260709 13:07:42.592948 25266 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:07:42.672936 24997 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.638684 25162 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.718616 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:42.719846 25252 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:42.720767 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:42.721158 25162 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:42.722936 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:42.726651 21070 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:42.727442 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:42.729408 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.723546 25036 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:42.740912 21767 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:42.746362 21146 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.000s
I20260709 13:07:42.746766 21146 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:07:42.740128 24997 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:42.725111 25268 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:42.749413 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 85323.4us
I20260709 13:07:42.723888 25192 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:42.750946 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.752492 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.753063 21072 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:42.754369 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:42.757671 25260 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:42.758172 25260 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.758539 25260 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:42.758672 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:42.740952 25036 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:42.760079 25036 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:42.759109 25273 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:07:42.762591 21066 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:42.764756 21070 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:42.764942 25269 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:07:42.766506 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55233.2us
W20260709 13:07:42.766654 25234 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.714s	user 0.002s	sys 0.000s
I20260709 13:07:42.750222 25170 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:42.781273 21064 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:42.793953 25163 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.794653 25163 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.796833 25163 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.809966 21070 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:42.811755 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 88910.9us
I20260709 13:07:42.771874 25269 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.779251 25170 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.814021 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:42.816591 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.820565 21065 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:42.831625 21072 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:42.779004 25268 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.816679 25170 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:42.836228 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.836658 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:42.837267 21065 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:07:42.815480 25269 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:42.840945 25233 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.865s	user 0.001s	sys 0.004s
I20260709 13:07:42.824448 25163 raft_consensus.cc:2764] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:07:42.847740 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 506) took 1148 ms. Trace:
I20260709 13:07:42.848161 21066 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:42.848919 25247 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:42.848155 21217 rpcz_store.cc:276] 0709 13:07:41.699040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.699158 (+   118us) service_pool.cc:224] Handling call
0709 13:07:41.738255 (+ 39097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:41.738764 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:41.738793 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:41.739334 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:07:41.780089 (+ 40755us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:41.780130 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.780190 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:42.838282 (+1058092us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:42.847377 (+  9095us) raft_consensus.cc:1685] finished
0709 13:07:42.847404 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.847657 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1300,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":153,"spinlock_wait_cycles":9585408,"thread_start_us":40306,"threads_started":1,"wal-append.queue_time_us":156280}
I20260709 13:07:42.853410 25268 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:42.854132 21067 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:42.854810 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:42.840760 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 113396us
I20260709 13:07:42.857084 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.858371 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:42.858481 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.868778 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.869807 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.870532 21066 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:42.872102 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:42.873555 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:42.876896 25033 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:42.877612 25033 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:42.877466 25260 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.864964 25247 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:42.886325 21065 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:42.889201 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:42.890996 25243 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:42.895421 24997 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:42.859680 25249 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.900375 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.901300 24927 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:42.895145 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 57489us
W20260709 13:07:42.903381 25237 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.821s	user 0.002s	sys 0.000s
W20260709 13:07:42.904327 25281 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:07:42.865403 25280 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:42.908541 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:42.909394 24970 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:42.909883 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:42.910025 25186 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.910718 24970 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:42.911221 24970 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:42.911814 24970 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:42.912283 25186 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.925019 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:42.902125 25249 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:42.932180 24927 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:07:42.934653 25235 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.893s	user 0.001s	sys 0.001s
I20260709 13:07:42.935377 25260 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:42.936417 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.937389 25162 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:42.937968 25162 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:42.938638 21187 leader_election.cc:400] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12.
I20260709 13:07:42.938902 25162 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.939661 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.940829 25082 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:42.940900 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:42.941318 25082 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.941720 25082 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:42.941973 25000 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:42.942448 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:42.943239 25182 raft_consensus.cc:2764] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:07:42.944787 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:42.945549 25182 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:42.946028 25182 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:42.946833 25182 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:42.952431 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:42.957901 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:42.960073 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:42.967800 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51810us
I20260709 13:07:42.895654 25170 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:42.979949 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:42.980646 21146 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:42.984453 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:42.985074 21145 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:42.992575 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47918.6us
I20260709 13:07:42.982925 25260 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:43.003760 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:43.001709 21146 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
W20260709 13:07:43.005379 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.011807 25242 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.012403 25242 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:43.014627 25269 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:43.023452 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:43.024140 21069 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:43.014880 25242 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:43.017889 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:43.025571 25245 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:43.027637 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:43.033025 21144 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:43.035211 25269 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:43.035919 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:43.036639 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:43.035989 25260 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:43.037909 25260 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:43.012961 25170 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:43.040177 25036 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:43.036680 21063 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:43.042207 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 101394us
I20260709 13:07:43.047001 21145 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:43.048269 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:43.036924 25252 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:43.044911 25139 raft_consensus.cc:2764] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
W20260709 13:07:43.068849 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 51086.4us
I20260709 13:07:43.069792 21036 leader_election.cc:400] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:43.070091 21036 leader_election.cc:403] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:43.071993 21069 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:43.073566 21767 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:43.075676 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:43.076587 21146 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:43.062141 25162 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:43.061914 25242 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:43.083477 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:43.085846 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:43.050379 25249 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:07:43.051560 25178 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:07:43.088776 21063 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:43.077687 25023 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:43.075906 25252 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:43.077183 24927 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:43.093494 21767 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:43.103821 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.091711 25162 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:43.087697 24818 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:07:43.103114 25145 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:43.104967 25023 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:43.112964 24818 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:07:43.113247 25295 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:07:43.113461 24818 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:43.113025 25023 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:43.116060 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2834) took 1082 ms. Trace:
I20260709 13:07:43.116495 21142 rpcz_store.cc:276] 0709 13:07:42.033677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.033858 (+   181us) service_pool.cc:224] Handling call
0709 13:07:43.115814 (+1081956us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54002e3e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:43.115978 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337489792}
I20260709 13:07:43.102658 25182 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:07:43.131600 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 68117.8us
W20260709 13:07:43.134886 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.114509 24818 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:43.138746 25039 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:43.139820 24818 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:43.107702 25269 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:43.151059 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:43.152308 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:43.155328 24927 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:43.155900 24927 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:43.156802 24927 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:43.165122 25269 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:43.199445 25182 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:43.216732 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:43.196094 25036 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:43.195621 25268 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:43.221876 21145 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:43.222186 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:43.225801 21067 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:07:43.226498 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.213610 25097 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:43.221876 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:43.227980 21792 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:43.199119 24970 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:43.264102 21138 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:43.265188 24970 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:43.269866 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 177443us
W20260709 13:07:43.297108 25301 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:07:43.235841 25182 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:43.239015 25170 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:07:43.206445 25252 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:43.329079 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 101597us
I20260709 13:07:43.208686 25162 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:43.266986 25036 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:07:43.247524 25268 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:43.311369 25269 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:43.370265 25309 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:43.347437 25097 tablet_bootstrap.cc:492] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:07:43.319525 25082 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:43.454114 25260 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:43.654520 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 311333us
W20260709 13:07:43.641090 24997 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.745s	user 0.011s	sys 0.001s
I20260709 13:07:43.651209 25097 ts_tablet_manager.cc:1403] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 1.991s	user 0.033s	sys 0.025s
W20260709 13:07:43.714043 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 345755us
W20260709 13:07:43.732043 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 509312us
W20260709 13:07:43.714300 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 245467us
I20260709 13:07:43.702811 24997 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:43.783792 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50680.4us
I20260709 13:07:43.657100 25260 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:43.746577 25182 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:43.773994 25253 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:43.828811 25000 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.871914 25000 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:43.934777 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 48607.5us
I20260709 13:07:43.914605 24818 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.933776 25097 raft_consensus.cc:359] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:43.922730 24927 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:43.928045 25311 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:07:43.942546 25097 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:07:43.941404 25082 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:43.934815 25315 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:07:43.923193 25253 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:43.944525 24818 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:43.952579 25268 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:43.961918 25268 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.962589 25253 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:43.962934 24818 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:43.964865 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 519) took 1014 ms. Trace:
I20260709 13:07:43.965385 21147 rpcz_store.cc:276] 0709 13:07:42.950913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.951683 (+   770us) service_pool.cc:224] Handling call
0709 13:07:42.976376 (+ 24693us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:43.236227 (+259851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.236267 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:43.260904 (+ 24637us) log.cc:844] Serialized 52 byte log entry
0709 13:07:43.316340 (+ 55436us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:43.316395 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.316493 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:43.964627 (+648134us) raft_consensus.cc:1685] finished
0709 13:07:43.964650 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:43.964805 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1611,"raft.queue_time_us":9744,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":11535,"thread_start_us":301335,"threads_started":2,"wal-append.queue_time_us":644642}
I20260709 13:07:43.962409 25268 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:43.968209 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:43.969112 25268 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:43.969300 24818 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:43.969911 24818 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:43.970686 24997 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:43.947216 24927 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:43.983206 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 653) took 1063 ms. Trace:
I20260709 13:07:43.984550 24927 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:43.970983 24818 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:43.969842 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:43.949318 25097 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:43.983664 21071 rpcz_store.cc:276] 0709 13:07:42.919728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.919851 (+   123us) service_pool.cc:224] Handling call
0709 13:07:43.211838 (+291987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:43.212279 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.212320 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:43.212868 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:07:43.777352 (+564484us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:43.777417 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.777525 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:43.981620 (+204095us) raft_consensus.cc:1685] finished
0709 13:07:43.981645 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:43.982909 (+  1264us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54003ca350. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:43.983109 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":2286,"spinlock_wait_cycles":670662528,"thread_start_us":449343,"threads_started":1,"wal-append.queue_time_us":720931}
I20260709 13:07:43.971647 25260 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:43.946388 25182 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:07:43.992003 25260 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:43.995738 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:43.996547 25082 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:43.997058 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:43.998786 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:43.997886 25082 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:44.016988 25192 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.999778 24965 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:44.023988 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:44.026877 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:44.018733 25313 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:44.037230 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:44.037869 21069 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:07:44.035051 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.040700 21071 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:44.036370 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.042274 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:44.043047 21070 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:44.051923 21072 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:44.052024 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:44.051836 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.052982 21066 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:44.052187 21067 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:44.054055 21069 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:44.057576 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.057721 21069 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:44.058184 21069 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:44.058238 21063 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:44.036074 24927 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:44.064589 24927 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:44.065238 24927 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.036429 25243 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:44.022302 25192 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.070051 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:44.071067 21065 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:44.074941 21065 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:44.075666 21065 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.076251 21065 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:44.047407 25252 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:44.059005 21069 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.038240 24965 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.079495 25097 ts_tablet_manager.cc:1434] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.365s	user 0.012s	sys 0.004s
I20260709 13:07:44.083451 21063 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:44.079842 25243 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.080265 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 525) took 1126 ms. Trace:
I20260709 13:07:44.059484 25268 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:44.081143 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 57289.3us
I20260709 13:07:44.081332 21069 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:44.085479 21070 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:44.086562 24965 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.087114 24965 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.084127 21138 rpcz_store.cc:276] 0709 13:07:42.953630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.953734 (+   104us) service_pool.cc:224] Handling call
0709 13:07:42.953987 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:43.264623 (+310636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.080027 (+815404us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400447ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:44.080192 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712698240}
I20260709 13:07:44.085147 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:44.100113 21069 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:07:44.100643 25336 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.102007 21069 pending_rounds.cc:85] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 4
I20260709 13:07:44.102442 21069 pending_rounds.cc:107] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:07:44.102722 21069 raft_consensus.cc:2889] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:44.104617 25335 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:44.105370 25335 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:44.105544 24818 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:44.105715 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:44.101447 25336 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.106389 25335 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.080862 25252 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:44.108181 25252 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:44.089395 24965 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.110225 24965 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:44.084659 24997 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.084338 25315 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:44.134229 21065 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:07:44.134974 21065 pending_rounds.cc:85] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 6
I20260709 13:07:44.135352 21065 pending_rounds.cc:107] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:07:44.135742 21065 raft_consensus.cc:2889] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:44.138482 21066 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:44.113161 25162 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:44.140307 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:44.142405 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.102767 25268 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.122188 24927 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.146443 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.122059 24997 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.152657 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.153347 21138 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:44.154165 24997 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:44.139389 25329 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.147332 21147 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:44.147444 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.155174 25329 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.155794 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.156267 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.107146 24818 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:07:44.146797 25243 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:44.156927 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.156270 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.157435 25329 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:44.157629 21142 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:44.129347 25315 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.157893 21147 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:44.154155 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.162495 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:44.171777 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.174043 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.174376 25312 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:44.174937 25312 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:44.175992 25192 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:44.177170 21066 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:44.186731 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:44.159660 25315 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.192296 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:44.204768 25336 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.205209 25000 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.211716 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.196884 25145 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:44.219547 25162 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:44.221596 24997 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:44.221820 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:44.223060 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:44.232710 25331 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:07:44.175928 25312 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.237459 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.244294 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2383) took 1274 ms. Trace:
I20260709 13:07:44.279673 21071 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
W20260709 13:07:44.255244 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52753.8us
W20260709 13:07:44.264241 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 72014.7us
I20260709 13:07:44.285945 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.243024 25260 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:44.270681 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.273217 21792 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:44.273861 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 523) took 1321 ms. Trace:
I20260709 13:07:44.289067 21146 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:44.256096 24818 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:44.197340 25136 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:44.291877 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 70168.1us
I20260709 13:07:44.284785 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:44.308998 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:44.309755 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.311431 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.275310 25145 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.265064 25313 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:44.290481 21140 rpcz_store.cc:276] 0709 13:07:42.952890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.952998 (+   108us) service_pool.cc:224] Handling call
0709 13:07:42.976452 (+ 23454us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:42.983951 (+  7499us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:43.191131 (+207180us) log.cc:844] Serialized 14 byte log entry
0709 13:07:44.016769 (+825638us) spinlock_profiling.cc:243] Waited 9.54 ms on lock 0x7b48001a01d8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:07:44.242238 (+225469us) log.cc:844] Serialized 14 byte log entry
0709 13:07:44.242447 (+   209us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:44.243108 (+   661us) log.cc:844] Serialized 52 byte log entry
0709 13:07:44.243292 (+   184us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:44.243334 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.243412 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:44.273562 (+ 30150us) raft_consensus.cc:1685] finished
0709 13:07:44.273594 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.273779 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":887979,"prepare.queue_time_us":964565,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":915,"raft.queue_time_us":108237,"raft.run_cpu_time_us":21020,"raft.run_wall_time_us":1060670,"spinlock_wait_cycles":21935360,"thread_start_us":274602,"threads_started":2,"wal-append.queue_time_us":625639}
I20260709 13:07:44.263443 25252 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.279605 21139 rpcz_store.cc:276] 0709 13:07:42.969749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.984524 (+ 14775us) service_pool.cc:224] Handling call
0709 13:07:43.002274 (+ 17750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:43.002492 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:43.088395 (+ 85903us) log.cc:844] Serialized 14 byte log entry
0709 13:07:44.243775 (+1155380us) log.cc:844] Serialized 14 byte log entry
0709 13:07:44.243961 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:44.244007 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:44.244054 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.244115 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.244225 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":768340,"prepare.queue_time_us":822822,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":75,"raft.queue_time_us":295964,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":21853,"thread_start_us":459828,"threads_started":2,"wal-append.queue_time_us":734402}
I20260709 13:07:44.323988 25252 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:44.325357 21216 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:44.326706 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.327390 21792 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:44.307889 24818 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.308805 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:44.328449 21217 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:44.328862 21215 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:44.329449 24818 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.329875 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.317490 25136 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.330585 21215 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:07:44.332113 25136 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.327318 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.334923 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:44.336243 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:44.337699 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:44.319871 25339 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:44.272601 24997 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:44.320438 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:44.333029 21220 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:44.342322 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.348348 21792 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:44.289795 21138 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
W20260709 13:07:44.331959 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 226311us
I20260709 13:07:44.349985 21221 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:44.345897 21066 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:07:44.351047 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:44.267217 25308 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:07:44.282877 25267 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:44.329737 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.351656 21064 raft_consensus.cc:2376] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14. Re-sending same reply.
I20260709 13:07:44.351081 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.313627 25326 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:44.350744 24965 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:44.353062 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:44.353346 25327 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:44.353857 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:44.281713 25162 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.359833 21213 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:44.361579 25182 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:07:44.371505 25346 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.173s	user 0.002s	sys 0.003s
I20260709 13:07:44.372416 25315 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:44.339506 25145 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:44.340229 25312 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:44.347782 25339 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.349658 24997 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:44.354017 25139 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:44.384935 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2152) took 1509 ms. Trace:
I20260709 13:07:44.379693 25243 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:44.386086 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.408569 25192 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:44.427604 21064 rpcz_store.cc:276] 0709 13:07:42.867858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.867999 (+   141us) service_pool.cc:224] Handling call
0709 13:07:44.350943 (+1482944us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:07:44.376973 (+ 26030us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8504448}
I20260709 13:07:44.410537 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.433590 21215 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:44.384616 25326 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:44.424134 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 73273.2us
I20260709 13:07:44.432183 25139 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:44.439361 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:44.436708 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.428296 25247 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:44.384867 25267 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.349022 25313 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:44.469218 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.427975 25339 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:44.408840 24965 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.383898 25136 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:07:44.435978 25192 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.436977 25326 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:44.436352 25082 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:44.521541 21801 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
W20260709 13:07:44.523085 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83825.5us
I20260709 13:07:44.520714 21792 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:44.525465 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 650) took 1761 ms. Trace:
I20260709 13:07:44.526057 21068 rpcz_store.cc:276] 0709 13:07:42.763778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.764077 (+   299us) service_pool.cc:224] Handling call
0709 13:07:42.784330 (+ 20253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.854711 (+ 70381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:42.854745 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:42.854787 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:42.854817 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.525159 (+1670342us) spinlock_profiling.cc:243] Waited 70.2 ms on lock 0x7b5400019fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:44.525226 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.525388 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161380224}
I20260709 13:07:44.530030 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.534739 21214 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:44.483412 24965 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:44.492986 25192 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:44.484678 25247 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:44.520452 25341 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:44.539050 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.534231 25260 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:44.601466 25260 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.602126 25313 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:44.602397 25268 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.602752 25313 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.603230 25313 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.605232 25313 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:44.611994 25343 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:07:44.613190 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:44.614581 25269 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:44.590142 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.590472 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.593086 25267 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:44.595358 25345 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:07:44.595722 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.599906 25247 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:44.630791 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 106877us
I20260709 13:07:44.639663 25192 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:44.645382 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 118572us
W20260709 13:07:44.655553 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 317929us
I20260709 13:07:44.628052 25260 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.673324 21144 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:44.679689 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.681116 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:44.679000 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:44.682475 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.685169 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:44.686390 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.686455 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.687196 21140 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:44.687390 21216 raft_consensus.cc:2368] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11.
I20260709 13:07:44.690673 21144 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:07:44.689299 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.693331 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:44.693866 21221 raft_consensus.cc:2410] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:07:44.697193 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:44.697526 21143 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:44.698102 21220 raft_consensus.cc:2410] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:07:44.599954 25139 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.697244 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:44.694697 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 327659us
I20260709 13:07:44.700943 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:07:44.705015 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.705369 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.706686 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.706967 21796 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:07:44.707298 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.707674 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:44.707969 21071 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:44.708287 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:44.708462 21140 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:44.707998 25036 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:44.708859 25339 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.709434 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.710481 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.711053 21219 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:44.713330 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:44.713512 24978 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:44.714319 24978 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:44.714529 25036 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:44.714697 24978 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.715024 25036 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:44.715536 25182 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:44.715730 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:44.716095 25182 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:44.715926 25036 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.717011 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.717742 25192 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:44.717059 25036 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:44.717054 25182 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.718755 25082 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.682823 25178 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:44.720593 25082 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.720686 25178 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.722743 25082 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:44.706724 24965 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.725106 24965 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:44.722747 25178 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.697351 25341 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:44.728127 25341 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.731528 25351 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:44.732165 25351 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:44.732663 25351 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:44.697532 25308 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 13:07:44.739809 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.707753 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.701252 25039 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:44.745157 21071 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:44.745491 25178 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:44.697810 24970 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:07:44.749063 25178 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.749799 21215 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:44.749719 25350 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:07:44.750808 25349 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:07:44.748950 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:44.748901 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.708811 21069 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:44.752708 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.754222 21067 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:44.759050 21214 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:07:44.775419 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.697965 25317 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:44.776427 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.731051 25341 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:44.781132 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:44.782456 21063 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:44.715153 24978 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:44.782150 21796 raft_consensus.cc:2376] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11. Re-sending same reply.
I20260709 13:07:44.786337 21214 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:44.794289 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.799628 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.800333 21065 raft_consensus.cc:2376] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13. Re-sending same reply.
W20260709 13:07:44.804813 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 216218us
I20260709 13:07:44.806573 21071 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:44.809527 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:44.812502 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 98349.7us
I20260709 13:07:44.813184 21111 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:44.814880 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:44.802080 25244 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.004s	sys 0.035s
I20260709 13:07:44.817977 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:44.826629 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:44.751808 25039 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.845321 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.826267 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.804813 25339 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:44.850255 21214 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:44.822158 21069 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:44.810827 25178 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:44.821341 25244 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.004s	sys 0.035s
I20260709 13:07:44.846618 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:44.854171 21070 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:07:44.855687 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 140657us
I20260709 13:07:44.826426 24970 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.863556 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.816902 25036 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:44.848865 25317 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:44.826582 25354 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:44.881492 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:44.876953 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.837445 25245 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:07:44.858690 25313 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.868315 25269 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:44.845600 25056 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:07:44.879335 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:44.889936 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:44.887122 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.880426 25354 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.870287 24970 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.914850 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.886844 25357 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.890781 25269 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:44.920236 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.907437 25354 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:44.885257 25036 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.886809 25313 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:44.902726 25312 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:07:44.912179 25336 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:44.942113 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.916355 25361 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:44.950002 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:44.923417 25269 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:44.955376 21221 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
W20260709 13:07:44.956826 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64429.5us
W20260709 13:07:44.959689 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 66383.4us
I20260709 13:07:44.961143 21221 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:07:44.968307 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 90128.2us
I20260709 13:07:44.923177 25357 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.974754 21215 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:44.979437 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.979589 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:44.980062 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.980230 21213 raft_consensus.cc:2368] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11.
I20260709 13:07:44.982348 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:44.989992 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.989956 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:44.993376 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:45.001001 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:45.001700 21212 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:44.962680 25336 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.023871 25339 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.038755 21214 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:44.980706 24851 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.051457 25370 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:44.992411 25308 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:45.058774 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:45.063064 21219 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:45.085563 21214 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:07:45.084098 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 66449.1us
I20260709 13:07:45.056664 25352 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:45.093719 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:45.095100 21219 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:45.100073 21212 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:45.078799 25268 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.052649 25339 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:44.973119 25317 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:45.124969 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:45.062182 24851 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.093657 25369 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.224788 21214 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:45.190232 24970 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:45.232847 25375 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:07:45.237205 25357 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.221500 25352 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.246938 21212 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:45.225788 25036 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.131647 25268 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:45.261260 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 271281us
W20260709 13:07:45.261776 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 299437us
W20260709 13:07:45.249850 25366 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.247s	user 0.000s	sys 0.004s
I20260709 13:07:45.228292 25313 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:45.250080 25084 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.241410 25329 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:45.222342 25369 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.280383 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:45.363152 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.356765 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:45.345331 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.364814 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:45.347265 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.345331 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.315213 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:45.364528 21139 raft_consensus.cc:2410] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:07:45.366360 21068 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:45.318303 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.318370 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:45.310315 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.339000 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:45.314908 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:45.368527 21064 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:45.320039 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:45.319430 21217 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:45.264038 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:45.366274 21145 raft_consensus.cc:2368] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10.
I20260709 13:07:45.365034 21140 raft_consensus.cc:2368] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:07:45.347437 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:45.339000 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.364560 21147 raft_consensus.cc:2376] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13. Re-sending same reply.
I20260709 13:07:45.373379 21143 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 13:07:45.292783 25326 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.760s	user 0.003s	sys 0.010s
I20260709 13:07:45.377841 25326 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:45.380884 25326 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.382128 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:45.405287 21213 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:07:45.383620 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.384485 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:45.406404 21217 raft_consensus.cc:2376] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10. Re-sending same reply.
I20260709 13:07:45.313526 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:45.407434 21071 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:45.277909 21219 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:45.408668 21215 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:07:45.385327 21185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.018s	sys 0.023s
I20260709 13:07:45.331076 25000 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:45.409799 25000 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.389391 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:45.389542 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.390362 21216 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:07:45.362757 25364 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
W20260709 13:07:45.315613 25367 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
W20260709 13:07:45.388023 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 124129us
I20260709 13:07:45.396286 25386 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:45.412300 21216 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:45.412636 21185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.018s	sys 0.023s
I20260709 13:07:45.414014 21146 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:45.471096 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:45.416632 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:45.422291 21071 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:45.362418 25387 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.315884 25339 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.314632 25268 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.362664 25382 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:45.384538 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:45.471680 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 611913us
I20260709 13:07:45.418769 24851 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:45.291425 25336 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.478610 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:45.525245 21071 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:07:45.470952 21216 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.526301 21216 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:45.260176 25352 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:45.487888 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.530382 25394 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.530516 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.530939 25312 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:45.531252 24978 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.719s	user 0.011s	sys 0.000s
I20260709 13:07:45.521135 21221 raft_consensus.cc:2376] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10. Re-sending same reply.
I20260709 13:07:45.486042 25326 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:45.392892 25329 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.537111 21139 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:07:45.538144 21064 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:45.468604 25357 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.393090 25084 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:45.517064 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-8b6dd6862dd14baba4a82c5a837bc802 received from {username='slave'} at 127.0.0.1:54620
W20260709 13:07:45.486295 25366 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:07:45.554296 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.556527 21144 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:45.557771 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.572233 21139 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
W20260709 13:07:45.532372 25145 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.097s	user 0.008s	sys 0.005s
I20260709 13:07:45.568377 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:45.570550 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.537446 25399 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:45.580344 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:45.580560 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:45.581702 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:45.582383 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.586462 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.587486 21216 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:07:45.581225 21796 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:45.587960 21216 pending_rounds.cc:85] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 8
I20260709 13:07:45.588658 21216 pending_rounds.cc:107] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:07:45.589381 21216 raft_consensus.cc:2889] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:45.590639 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.593122 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:45.592988 25023 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:45.593907 25247 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:45.594310 25247 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:45.594676 25247 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:45.595515 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:45.596681 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:45.597522 21068 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:45.597939 21035 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:45.598691 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:45.600111 21071 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:07:45.600685 25392 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:07:45.601310 25383 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:07:45.603427 21145 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:45.603714 21147 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:45.485807 25386 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.612795 21067 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:07:45.627099 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 72882.1us
I20260709 13:07:45.537268 25382 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.632959 21141 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:07:45.492967 25387 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:45.634291 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 43726.6us
I20260709 13:07:45.647008 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:45.649449 21068 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:45.629990 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:45.639542 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:45.599210 25390 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:07:45.650976 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53136us
I20260709 13:07:45.573721 25357 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.652384 25380 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:45.653090 25388 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.654078 25369 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.654160 25387 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.654670 25398 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:45.655031 25379 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:45.556094 24978 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:45.551776 25312 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.650085 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-8b6dd6862dd14baba4a82c5a837bc802 on tablet 8b6dd6862dd14baba4a82c5a837bc802 with peer b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:45.648461 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 62123.3us
I20260709 13:07:45.571573 25018 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:07:45.633603 25326 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:45.655968 24997 consensus_meta.cc:287] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.221s	user 0.008s	sys 0.006s
I20260709 13:07:45.583431 25145 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:45.649714 25266 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:45.548784 25394 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:45.692566 25394 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.609719 24851 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:07:45.669606 25400 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:45.697672 25400 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.698043 25400 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:45.668926 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.672751 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:45.613417 25399 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.674260 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:45.701449 25249 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:45.702262 25399 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.682152 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:45.702965 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:45.703591 21799 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:45.704685 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:45.705971 25354 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.757s	user 0.010s	sys 0.000s
I20260709 13:07:45.687453 25326 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.706431 25354 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:45.707544 21142 raft_consensus.cc:2368] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:45.708285 25359 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:45.666718 25336 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:45.684526 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:45.714747 21796 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:45.687155 21035 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:45.702124 25249 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:45.660023 25380 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.671130 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:45.726397 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55493.3us
W20260709 13:07:45.711815 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49587.5us
I20260709 13:07:45.708848 25359 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.724599 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:45.709404 25269 consensus_meta.cc:287] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.718s	user 0.011s	sys 0.000s
I20260709 13:07:45.728367 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.731822 24818 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:45.728652 25396 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:45.738000 24818 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.738243 25084 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:45.738413 25396 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:45.739111 25396 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:45.739869 25398 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:45.738016 25382 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.738552 24818 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:45.740787 25357 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.665016 25388 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.743469 25407 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:07:45.744434 24978 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.708321 25326 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.754935 25352 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.723083 25399 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:45.759545 25377 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.551s	user 0.000s	sys 0.002s
I20260709 13:07:45.760483 21071 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:45.719126 25383 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:45.730588 25380 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:45.751618 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:45.717968 25387 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:07:45.709630 24997 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.744431 25407 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:45.738907 25084 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:07:45.798092 25247 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:45.764891 25351 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.032s	user 0.010s	sys 0.003s
I20260709 13:07:45.799317 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.799189 25398 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:07:45.796892 25400 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:45.785416 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50248.5us
I20260709 13:07:45.782536 25359 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.747910 25269 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.785225 25336 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:45.838330 21036 leader_election.cc:400] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11.
I20260709 13:07:45.844758 21036 leader_election.cc:403] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:45.810352 25329 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.803459 25407 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:45.801474 25354 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.842067 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:45.846095 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.839030 21140 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:45.854108 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:45.783977 25383 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.855232 25329 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:45.855804 25357 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:45.858404 25357 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.858850 25357 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:45.858911 25404 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:07:45.860538 25408 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:07:45.862782 25388 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.863453 25145 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:45.865718 25365 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.832s	user 0.000s	sys 0.002s
I20260709 13:07:45.867664 25339 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:45.840597 25243 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:45.868283 25339 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:45.871598 25312 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.872005 24818 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:45.872144 25339 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:45.873330 25269 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.874136 24818 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:45.875094 25400 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.877277 25162 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.877657 24997 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:45.878716 25308 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:45.879245 25308 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:45.845245 25336 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.885103 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.885746 25336 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:45.849376 25399 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11.
I20260709 13:07:45.894929 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:45.896238 21036 leader_election.cc:400] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:07:45.897003 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:45.849740 25084 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 13:07:45.900839 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 200132us
I20260709 13:07:45.904147 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:45.905121 25162 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:45.906456 25162 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:45.900295 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.842075 25351 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.912460 21036 leader_election.cc:400] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10.
I20260709 13:07:45.912806 21036 leader_election.cc:403] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:45.927186 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:45.907176 25184 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:45.899039 25400 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:45.898813 25329 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:45.890169 25407 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:45.933151 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:45.936579 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:45.926083 25056 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:45.941642 25162 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:45.929529 25398 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:45.944629 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:45.946390 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.933964 25184 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:45.935400 25249 raft_consensus.cc:2764] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:07:45.949050 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.953986 25361 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:45.930697 24818 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:07:45.951151 25400 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:45.942806 25247 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:45.942272 25387 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10.
I20260709 13:07:45.964697 25249 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:45.954480 25361 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:45.971367 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.971717 25247 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:45.972170 21141 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:45.973091 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:45.949141 25407 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:45.962946 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:45.976243 21141 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:45.961404 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:45.978536 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:45.948173 25056 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.955447 25329 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:45.993582 25387 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:45.994073 25387 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:45.994449 25387 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:45.938582 25396 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:45.998736 25056 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:46.001531 25351 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:46.002388 25336 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.003614 25243 raft_consensus.cc:2764] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:07:46.005384 25336 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.985097 25383 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:45.985839 25184 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.008450 25184 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:45.992471 25357 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.012992 25357 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:45.959625 25084 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:46.005936 25268 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:46.021930 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.022368 21220 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:45.652851 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:46.024574 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:45.976286 21147 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:46.029258 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:46.029439 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.035816 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.036553 21216 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:07:45.998616 25396 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:46.039582 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:46.046645 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:46.047374 21069 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:46.047367 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.049031 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:46.051383 21035 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:46.051723 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.043895 21798 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:46.055621 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:46.057915 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:46.055052 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:46.059648 25396 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:46.060096 25396 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:46.060405 25398 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:46.060535 25396 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:46.060952 25398 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:46.061408 25398 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:46.064075 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.069499 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.070143 21212 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:46.072220 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.072959 21218 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:46.043639 25329 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.076682 25329 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.081713 25386 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:46.082578 25396 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.084803 25396 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:46.084945 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:46.086659 25023 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:46.087109 25023 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:46.087482 25023 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:46.091579 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:46.064090 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:46.097877 21141 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:46.079020 25329 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:46.101393 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:46.105693 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:46.105824 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:46.107178 21143 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:46.109798 21212 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:46.106416 21795 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:46.066545 25363 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:46.129485 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:46.130210 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:46.072530 25380 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.134761 25386 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:46.132722 21767 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:46.136900 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:46.151995 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:46.153245 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:46.110870 25396 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:46.154826 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:46.155752 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 216937us
W20260709 13:07:46.167210 21036 leader_election.cc:336] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.858s (SENT)
W20260709 13:07:46.225824 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:46.231696 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 287106us
I20260709 13:07:46.233018 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:46.325426 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 92529.8us
I20260709 13:07:46.107120 25267 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:07:46.311057 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:46.326784 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:46.328114 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 173328us
I20260709 13:07:46.154393 25383 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:46.108886 25308 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.358659 25268 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:46.404963 25268 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:07:46.406961 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.411819 25399 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.412528 25399 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.361348 21066 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:46.414988 25399 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:46.349360 25328 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:46.344601 25351 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:46.362864 25379 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:46.363874 25427 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:46.364511 21036 leader_election.cc:336] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 2.159s (SENT)
W20260709 13:07:46.365867 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 54902.6us
W20260709 13:07:46.433619 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 69138.9us
I20260709 13:07:46.145780 25363 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.366685 25414 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:46.367405 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.368436 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.371716 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.456854 21221 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:46.371999 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.457736 21213 raft_consensus.cc:2376] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13. Re-sending same reply.
I20260709 13:07:46.372159 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.372431 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.329161 25382 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:46.372694 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.373070 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.237152 25370 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.139639 25381 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.203s
I20260709 13:07:46.373219 25252 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:46.385390 25308 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:46.387324 25380 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:46.388036 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:46.388849 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:46.324606 25182 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:46.399529 25387 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.359115 25036 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:46.157147 25396 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:46.442476 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.466428 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.466553 21065 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:07:46.469455 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.468724 21214 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:46.467453 25414 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:46.481719 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.543946 21220 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:46.544204 21141 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:07:46.417536 25328 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:46.457779 25329 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:46.481269 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.550346 21218 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:46.467201 25379 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.467684 25399 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.539094 25036 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.550666 25328 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:46.551405 25396 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:46.551793 25414 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:46.592280 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:46.563889 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.575758 25329 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.596204 21217 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:46.596635 25400 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:46.598453 25329 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:46.576134 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.599781 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:46.600611 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.601477 21214 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:46.602497 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.583859 25428 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:46.603632 21217 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:46.563669 25268 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:46.603978 25428 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.519152 25182 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.606302 25182 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:46.604707 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.607568 21217 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:46.607611 25428 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:46.573666 25379 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:46.608448 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.609472 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.509092 25370 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.612176 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.571800 25399 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:46.483037 25357 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.620575 25357 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.621837 25399 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:46.621956 21216 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:46.612381 25370 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:46.509262 25363 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:46.617110 25425 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:46.576577 25036 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:46.584089 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:46.623371 25425 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.546519 25267 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:07:46.627696 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.628305 25425 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:46.628973 21220 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:46.630625 21220 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:46.634047 25036 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.601492 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:46.652890 21217 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:46.603145 21214 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:46.467521 25382 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:46.675752 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 74319us
I20260709 13:07:46.677765 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:46.456856 25170 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:07:46.681141 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:46.682240 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:46.580753 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:46.458125 25386 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.208s
I20260709 13:07:46.680123 25329 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:46.688781 21214 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:46.710801 21214 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:07:46.711575 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:46.712203 21141 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:46.683736 25425 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:46.714296 25425 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.705956 21218 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:07:46.708495 21216 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:46.683490 25399 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:46.681336 25382 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:46.683636 25400 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:46.735793 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:46.680562 25266 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:07:46.419008 20958 maintenance_manager.cc:643] P b068e036d3f142f4b255651ac6a0e101: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 8.160s	user 0.187s	sys 0.079s Metrics: {"bytes_written":30942,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":126248,"dirs.run_cpu_time_us":556,"dirs.run_wall_time_us":40549,"drs_written":1,"lbm_read_time_us":529,"lbm_reads_lt_1ms":4,"lbm_write_time_us":193221,"lbm_writes_1-10_ms":3,"lbm_writes_10-100_ms":8,"lbm_writes_lt_1ms":22,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":141520,"threads_started":2,"update_count":25,"wal-append.queue_time_us":101884}
I20260709 13:07:46.727028 25399 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:46.746884 21215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.036s	sys 0.061s
W20260709 13:07:46.748240 21215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.036s	sys 0.061s
I20260709 13:07:46.731024 25400 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.467813 25427 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.489153 25361 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:46.750419 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:46.757501 21219 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:46.773458 25387 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:46.778028 25398 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.000s
I20260709 13:07:46.754822 21141 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
W20260709 13:07:46.754850 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 219156us
I20260709 13:07:46.865769 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:46.749709 25399 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:46.779516 25023 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.669s	user 0.006s	sys 0.005s
I20260709 13:07:46.737483 25329 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:46.899511 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:46.906284 20959 maintenance_manager.cc:419] P b068e036d3f142f4b255651ac6a0e101: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8417 bytes on disk
I20260709 13:07:46.909013 25396 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.796165 25308 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.911916 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2166) took 2151 ms. Trace:
I20260709 13:07:46.917135 25425 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:46.756718 25400 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:46.981071 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:46.778545 25398 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:46.983985 21070 rpcz_store.cc:276] 0709 13:07:44.752782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.752905 (+   123us) service_pool.cc:224] Handling call
0709 13:07:44.853864 (+100959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:44.862070 (+  8206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.903614 (+2041544us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540041d550. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:46.903729 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231566592}
W20260709 13:07:47.056491 21146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2859) took 1738 ms (client timeout 1858 ms). Trace:
W20260709 13:07:47.057291 21146 rpcz_store.cc:269] 0709 13:07:45.318039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.320178 (+  2139us) service_pool.cc:224] Handling call
0709 13:07:47.055948 (+1735770us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b5400258ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:47.056395 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97458560}
I20260709 13:07:47.064941 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:47.063162 25039 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:47.063028 25315 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.066694 25315 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.070178 25315 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.075732 25036 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.078436 25396 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:46.990680 25308 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.086483 25308 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.087745 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.104771 21141 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:47.088694 25387 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.089758 25357 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.092300 25398 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.092963 21069 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:46.926976 25023 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.102128 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 532) took 2181 ms. Trace:
I20260709 13:07:47.110570 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:46.984654 25329 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:47.115837 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 433618us
I20260709 13:07:47.115603 21138 rpcz_store.cc:276] 0709 13:07:44.880334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.880534 (+   200us) service_pool.cc:224] Handling call
0709 13:07:44.880808 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:44.959505 (+ 78697us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:44.968835 (+  9330us) log.cc:844] Serialized 14 byte log entry
0709 13:07:45.054366 (+ 85531us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:45.054411 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:45.054450 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.060870 (+2006420us) spinlock_profiling.cc:243] Waited 78.5 ms on lock 0x7b540032c8d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:47.060954 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.061156 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5311,"prepare.queue_time_us":9618,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2778,"raft.queue_time_us":136692,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":57339,"spinlock_wait_cycles":180507520,"thread_start_us":79800,"threads_started":2,"wal-append.queue_time_us":14964,"wal-append.run_cpu_time_us":7775,"wal-append.run_wall_time_us":1265010}
I20260709 13:07:47.117414 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:47.111932 25328 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.000s
I20260709 13:07:47.119302 21146 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:47.125530 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2872) took 1267 ms. Trace:
I20260709 13:07:47.126012 21146 rpcz_store.cc:276] 0709 13:07:45.857738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.103042 (+1245304us) service_pool.cc:224] Handling call
0709 13:07:47.125436 (+ 22394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:47.119226 20958 maintenance_manager.cc:643] P b068e036d3f142f4b255651ac6a0e101: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.056s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":442,"lbm_reads_lt_1ms":4}
I20260709 13:07:47.129670 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:47.152570 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.155462 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.101040 25396 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.165498 21069 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:47.163643 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:47.168002 21138 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:47.157024 21066 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.027s	sys 0.065s
W20260709 13:07:47.168944 21066 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.027s	sys 0.065s
I20260709 13:07:47.168835 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:47.098614 25414 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.177335 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.179122 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.179915 21146 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:47.157300 25398 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:47.117235 25387 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:47.180239 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 502412us
I20260709 13:07:47.185642 21146 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:47.183895 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:47.173645 25268 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:47.192414 21219 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:47.192574 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 539) took 1582 ms. Trace:
I20260709 13:07:47.155005 25328 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:47.190693 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.193015 21146 rpcz_store.cc:276] 0709 13:07:45.604166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.185160 (+1580994us) service_pool.cc:224] Handling call
0709 13:07:47.185473 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.186275 (+   802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.186442 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:47.192584 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.195242 21138 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:47.164391 25396 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.169834 25308 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:07:47.212767 21138 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:47.215715 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2411) took 1134 ms. Trace:
I20260709 13:07:47.216099 21138 rpcz_store.cc:276] 0709 13:07:46.080920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.212206 (+1131286us) service_pool.cc:224] Handling call
0709 13:07:47.212454 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.215472 (+  3018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.215661 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:47.221254 21796 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:47.225337 25380 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:47.225944 25380 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:47.231592 25371 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:47.232194 25371 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.232679 25328 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:47.234215 25371 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.234346 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:47.234937 21070 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:47.236466 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1346) took 1216 ms. Trace:
I20260709 13:07:47.236735 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.240091 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 546) took 1096 ms. Trace:
I20260709 13:07:47.241346 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.244910 21213 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:47.244529 21138 rpcz_store.cc:276] 0709 13:07:46.143642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.216803 (+1073161us) service_pool.cc:224] Handling call
0709 13:07:47.239617 (+ 22814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.239773 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:47.239801 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.239839 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:47.239869 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.239931 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.240036 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:47.242503 21141 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:47.247541 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 542) took 1105 ms. Trace:
I20260709 13:07:47.236904 21215 rpcz_store.cc:276] 0709 13:07:46.019712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.044895 (+ 25183us) service_pool.cc:224] Handling call
0709 13:07:46.045242 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.045692 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.045734 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.046319 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.750686 (+704367us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:46.750728 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.750803 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.236252 (+485449us) raft_consensus.cc:1685] finished
0709 13:07:47.236272 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.236412 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":880,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":12046,"thread_start_us":654081,"threads_started":1,"wal-append.queue_time_us":1168248}
I20260709 13:07:47.250461 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.252178 21215 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:47.248040 21141 rpcz_store.cc:276] 0709 13:07:46.142592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.214258 (+1071666us) service_pool.cc:224] Handling call
0709 13:07:47.242278 (+ 28020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.247149 (+  4871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.247465 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:47.186779 25395 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:07:47.261484 21141 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:47.259711 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:47.264029 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.266680 21215 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:47.266923 21138 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:47.267647 21215 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:47.267863 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 549) took 1123 ms. Trace:
I20260709 13:07:47.268394 21138 rpcz_store.cc:276] 0709 13:07:46.144377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.246054 (+1101677us) service_pool.cc:224] Handling call
0709 13:07:47.266668 (+ 20614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.267541 (+   873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.267795 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:47.219075 25398 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:47.256541 21070 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:47.271729 21141 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:07:47.272061 21767 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:47.273727 25000 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.236554 25387 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.280602 21213 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:47.281426 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:47.284471 21070 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:47.257510 25170 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.286731 21213 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:47.290726 21138 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:47.289050 21213 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.293792 21213 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:47.237262 25433 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:07:47.283743 25328 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:07:47.310976 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 181376us
I20260709 13:07:47.209012 25329 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.312016 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:47.313287 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 130908us
I20260709 13:07:47.288344 25398 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:47.321790 21138 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:47.323570 21138 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:47.325413 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:47.303011 25126 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:47.330868 25126 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:47.227171 25380 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.291383 25427 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:47.387362 21138 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:07:47.294277 25170 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.331116 25374 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:47.332628 25328 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:47.335006 25387 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.333997 25023 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.335393 25437 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:47.335986 25000 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:47.394239 25182 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.787s	user 0.010s	sys 0.003s
W20260709 13:07:47.394047 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 142477us
I20260709 13:07:47.395589 25380 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:47.416970 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.417934 21138 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:47.418839 21219 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:47.419684 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:47.420516 21215 raft_consensus.cc:2368] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:47.394115 25374 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:47.411994 25126 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.435353 25374 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:47.422736 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.422611 25441 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:47.424759 25170 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:47.428246 25382 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.000s
I20260709 13:07:47.427740 25328 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.412366 25000 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.431679 25396 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.462703 25396 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.409343 25437 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:47.463548 25437 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:47.464051 25000 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.413262 25414 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.411521 25387 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.469743 25387 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.434108 25400 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.411242 25438 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.471707 25438 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.474486 25438 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.444834 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:47.477828 21036 leader_election.cc:400] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:47.478194 21036 leader_election.cc:403] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:07:47.451090 21213 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:07:47.479485 21213 pending_rounds.cc:85] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 5
I20260709 13:07:47.479823 21213 pending_rounds.cc:107] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:07:47.480129 21213 raft_consensus.cc:2889] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:47.461316 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.486188 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.486864 21215 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:47.488149 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.489189 21219 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:47.493124 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.416476 25182 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.503860 25182 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.504577 21144 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:47.505856 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.507474 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2869) took 1913 ms. Trace:
I20260709 13:07:47.508325 25414 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:07:47.463204 25396 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.512295 25396 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:47.513049 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.515254 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 538) took 1911 ms. Trace:
I20260709 13:07:47.507846 21144 rpcz_store.cc:276] 0709 13:07:45.594295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.604232 (+  9937us) service_pool.cc:224] Handling call
0709 13:07:47.056094 (+1451862us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.505013 (+448919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.506358 (+  1345us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b5400258ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:47.507243 (+   885us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400258ad0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:47.507407 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4369293056}
I20260709 13:07:47.515810 21140 rpcz_store.cc:276] 0709 13:07:45.603893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.752979 (+149086us) service_pool.cc:224] Handling call
0709 13:07:45.764661 (+ 11682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.852199 (+ 87538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.515011 (+1662812us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b54002b43d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:47.515194 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132699520}
I20260709 13:07:47.517002 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.517299 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.517913 21144 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:47.521371 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:47.521952 21144 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:47.522388 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.522765 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.523777 25387 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:47.527215 25387 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:47.527743 25387 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:47.528069 21142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2863) took 2182 ms (client timeout 2159 ms). Trace:
W20260709 13:07:47.528512 21142 rpcz_store.cc:269] 0709 13:07:45.345612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.381513 (+ 35901us) service_pool.cc:224] Handling call
0709 13:07:47.527808 (+2146295us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54004236d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:47.527972 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679412096}
I20260709 13:07:47.529639 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.531172 21145 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:47.533089 21145 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:47.531476 21139 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:47.530726 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:47.537151 21142 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:47.417356 25439 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:07:47.520134 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:47.541213 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 47463.5us
I20260709 13:07:47.545065 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.523623 25438 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9.
I20260709 13:07:47.545857 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.484727 25382 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:47.541613 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 154094us
I20260709 13:07:47.548548 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.554833 21145 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:47.554941 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:47.554150 25000 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:47.558426 25395 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:47.559036 25395 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:47.559507 25395 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:47.542908 25396 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:47.558571 25000 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.485253 25398 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.564582 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.566493 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:47.563359 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:47.569078 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:47.570257 21147 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:47.570307 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 540) took 1912 ms. Trace:
I20260709 13:07:47.575029 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:47.571090 25036 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:47.563582 25396 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.581841 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.584295 21219 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:47.586261 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:47.604722 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:47.605648 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:47.591989 25447 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:47.590101 25357 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.593812 25382 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.579918 25308 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:47.590606 25400 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.571373 25438 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.602370 21146 rpcz_store.cc:276] 0709 13:07:45.658284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.211519 (+1553235us) service_pool.cc:224] Handling call
0709 13:07:47.211820 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.212028 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:47.237970 (+ 25942us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.563231 (+325261us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.569478 (+  6247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.569541 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:47.569901 (+   360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.570072 (+   171us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.570232 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25188,"prepare.queue_time_us":47916,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":46786,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":3843,"spinlock_wait_cycles":749440,"thread_start_us":320661,"threads_started":2,"wal-append.queue_time_us":174082}
I20260709 13:07:47.598567 25437 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.613116 25438 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.614359 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2417) took 1147 ms. Trace:
I20260709 13:07:47.614915 25437 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.615697 25400 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:47.615837 25447 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:47.614775 21141 rpcz_store.cc:276] 0709 13:07:46.466907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.272498 (+805591us) service_pool.cc:224] Handling call
0709 13:07:47.281533 (+  9035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:47.282030 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:47.282074 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:47.283095 (+  1021us) log.cc:844] Serialized 26 byte log entry
0709 13:07:47.459079 (+175984us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:47.459117 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.459185 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.614129 (+154944us) raft_consensus.cc:1685] finished
0709 13:07:47.614151 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.614301 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9055,"raft.queue_time_us":620,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4094,"thread_start_us":147184,"threads_started":1,"wal-append.queue_time_us":248728}
I20260709 13:07:47.616268 25400 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.618603 25400 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.609807 25357 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.624989 25357 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:47.625691 25449 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.626223 25449 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.627583 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:47.578753 25182 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:47.577534 25329 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.632613 25449 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.633035 25387 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:47.616354 25447 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:47.610551 25396 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:47.637938 21144 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:07:47.639797 25395 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.635656 25182 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:47.652019 25382 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:47.652629 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 546) took 1077 ms. Trace:
I20260709 13:07:47.668833 21218 rpcz_store.cc:276] 0709 13:07:46.574972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.716989 (+142017us) service_pool.cc:224] Handling call
0709 13:07:46.717355 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.729422 (+ 12067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.729471 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:46.729530 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:46.729584 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.652391 (+922807us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b540013b854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:47.652434 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.652573 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27076992}
I20260709 13:07:47.653051 21220 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:47.671691 25421 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:07:47.653264 21216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1362) took 1267 ms (client timeout 1675 ms). Trace:
W20260709 13:07:47.655211 25399 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.000s
I20260709 13:07:47.672911 25399 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:47.672727 21216 rpcz_store.cc:269] 0709 13:07:46.385668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.469118 (+ 83450us) service_pool.cc:224] Handling call
0709 13:07:47.653076 (+1183958us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b540034cbd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:47.653186 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182030592}
I20260709 13:07:47.656786 21217 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:07:47.636071 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:47.850489 25454 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.921085 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:48.041108 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 405143us
I20260709 13:07:47.924938 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:48.041880 21072 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:48.042200 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:48.042320 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:47.921483 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:47.934808 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:48.042723 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:47.888419 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:47.808054 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.806962 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:47.709309 21215 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:48.236790 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:48.237411 21215 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:47.711762 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:48.248440 21220 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:48.249024 21215 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:47.982373 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:48.254024 21212 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
W20260709 13:07:48.254585 21212 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2680) took 1886 ms (client timeout 1717 ms). Trace:
I20260709 13:07:47.723752 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:48.256116 21219 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:47.813100 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:47.703644 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:48.260867 21216 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 4.
I20260709 13:07:47.701987 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:47.806087 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:47.848534 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.265595 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:47.752198 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.194428 25469 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.498481 25469 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:48.196779 25468 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:48.217008 25471 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:47.980278 20968 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 165) took 15127 ms (client timeout 15000 ms). Trace:
I20260709 13:07:48.502182 25469 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:48.087239 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.701s (SENT) [suppressed 29 similar messages]
I20260709 13:07:48.047451 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:48.628484 21063 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:48.228127 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:48.632920 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:48.633622 21066 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:48.634503 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:48.043308 21068 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:48.637161 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:48.637756 21068 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 4.
I20260709 13:07:48.638600 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:48.120886 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 669) took 2543 ms. Trace:
I20260709 13:07:48.641003 21071 rpcz_store.cc:276] 0709 13:07:45.577530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.600907 (+ 23377us) service_pool.cc:224] Handling call
0709 13:07:45.601162 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.768500 (+167338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.120473 (+2351973us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540001b3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:48.120768 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365530368}
I20260709 13:07:48.642467 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:47.925637 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:48.644109 21070 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:48.200677 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:48.649221 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:48.241102 21069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 682) took 1853 ms (client timeout 1757 ms). Trace:
W20260709 13:07:48.651042 21069 rpcz_store.cc:269] 0709 13:07:46.387599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.387753 (+   154us) service_pool.cc:224] Handling call
0709 13:07:48.240924 (+1853171us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b5400214654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:48.241037 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333204224}
I20260709 13:07:48.652504 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:48.043426 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 108691us
I20260709 13:07:48.046809 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:07:48.661625 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 614850us
I20260709 13:07:48.186194 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:48.092129 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d78b35ac92d402db8a5e3fe45be52b5 with delete_type TABLET_DATA_TOMBSTONED (TS 5e5e65640c8b4f5ca7eadc6a14312c55 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:46800
I20260709 13:07:48.196446 21138 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:48.199079 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2870) took 2604 ms. Trace:
I20260709 13:07:48.672531 21139 rpcz_store.cc:276] 0709 13:07:45.594703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.598206 (+  3503us) service_pool.cc:224] Handling call
0709 13:07:47.531224 (+1933018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.536086 (+  4862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.198879 (+662793us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54004236d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:48.199022 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4444795648}
I20260709 13:07:48.187824 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:48.203925 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2410) took 2601 ms. Trace:
I20260709 13:07:48.679730 21143 rpcz_store.cc:276] 0709 13:07:45.602509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.602653 (+   144us) service_pool.cc:224] Handling call
0709 13:07:45.602931 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.112293 (+509362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.203646 (+2091353us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b5400279554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:48.203858 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159229184}
W20260709 13:07:48.193223 21147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 547) took 2049 ms (client timeout 2190 ms). Trace:
W20260709 13:07:48.693902 21147 rpcz_store.cc:269] 0709 13:07:46.143884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.116344 (+972460us) service_pool.cc:224] Handling call
0709 13:07:48.192936 (+1076592us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540030af54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:48.193143 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941714304}
W20260709 13:07:48.166954 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:48.697770 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:48.095530 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.202157 21214 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:48.151834 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 539) took 1579 ms. Trace:
I20260709 13:07:48.217015 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:48.262326 21218 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:48.254956 21212 rpcz_store.cc:269] 0709 13:07:46.367738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.371373 (+  3635us) service_pool.cc:224] Handling call
0709 13:07:47.982241 (+1610868us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:07:48.254513 (+272272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4233728}
I20260709 13:07:48.319921 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:48.712857 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 393022us
I20260709 13:07:48.715408 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:48.343950 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:48.717157 21144 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:48.718194 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:48.718412 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:48.719488 25056 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:48.272749 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:48.270690 21794 raft_consensus.cc:1217] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEARNER]: Deduplicated request from leader. Original: 1.2->[5.3-8.5]   Dedup: 8.5->[]
I20260709 13:07:48.725189 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:48.269809 21798 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:48.727137 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:48.728266 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:48.729291 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:48.282019 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:48.732197 21799 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:48.733158 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:48.281181 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:48.735100 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:48.736446 21800 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:48.273490 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:48.739990 21801 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:07:48.740319 21801 pending_rounds.cc:85] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 8
I20260709 13:07:48.740553 21801 pending_rounds.cc:107] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:07:48.740774 21801 raft_consensus.cc:2889] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:47.905814 25000 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:48.744855 25000 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:48.745393 25000 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.955621 25136 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:48.749567 25136 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:48.291008 25184 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:48.752492 25184 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:48.754858 25184 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:47.611564 25308 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:48.757025 25308 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:48.160880 25313 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:48.759989 25313 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:48.760991 25313 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:48.762033 25313 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:47.707525 25350 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:48.764147 25350 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:48.766389 25350 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:48.130239 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 843725us
I20260709 13:07:47.759145 25126 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:48.117686 25178 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:48.776690 25178 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:48.777774 25178 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.924080 25243 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:48.780208 25243 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.800246 25268 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:47.656658 25329 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:48.122776 24965 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:07:48.271910 21826 heartbeater.cc:646] Failed to heartbeat to 127.19.127.254:37743 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.127.254:37743 timed out after 15.000s (SENT)
I20260709 13:07:47.801432 25036 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:47.667516 25182 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:48.343451 25266 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:48.064240 25327 raft_consensus.cc:2764] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:48.223331 25335 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:47.706851 25357 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:48.291846 25359 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:48.287181 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:47.682410 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 481) took 1285 ms. Trace:
I20260709 13:07:48.272480 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:47.706674 25374 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:47.798058 25363 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:47.935730 25382 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:47.942287 25361 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.715128 25370 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:47.696192 25387 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.737603 25396 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.676116 25399 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:47.926106 25400 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.706415 25414 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.675716 25395 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:47.897061 25398 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.291605 25416 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:47.672305 25421 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:47.749509 25433 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:48.208264 25437 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:47.665606 25438 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:48.283136 25422 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:47.666996 25447 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:47.926268 25449 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:48.040835 25454 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:48.500368 25468 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:48.500523 25471 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:48.503676 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:48.502445 20968 rpcz_store.cc:269] 0709 13:07:32.852806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.855481 (+  2675us) service_pool.cc:224] Handling call
0709 13:07:47.980157 (+15124676us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:42.821984 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:42.822645 (+   661us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:42.822695 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:07:42.822719 (+    24us) tablet.cc:662] Decoding operations
0709 13:07:42.903962 (+ 81243us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:42.904068 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:42.904113 (+    45us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:07:42.985622 (+ 81509us) tablet.cc:701] Row locks acquired
0709 13:07:42.993547 (+  7925us) write_op.cc:260] PREPARE: finished
0709 13:07:43.062686 (+ 69139us) write_op.cc:270] Start()
0709 13:07:43.084964 (+ 22278us) write_op.cc:276] Timestamp: P: 1783602463041779 usec, L: 0
0709 13:07:43.085011 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:07:43.100019 (+ 15008us) log.cc:844] Serialized 7484 byte log entry
0709 13:07:43.296779 (+196760us) op_driver.cc:464] REPLICATION: finished
0709 13:07:43.904893 (+608114us) write_op.cc:301] APPLY: starting
0709 13:07:43.914380 (+  9487us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:43.933954 (+ 19574us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:43.944470 (+ 10516us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:43.972862 (+ 28392us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:43.973229 (+   367us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=38
0709 13:07:43.973273 (+    44us) write_op.cc:312] APPLY: finished
0709 13:07:43.973708 (+   435us) log.cc:844] Serialized 132 byte log entry
0709 13:07:43.974874 (+  1166us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:43.975096 (+   222us) write_op.cc:454] Released schema lock
0709 13:07:43.975190 (+    94us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:43.975724 (+   534us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:43.975755 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":856769,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":516001,"apply.run_cpu_time_us":6500,"apply.run_wall_time_us":130036,"num_ops":19,"prepare.queue_time_us":114342,"prepare.run_cpu_time_us":20814,"prepare.run_wall_time_us":306010,"raft.queue_time_us":97633,"raft.run_cpu_time_us":6658,"raft.run_wall_time_us":347464,"replication_time_us":188214,"thread_start_us":451021,"threads_started":4,"wal-append.queue_time_us":28289,"wal-append.run_cpu_time_us":18346,"wal-append.run_wall_time_us":1848783}]]}
I20260709 13:07:48.569862 21220 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:48.570732 25472 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:48.571085 25469 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.584833 25354 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.657084 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:48.668350 25487 tablet_replica.cc:333] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:07:48.684444 25448 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:48.701897 21217 rpcz_store.cc:276] 0709 13:07:46.572897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.673677 (+100780us) service_pool.cc:224] Handling call
0709 13:07:47.656558 (+982881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:47.674293 (+ 17735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.151615 (+477322us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b540034cbd0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:48.151768 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2259435776}
I20260709 13:07:48.704990 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:48.716061 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:48.722563 21792 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:48.723165 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:48.737845 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:48.750528 25136 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:49.055792 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 350844us
I20260709 13:07:49.056707 21141 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:48.889559 21063 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:48.635106 21066 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
W20260709 13:07:49.058182 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 272281us
I20260709 13:07:49.058282 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:48.811510 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:48.875149 21218 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:49.059460 21215 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:07:48.948078 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 860819us
I20260709 13:07:49.058337 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2193) took 1121 ms. Trace:
W20260709 13:07:49.057906 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.061542 21066 rpcz_store.cc:276] 0709 13:07:47.936944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.634244 (+697300us) service_pool.cc:224] Handling call
0709 13:07:49.058280 (+424036us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.062749 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.059515 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2707) took 1354 ms. Trace:
I20260709 13:07:48.784170 21072 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:49.057955 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2186) took 1123 ms. Trace:
I20260709 13:07:48.788205 21799 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:49.064435 21063 rpcz_store.cc:276] 0709 13:07:47.934536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.047181 (+112645us) service_pool.cc:224] Handling call
0709 13:07:49.057848 (+1010667us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:48.798811 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:49.056370 21796 rpcz_store.cc:276] 0709 13:07:46.394571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.394738 (+   167us) service_pool.cc:224] Handling call
0709 13:07:46.395107 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.237638 (+842531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.679519 (+441881us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b540046aad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:47.679793 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1899569024}
W20260709 13:07:49.065604 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 375595us
I20260709 13:07:49.063786 21220 rpcz_store.cc:276] 0709 13:07:47.704851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.705023 (+   172us) service_pool.cc:224] Handling call
0709 13:07:49.059200 (+1354177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:48.671206 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2425) took 2057 ms. Trace:
W20260709 13:07:49.059983 21218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2705) took 1358 ms (client timeout 1646 ms). Trace:
I20260709 13:07:48.832952 21070 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:07:49.066988 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:49.067531 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1315) took 1147 ms. Trace:
I20260709 13:07:49.068300 21063 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 8: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:49.068199 21070 rpcz_store.cc:276] 0709 13:07:47.920008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.925412 (+  5404us) service_pool.cc:224] Handling call
0709 13:07:49.067474 (+1142062us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.066525 21138 rpcz_store.cc:276] 0709 13:07:46.614292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.459783 (+845491us) service_pool.cc:224] Handling call
0709 13:07:48.196261 (+736478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:48.669466 (+473205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.670987 (+  1521us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b540030af50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:48.671143 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1692216192}
I20260709 13:07:49.068871 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1322) took 1147 ms. Trace:
I20260709 13:07:48.806315 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:49.069284 21063 rpcz_store.cc:276] 0709 13:07:47.921936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.065577 (+1143641us) service_pool.cc:224] Handling call
0709 13:07:49.068817 (+  3240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.070003 21071 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:49.019094 21168 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2f20 after lost signal to thread 19970
W20260709 13:07:49.073536 21168 debug-util.cc:398] Leaking SignalData structure 0x7b08001bd2c0 after lost signal to thread 21004
W20260709 13:07:49.074087 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.074884 21168 debug-util.cc:398] Leaking SignalData structure 0x7b0800331e20 after lost signal to thread 21096
W20260709 13:07:49.066829 21218 rpcz_store.cc:269] 0709 13:07:47.701438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.701601 (+   163us) service_pool.cc:224] Handling call
0709 13:07:49.059930 (+1358329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.075341 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 554) took 1268 ms. Trace:
I20260709 13:07:49.075773 21215 rpcz_store.cc:276] 0709 13:07:47.806606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.249934 (+443328us) service_pool.cc:224] Handling call
0709 13:07:49.059208 (+809274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:49.059989 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.075126 (+ 15137us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54000bc2d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:49.075270 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860383616}
I20260709 13:07:49.082054 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2198) took 1142 ms. Trace:
I20260709 13:07:49.082468 21072 rpcz_store.cc:276] 0709 13:07:47.939604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.042491 (+102887us) service_pool.cc:224] Handling call
0709 13:07:49.081841 (+1039350us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54000dc354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:49.081995 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703456128}
W20260709 13:07:49.084659 21168 debug-util.cc:398] Leaking SignalData structure 0x7b0800362420 after lost signal to thread 21171
W20260709 13:07:49.084985 21168 debug-util.cc:398] Leaking SignalData structure 0x7b08003f6d40 after lost signal to thread 21172
W20260709 13:07:49.085986 21168 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6520 after lost signal to thread 21245
W20260709 13:07:49.087018 21168 debug-util.cc:398] Leaking SignalData structure 0x7b080020a240 after lost signal to thread 21825
I20260709 13:07:49.087659 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:49.088199 21070 raft_consensus.cc:2376] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11. Re-sending same reply.
W20260709 13:07:49.089391 21168 debug-util.cc:398] Leaking SignalData structure 0x7b0800366160 after lost signal to thread 25365
I20260709 13:07:49.072924 25350 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:49.091789 25399 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:49.092355 25399 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.063679 25268 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:49.095045 25454 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.063818 25422 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.073591 25354 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.097370 25422 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:49.097959 25422 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:49.054879 25243 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.098273 25354 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.099318 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1316) took 1179 ms. Trace:
I20260709 13:07:49.072973 25437 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.100188 21069 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:49.099697 21064 rpcz_store.cc:276] 0709 13:07:47.920287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.131577 (+211290us) service_pool.cc:224] Handling call
0709 13:07:49.081998 (+950421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.082195 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.082220 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.082261 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:49.082297 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.082347 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.099111 (+ 16764us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b54000dc350. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:49.099224 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2184742656}
I20260709 13:07:49.100673 25471 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.101243 25437 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.073462 25414 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.073722 25400 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.102559 21146 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:49.103618 25414 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:49.103900 25400 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.056591 25370 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:49.105463 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1384) took 1170 ms. Trace:
I20260709 13:07:49.105265 25370 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:49.106208 25370 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:49.105836 21216 rpcz_store.cc:276] 0709 13:07:47.935165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.505337 (+570172us) service_pool.cc:224] Handling call
0709 13:07:48.505645 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.103706 (+598061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.103755 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.103803 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:49.103840 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.105228 (+  1388us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b540015ca54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:49.105278 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.105404 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1371394432}
I20260709 13:07:49.101318 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.111802 21064 raft_consensus.cc:2410] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:07:49.112564 21066 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:49.104146 25414 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:49.119782 21792 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:49.078322 25315 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:48.979807 20967 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:49.134105 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.142674 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2181) took 1209 ms. Trace:
I20260709 13:07:49.135531 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:49.143038 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65048.1us
I20260709 13:07:49.136653 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.144510 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.143064 21064 rpcz_store.cc:276] 0709 13:07:47.933298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.101060 (+1167762us) service_pool.cc:224] Handling call
0709 13:07:49.142605 (+ 41545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.145385 21069 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:49.143085 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2174) took 1220 ms. Trace:
I20260709 13:07:49.145591 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:49.145913 21063 rpcz_store.cc:276] 0709 13:07:47.922996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.070147 (+1147151us) service_pool.cc:224] Handling call
0709 13:07:49.143027 (+ 72880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.148056 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2192) took 1211 ms. Trace:
I20260709 13:07:49.148634 21066 rpcz_store.cc:276] 0709 13:07:47.936202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.062428 (+1126226us) service_pool.cc:224] Handling call
0709 13:07:49.147855 (+ 85427us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b5400214154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:49.147985 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112603008}
W20260709 13:07:49.152590 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.064606 25036 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:49.076067 25335 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:48.970144 25488 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:49.159209 25488 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.161360 25488 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.161448 25468 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.076210 25357 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.163151 25396 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.163972 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1323) took 1241 ms. Trace:
I20260709 13:07:49.073048 25438 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:49.164376 21069 rpcz_store.cc:276] 0709 13:07:47.922243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.652221 (+729978us) service_pool.cc:224] Handling call
0709 13:07:49.163790 (+511569us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54003ca354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:49.163916 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1028161408}
I20260709 13:07:49.165799 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:49.165892 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.166450 21072 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:49.166801 21069 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:07:49.075672 25449 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.168118 21068 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:49.063179 25448 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.177146 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.055229 25433 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:49.178373 21071 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:49.178942 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2189) took 1243 ms. Trace:
I20260709 13:07:49.179466 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.179327 21071 rpcz_store.cc:276] 0709 13:07:47.935409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.805945 (+870536us) service_pool.cc:224] Handling call
0709 13:07:49.178884 (+372939us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.163758 25357 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:49.187785 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.191258 21070 raft_consensus.cc:2368] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11.
I20260709 13:07:49.191802 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2180) took 1259 ms. Trace:
I20260709 13:07:49.192165 21070 rpcz_store.cc:276] 0709 13:07:47.932445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.187435 (+1254990us) service_pool.cc:224] Handling call
0709 13:07:49.191728 (+  4293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.163590 25000 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:48.975497 25126 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.201s
I20260709 13:07:49.207551 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:49.203465 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.098294 25422 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:49.134549 24965 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:49.076383 25266 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.180512 25448 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:49.201756 25136 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:49.208981 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.228567 21066 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:49.228664 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.229455 21071 raft_consensus.cc:2368] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11.
I20260709 13:07:49.202240 21798 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
W20260709 13:07:49.211714 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 138264us
W20260709 13:07:49.233577 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 495789us
I20260709 13:07:49.214241 25492 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:49.214461 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 96919.3us
I20260709 13:07:49.208709 21063 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:49.073812 25469 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.075906 25472 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.226694 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:49.215216 25422 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:49.230814 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2190) took 1295 ms. Trace:
I20260709 13:07:49.255251 21068 rpcz_store.cc:276] 0709 13:07:47.935706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.638326 (+702620us) service_pool.cc:224] Handling call
0709 13:07:49.230575 (+592249us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b540011c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:49.230745 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116549888}
I20260709 13:07:49.240005 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:49.199364 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 542484us
I20260709 13:07:49.223698 25315 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:49.229660 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:49.213063 25494 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.223922 20967 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 6 to 9, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 9 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:49.266661 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.267318 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.266589 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:49.243186 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.237857 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:49.248968 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.249243 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:49.266813 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.272993 21063 raft_consensus.cc:1813] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Handling vote request from an unknown peer 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.273579 21063 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:49.240922 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.250751 21142 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:49.236438 25184 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:49.280205 21067 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:49.285404 21068 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:49.286010 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:49.287626 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.251732 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 174247us
I20260709 13:07:49.251871 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.227438 21072 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:49.253449 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.076227 25398 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.189997 25438 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.259239 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:49.296921 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.261262 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1324) took 1338 ms. Trace:
I20260709 13:07:49.208541 25357 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:07:49.258141 25399 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.289687 21140 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:49.291661 21071 raft_consensus.cc:2410] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:07:49.291486 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:49.234411 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:49.294718 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.294260 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2173) took 1371 ms. Trace:
I20260709 13:07:49.254544 25350 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:49.294996 21035 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:49.293144 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 149108us
I20260709 13:07:49.332823 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2179) took 1400 ms. Trace:
I20260709 13:07:49.333240 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:49.307240 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.298055 21069 rpcz_store.cc:276] 0709 13:07:47.922506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.260240 (+1337734us) service_pool.cc:224] Handling call
0709 13:07:49.260723 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.260918 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.260950 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.260991 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:49.261023 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.261079 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.261198 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.293228 21063 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:49.310464 20967 catalog_manager.cc:5697] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 3 to 6, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:49.333232 21071 rpcz_store.cc:276] 0709 13:07:47.932165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.229996 (+1297831us) service_pool.cc:224] Handling call
0709 13:07:49.332762 (+102766us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.335038 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2176) took 1408 ms. Trace:
I20260709 13:07:49.330116 21068 rpcz_store.cc:276] 0709 13:07:47.922743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.261596 (+1338853us) service_pool.cc:224] Handling call
0709 13:07:49.285226 (+ 23630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.294033 (+  8807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.294209 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.336071 21063 rpcz_store.cc:276] 0709 13:07:47.927125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.274576 (+1347451us) service_pool.cc:224] Handling call
0709 13:07:49.293064 (+ 18488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.334782 (+ 41718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.334982 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.303268 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.366901 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.294253 25396 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.288854 25312 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:49.368466 21064 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:49.387738 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.288595 25184 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:49.370568 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.295791 24978 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:49.380110 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.289356 25495 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:49.286696 25503 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.306769 25468 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.310124 25489 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.216796 25421 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.337232 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.386762 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.254041 25308 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.314059 25315 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.254252 25492 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.389578 25496 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:49.453433 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.454542 21147 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:07:49.394435 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 160099us
I20260709 13:07:49.455936 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.387193 25400 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:49.392227 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.462941 21069 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:49.395666 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.393102 20967 catalog_manager.cc:5697] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 7 to 10, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 10 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:49.397068 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.398311 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.399817 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.340338 25178 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:49.401432 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.402474 21072 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:49.466594 21143 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:49.467658 21072 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:07:49.468349 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2185) took 1534 ms. Trace:
I20260709 13:07:49.468706 21072 rpcz_store.cc:276] 0709 13:07:47.934306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.467154 (+1532848us) service_pool.cc:224] Handling call
0709 13:07:49.467468 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:49.468028 (+   560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.468291 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.403472 25056 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:49.418488 21215 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:49.371635 25387 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:49.419474 21071 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 7.
I20260709 13:07:49.417286 25395 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.407344 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.419890 21063 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:49.288822 25494 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.294888 25000 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:07:49.254597 25433 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.341058 25442 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.406546 25437 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:07:49.421566 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:49.499162 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 211050us
I20260709 13:07:49.420493 25501 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:07:49.449172 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.450376 25308 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.456718 21212 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:49.502199 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:49.502854 21219 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:49.472425 25178 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:49.508920 25178 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:49.510104 25503 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:49.457960 21221 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:49.462461 25036 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.394397 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.511870 25469 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.487535 25492 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:49.512671 25492 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:49.487890 25438 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.513036 25492 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:49.513949 25492 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.516732 25472 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.485363 25184 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.513036 25036 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.470293 25182 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:49.460253 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.471426 20967 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 6 to 9, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 9 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:49.512590 21139 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:49.381395 25335 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.473029 21072 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:49.398936 25471 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:49.475075 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:49.477288 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.487229 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2191) took 1551 ms. Trace:
I20260709 13:07:49.533754 21063 rpcz_store.cc:276] 0709 13:07:47.935975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.486490 (+1550515us) service_pool.cc:224] Handling call
0709 13:07:49.486779 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.486936 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.486959 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.487000 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:49.487035 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.487086 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.487178 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.535327 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:49.310238 25488 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:07:49.468062 25329 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.487915 21141 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
W20260709 13:07:49.556891 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.488890 25421 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:49.331046 24965 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.420867 25312 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:49.390120 25396 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.417914 25416 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:49.421917 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.485159 25387 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:49.417798 25489 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:49.496284 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800212180 after lost signal to thread 19970
I20260709 13:07:49.496474 25371 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:49.496702 25508 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:49.496963 25433 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:49.497354 25442 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.496740 21071 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:49.417526 24978 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:49.460222 25496 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:49.405889 25468 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.505582 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.486006 25500 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.508466 25506 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:49.459007 21214 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:49.632136 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.632755 21214 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:07:49.633533 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:49.394654 25502 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:49.474090 25512 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.423302 21068 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 8. Current term is 10. Ops: []
W20260709 13:07:49.523850 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 129520us
I20260709 13:07:49.656558 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2178) took 1727 ms. Trace:
I20260709 13:07:49.656641 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.657140 21068 rpcz_store.cc:276] 0709 13:07:47.929668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.398281 (+1468613us) service_pool.cc:224] Handling call
0709 13:07:49.419539 (+ 21258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.656211 (+236672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.656483 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.658732 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:49.526432 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:49.660183 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.660300 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:49.661090 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.661432 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.661093 21068 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:49.662020 21221 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:49.662858 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.663547 25471 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:49.526633 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.530800 21219 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:07:49.487862 25056 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:49.532807 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2187) took 1598 ms. Trace:
I20260709 13:07:49.669848 21072 rpcz_store.cc:276] 0709 13:07:47.934774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.469615 (+1534841us) service_pool.cc:224] Handling call
0709 13:07:49.472820 (+  3205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.529797 (+ 56977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.532750 (+  2953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.532933 21217 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:49.501964 25449 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.475677 25350 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.499794 25395 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.612202 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:49.622087 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800046c20 after lost signal to thread 21004
W20260709 13:07:49.623567 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.529320 25036 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.625381 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2188) took 1690 ms. Trace:
I20260709 13:07:49.602295 20967 catalog_manager.cc:5697] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:49.530872 25495 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:49.584514 25313 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.822s	user 0.013s	sys 0.000s
W20260709 13:07:49.628499 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 175134us
I20260709 13:07:49.681205 20967 catalog_manager.cc:5697] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 6 to 7, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 7 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:49.682240 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:49.683229 25313 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:49.651893 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 361466us
I20260709 13:07:49.595724 25441 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:49.664896 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.622762 25387 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:49.665174 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.658960 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:49.596271 25494 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.596967 25398 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:49.621605 25400 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:49.668530 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.669154 21219 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:49.709558 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:49.710563 21219 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:49.673262 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.531358 25000 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.558991 25357 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 13:07:49.588900 25335 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:49.677345 21072 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 7. Current term is 8. Ops: []
W20260709 13:07:49.680289 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800314320 after lost signal to thread 21096
I20260709 13:07:49.679675 21071 rpcz_store.cc:276] 0709 13:07:47.935139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.485720 (+1550581us) service_pool.cc:224] Handling call
0709 13:07:49.496567 (+ 10847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.625033 (+128466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.625295 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.728374 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2195) took 1790 ms. Trace:
W20260709 13:07:49.729000 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08000a23c0 after lost signal to thread 21171
I20260709 13:07:49.728737 21072 rpcz_store.cc:276] 0709 13:07:47.937653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.676679 (+1739026us) service_pool.cc:224] Handling call
0709 13:07:49.677086 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.728107 (+ 51021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.728318 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.683737 20967 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 7 to 8, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:49.695119 21217 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:07:49.730719 21217 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:49.730926 20967 catalog_manager.cc:5697] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 4 to 7, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 7 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:07:49.695202 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 218148us
I20260709 13:07:49.653196 25502 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:49.588248 25268 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.733124 25502 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:49.733662 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.734634 25268 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.699200 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.735951 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:49.736869 25506 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:49.739629 21065 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:49.673669 25056 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:49.740474 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.740497 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.743726 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:49.744014 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:49.744349 25471 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:49.744889 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:49.745069 25471 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:49.745720 25471 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:49.745077 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:49.745939 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:49.745841 25488 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:49.624548 25312 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.749091 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.749493 25312 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.749727 21144 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:49.750102 25312 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:49.626621 25508 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.751075 21144 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:49.751151 25513 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:49.751724 25513 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:49.752838 25508 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.752665 25513 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.753948 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.754043 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.754271 25498 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:49.755481 21214 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:49.755553 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.756187 25359 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:49.739650 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:49.756703 25359 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:49.756876 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:07:49.702620 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 129435us
I20260709 13:07:49.757918 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:49.708460 21215 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:49.758607 25499 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:49.759581 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.759788 25499 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:49.746658 25471 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.626194 25371 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:49.720144 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.699352 25036 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.761955 21145 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:49.762048 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:49.729342 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800139820 after lost signal to thread 21172
I20260709 13:07:49.680235 21068 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:07:49.763801 25494 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:49.764178 25494 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:49.743876 21799 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:49.764478 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2200) took 1597 ms. Trace:
I20260709 13:07:49.762967 25036 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.757647 25359 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.765139 25494 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.765877 25359 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:49.766433 25494 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:49.715350 21792 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:49.770917 21110 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:49.717730 21212 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:07:49.771950 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:49.772671 25036 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:49.773727 25506 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:49.775416 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.707868 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:49.775779 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.776962 21799 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:49.712412 21221 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:07:49.764276 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08000ff920 after lost signal to thread 21245
I20260709 13:07:49.739835 21795 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEARNER]: Advancing to term 9
I20260709 13:07:49.764896 21068 rpcz_store.cc:276] 0709 13:07:48.167123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.763710 (+1596587us) service_pool.cc:224] Handling call
0709 13:07:49.764001 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.764175 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.764202 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:49.764243 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:49.764280 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.764332 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.764424 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.767433 25449 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:49.762914 25371 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:49.767617 25498 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:49.753890 25513 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:49.780620 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 72874.3us
W20260709 13:07:49.781172 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800299900 after lost signal to thread 21246
I20260709 13:07:49.783249 25449 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:49.781893 25447 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:49.768019 25499 raft_consensus.cc:2764] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:07:49.732746 20967 catalog_manager.cc:5697] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:49.754146 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:49.786969 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:49.785784 21068 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:49.788050 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:49.788471 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.784438 21792 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 6: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:49.793915 21068 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:49.782147 21242 debug-util.cc:398] Leaking SignalData structure 0x7b080032ebe0 after lost signal to thread 21796
I20260709 13:07:49.781526 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:49.794759 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003b72a0 after lost signal to thread 21825
W20260709 13:07:49.795105 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08000d2260 after lost signal to thread 25000
I20260709 13:07:49.788506 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:49.789948 21072 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:49.693646 25441 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:49.667387 25422 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.800487 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.707234 25495 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:49.797883 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2199) took 1650 ms. Trace:
I20260709 13:07:49.781721 25510 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:49.784338 25498 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:49.795205 21068 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:07:49.795367 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08002fef00 after lost signal to thread 25036
I20260709 13:07:49.697237 25313 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.099s
W20260709 13:07:49.809518 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51680.5us
I20260709 13:07:49.807406 21072 rpcz_store.cc:276] 0709 13:07:48.147163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.789275 (+1642112us) service_pool.cc:224] Handling call
0709 13:07:49.789695 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.797459 (+  7764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.797802 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:49.785768 25447 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.654738 25500 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:49.726050 25000 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.801076 20967 catalog_manager.cc:5697] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 5 to 7, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 7 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:49.784927 25514 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:07:49.668087 25512 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.788612 25449 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:49.864439 21068 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:49.806084 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:49.866158 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60081.5us
I20260709 13:07:49.866808 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:49.810979 25382 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:49.778115 25506 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 13:07:49.860509 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800222ee0 after lost signal to thread 25126
I20260709 13:07:49.832358 21219 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:49.835275 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 132675us
I20260709 13:07:49.809944 25495 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.851930 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.799121 25268 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:49.835059 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:49.868363 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003778e0 after lost signal to thread 25136
I20260709 13:07:49.868080 20967 catalog_manager.cc:5697] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 3 to 4, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 4 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:49.892365 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800396280 after lost signal to thread 25182
I20260709 13:07:49.870451 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:49.799027 25308 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:49.832582 25498 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:49.893577 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2435) took 1179 ms. Trace:
I20260709 13:07:49.930845 21139 rpcz_store.cc:276] 0709 13:07:48.714422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.798509 (+ 84087us) service_pool.cc:224] Handling call
0709 13:07:49.893387 (+1094878us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b540036cc54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:49.893520 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026848640}
I20260709 13:07:49.786424 24978 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.897357 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:49.899734 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003f8f80 after lost signal to thread 25243
I20260709 13:07:49.865041 25447 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:49.800280 25371 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:07:49.831420 25335 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:49.883896 25488 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:49.881002 25398 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:49.900930 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 80724.3us
W20260709 13:07:49.960587 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 126709us
W20260709 13:07:49.947528 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800364980 after lost signal to thread 25268
I20260709 13:07:49.721024 25496 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.857268 25414 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:07:49.951855 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.950706 25363 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:49.796739 25416 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:49.881606 25400 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:49.884400 25495 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:49.804872 25516 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.147s
W20260709 13:07:49.966812 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003965e0 after lost signal to thread 25308
W20260709 13:07:49.990708 21242 debug-util.cc:398] Leaking SignalData structure 0x7b080012fd80 after lost signal to thread 25329
W20260709 13:07:49.991027 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08002d2300 after lost signal to thread 25350
W20260709 13:07:49.991250 21242 debug-util.cc:398] Leaking SignalData structure 0x7b080035b400 after lost signal to thread 25357
W20260709 13:07:49.991505 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800210b80 after lost signal to thread 25361
W20260709 13:07:49.991707 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003188e0 after lost signal to thread 25363
W20260709 13:07:49.991909 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800399b60 after lost signal to thread 25370
W20260709 13:07:49.992118 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800322920 after lost signal to thread 25374
W20260709 13:07:49.992322 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003e8f40 after lost signal to thread 25382
W20260709 13:07:49.992544 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003e8e80 after lost signal to thread 25387
W20260709 13:07:49.992741 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800397ee0 after lost signal to thread 25395
W20260709 13:07:49.992936 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08001e8480 after lost signal to thread 25396
W20260709 13:07:49.993382 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08000eb400 after lost signal to thread 25398
W20260709 13:07:49.993625 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800384740 after lost signal to thread 25399
W20260709 13:07:49.993854 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08000390e0 after lost signal to thread 25400
I20260709 13:07:49.939234 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.967144 20967 catalog_manager.cc:5697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 5 to 8, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 8 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:49.895148 21795 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEARNER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:49.901646 25313 consensus_queue.cc:1243] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:07:49.930388 25329 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:07:49.972759 21065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2182) took 2039 ms (client timeout 1544 ms). Trace:
I20260709 13:07:49.971681 21144 raft_consensus.cc:2368] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:49.994336 21139 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:49.997632 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.998843 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.953043 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:50.000765 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.945691 25308 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:49.984441 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:49.899921 25178 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:49.988900 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:50.002144 21139 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:50.003193 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:49.995725 21065 rpcz_store.cc:269] 0709 13:07:47.933572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.042029 (+108457us) service_pool.cc:224] Handling call
0709 13:07:49.972520 (+1930491us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54003a2fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:49.972660 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3901431040}
I20260709 13:07:49.995793 25329 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:50.006906 25000 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.007613 25329 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:50.008824 25525 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:07:50.012558 25036 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:49.978544 25363 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.015604 25363 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:49.968990 25374 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.018199 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:50.018698 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:50.019410 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:50.020224 21212 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:50.018615 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:50.002609 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003e8ea0 after lost signal to thread 25414
W20260709 13:07:50.022387 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08001cf3e0 after lost signal to thread 25421
W20260709 13:07:50.022676 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003e8e60 after lost signal to thread 25433
W20260709 13:07:50.022907 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08003b56e0 after lost signal to thread 25438
W20260709 13:07:50.023211 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08001da3a0 after lost signal to thread 25447
W20260709 13:07:50.023461 21242 debug-util.cc:398] Leaking SignalData structure 0x7b080033a740 after lost signal to thread 25449
I20260709 13:07:49.970329 25488 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:49.983906 25357 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:50.028766 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:49.969991 25520 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:50.027563 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:50.030820 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:50.034848 21795 raft_consensus.cc:2376] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11. Re-sending same reply.
I20260709 13:07:49.981930 25400 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:50.036291 25400 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:50.045789 25400 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.052292 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:50.052985 21799 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:07:50.056298 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:50.061756 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:50.064944 21142 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:50.067164 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:50.068189 21143 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:07:50.062364 25528 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:50.055444 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:49.996888 25468 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.026445 25442 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:50.073729 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:50.071766 20967 catalog_manager.cc:5697] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:50.081168 21799 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
W20260709 13:07:50.062737 25517 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
I20260709 13:07:50.051764 25313 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:50.051779 25357 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:50.033468 25395 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.053205 25501 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:50.081492 20967 catalog_manager.cc:5697] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:50.065423 25516 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:07:50.084735 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 215841us
W20260709 13:07:50.082144 25448 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.846s	user 0.009s	sys 0.002s
I20260709 13:07:50.066511 25514 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:07:50.076673 25506 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:07:50.141469 25524 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 13:07:50.140223 25523 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:07:50.075610 25528 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:50.239410 25528 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:50.240702 25528 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:50.211376 25448 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:50.242048 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:50.242632 25448 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:50.243491 25471 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:50.244001 25471 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:50.077435 25535 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:50.244869 25471 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:50.246460 25502 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.000s
I20260709 13:07:50.161137 20967 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 6 to 9, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 9 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:07:50.226941 25178 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:50.165241 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.091658 25357 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:50.169078 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 152523us
I20260709 13:07:50.249357 20967 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:50.267877 21070 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:50.268635 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1340) took 1150 ms. Trace:
I20260709 13:07:50.268970 21070 rpcz_store.cc:276] 0709 13:07:49.118284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.267401 (+1149117us) service_pool.cc:224] Handling call
0709 13:07:50.267691 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.268389 (+   698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.268581 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.270174 21070 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:50.270711 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1341) took 1152 ms. Trace:
W20260709 13:07:50.274384 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.276526 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:07:50.267966 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.172446 25516 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:50.217962 25313 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:50.173508 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.210687 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:50.213644 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:50.063246 25489 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.166800 25400 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.095345 25422 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.178203 25136 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:50.219405 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:50.219516 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:50.078938 25335 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.228116 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2177) took 2300 ms. Trace:
I20260709 13:07:50.296181 21066 rpcz_store.cc:276] 0709 13:07:47.928745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.279336 (+1350591us) service_pool.cc:224] Handling call
0709 13:07:49.306963 (+ 27627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:49.312485 (+  5522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.312527 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:49.313093 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:07:49.686414 (+373321us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:49.731382 (+ 44968us) log.cc:844] Serialized 14 byte log entry
0709 13:07:49.731585 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.731652 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:50.227896 (+496244us) raft_consensus.cc:1685] finished
0709 13:07:50.227921 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.228054 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":405612,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":249,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"thread_start_us":296430,"threads_started":2,"wal-append.queue_time_us":488413}
I20260709 13:07:50.294292 20967 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 4 to 6, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 6 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:07:50.232774 25530 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.155s	user 0.002s	sys 0.000s
I20260709 13:07:50.233433 25414 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:50.213963 25488 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:50.234826 25529 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 13:07:50.235930 25536 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:07:50.236796 21792 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:50.237607 25515 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:50.219097 25541 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:07:50.163256 25308 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.243165 21035 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:50.285961 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:50.271006 21070 rpcz_store.cc:276] 0709 13:07:49.118553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.269723 (+1151170us) service_pool.cc:224] Handling call
0709 13:07:50.270013 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.270485 (+   472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.270661 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.287563 21140 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:50.145203 25527 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.222482 25506 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:50.285056 25126 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:50.302454 25182 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:50.305275 20967 catalog_manager.cc:5697] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:07:50.298769 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 243413us
I20260709 13:07:50.283806 25512 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:50.246891 25502 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.291553 25513 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:07:50.299610 25056 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.556s	user 0.006s	sys 0.007s
W20260709 13:07:50.324824 25433 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.697s	user 0.010s	sys 0.000s
W20260709 13:07:50.316252 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 96795.1us
W20260709 13:07:50.357404 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:50.358047 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.358772 21063 raft_consensus.cc:2410] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:07:50.360759 25531 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:07:50.313205 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 461380us
I20260709 13:07:50.308490 25537 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.302934 25447 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:50.327116 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1333) took 1249 ms. Trace:
I20260709 13:07:50.311877 25422 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:50.331180 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1332) took 1354 ms. Trace:
W20260709 13:07:50.345400 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 102273us
I20260709 13:07:50.372598 21220 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:50.347422 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61550.9us
I20260709 13:07:50.375694 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:07:50.361909 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.361814 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:50.366412 21068 rpcz_store.cc:276] 0709 13:07:49.078010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.873936 (+795926us) service_pool.cc:224] Handling call
0709 13:07:49.874356 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.874624 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:07:50.089242 (+214618us) log.cc:844] Serialized 14 byte log entry
0709 13:07:50.326732 (+237490us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:50.326795 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:50.326841 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.326921 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.327050 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":346635,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":1574,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":4935,"thread_start_us":432752,"threads_started":2,"wal-append.queue_time_us":270220}
I20260709 13:07:50.367588 20967 catalog_manager.cc:5697] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 4 to 7, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 7 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:50.377604 20967 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:50.369199 25537 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.353036 25468 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:50.349063 25178 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:50.353384 25513 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:50.303638 25516 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:50.337026 25515 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:50.381677 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2197) took 2442 ms. Trace:
I20260709 13:07:50.394919 20967 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:50.361235 25531 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:50.391721 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:50.393215 21139 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:50.376916 21072 rpcz_store.cc:276] 0709 13:07:48.976790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.850115 (+873325us) service_pool.cc:224] Handling call
0709 13:07:49.850559 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.850820 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:50.168656 (+317836us) log.cc:844] Serialized 14 byte log entry
0709 13:07:50.313459 (+144803us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:50.313514 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:50.313558 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.313651 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.331104 (+ 17453us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":327668,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":2509,"raft.queue_time_us":13370,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":1681,"spinlock_wait_cycles":16706944,"thread_start_us":461598,"threads_started":2,"wal-append.queue_time_us":166477}
I20260709 13:07:50.395354 21071 rpcz_store.cc:276] 0709 13:07:47.939337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.728422 (+1789085us) service_pool.cc:224] Handling call
0709 13:07:49.728699 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:49.729019 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.729051 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:49.729520 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:07:49.888134 (+158614us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:49.888176 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.888243 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:50.381447 (+493204us) raft_consensus.cc:1685] finished
0709 13:07:50.381467 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.381609 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":701,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":8965,"spinlock_wait_cycles":1960320,"thread_start_us":88356,"threads_started":1,"wal-append.queue_time_us":151078}
I20260709 13:07:50.402719 20967 catalog_manager.cc:5697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:07:50.401672 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1344) took 1245 ms. Trace:
I20260709 13:07:50.407586 21072 rpcz_store.cc:276] 0709 13:07:49.156297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.400883 (+1244586us) service_pool.cc:224] Handling call
0709 13:07:50.401198 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.401362 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:50.401387 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:50.401431 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:50.401467 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.401518 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.401621 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.407595 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:50.387651 25468 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:50.412550 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.365087 25126 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.346202 25313 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:50.411381 25545 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:07:50.413796 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 697) took 1236 ms. Trace:
I20260709 13:07:50.392855 25513 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:50.364019 25056 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.429111 21066 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:07:50.397698 25182 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:50.425783 21072 rpcz_store.cc:276] 0709 13:07:49.177238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.408793 (+1231555us) service_pool.cc:224] Handling call
0709 13:07:50.409196 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.411112 (+  1916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:50.411141 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:50.411201 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:50.411237 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.413578 (+  2341us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.413726 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.433781 21072 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:50.434401 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 698) took 1249 ms. Trace:
I20260709 13:07:50.413044 25350 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:50.352191 25527 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:50.406912 25542 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 13:07:50.439500 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 77766.6us
W20260709 13:07:50.442016 25359 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.004s
I20260709 13:07:50.412173 25178 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:50.415915 25468 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:50.434792 21072 rpcz_store.cc:276] 0709 13:07:49.184602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.433204 (+1248602us) service_pool.cc:224] Handling call
0709 13:07:50.433567 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.434161 (+   594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.434352 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.453403 21072 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:50.453954 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 699) took 1263 ms. Trace:
I20260709 13:07:50.454146 21072 rpcz_store.cc:276] 0709 13:07:49.190864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.452932 (+1262068us) service_pool.cc:224] Handling call
0709 13:07:50.453241 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.453783 (+   542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.453915 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.416224 25516 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:50.431630 25395 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
W20260709 13:07:50.451347 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.458137 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:50.459517 25556 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.460052 25556 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:50.461351 25387 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.745s	user 0.011s	sys 0.000s
I20260709 13:07:50.461792 25387 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.462335 25556 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:50.462977 25449 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.001s
I20260709 13:07:50.445268 25182 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:50.463716 25547 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:50.464730 25056 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:50.465219 25126 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:50.429333 25448 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:07:50.476665 25449 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.447268 25531 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.476264 25182 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:50.432427 25354 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:07:50.484102 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:50.462913 25556 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:50.490547 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 700) took 1287 ms. Trace:
I20260709 13:07:50.491178 21072 rpcz_store.cc:276] 0709 13:07:49.203350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.454764 (+1251414us) service_pool.cc:224] Handling call
0709 13:07:50.489896 (+ 35132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.490101 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:50.490135 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:50.490197 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:50.490243 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.490318 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.490471 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.493821 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1342) took 1358 ms. Trace:
I20260709 13:07:50.494508 21065 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:50.495224 21070 rpcz_store.cc:276] 0709 13:07:49.135481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.342704 (+1207223us) service_pool.cc:224] Handling call
0709 13:07:50.343141 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.343608 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:50.344011 (+   403us) log.cc:844] Serialized 14 byte log entry
0709 13:07:50.493491 (+149480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:50.493542 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:50.493588 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.493653 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.493766 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1331,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":544,"thread_start_us":148564,"threads_started":1}
I20260709 13:07:50.500413 25395 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:50.500131 25492 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:50.432132 25535 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 13:07:50.512616 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.502763 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1343) took 1347 ms. Trace:
I20260709 13:07:50.504195 21070 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:50.504302 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:50.529307 21187 leader_election.cc:400] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11.
I20260709 13:07:50.529760 21187 leader_election.cc:403] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:50.505098 21072 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:50.412797 25506 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.488467 25547 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:50.526963 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 702) took 1310 ms. Trace:
I20260709 13:07:50.530646 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 701) took 1319 ms. Trace:
I20260709 13:07:50.548619 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:50.548663 21070 rpcz_store.cc:276] 0709 13:07:49.216486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.503638 (+1287152us) service_pool.cc:224] Handling call
0709 13:07:50.503901 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.526649 (+ 22748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.526905 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.498674 25266 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:50.555941 21070 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:50.548971 21072 rpcz_store.cc:276] 0709 13:07:49.211578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.492402 (+1280824us) service_pool.cc:224] Handling call
0709 13:07:50.504858 (+ 12456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.530375 (+ 25517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.530590 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.563237 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:50.565711 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:50.502739 25542 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:07:50.556975 21067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2194) took 2619 ms (client timeout 1864 ms). Trace:
I20260709 13:07:50.557754 21221 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:50.579753 21221 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:50.579665 21067 rpcz_store.cc:269] 0709 13:07:47.937373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.648944 (+711571us) service_pool.cc:224] Handling call
0709 13:07:50.556776 (+1907832us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b5400232dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:50.556906 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449519872}
I20260709 13:07:50.559629 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1389) took 1146 ms. Trace:
I20260709 13:07:50.581408 21220 rpcz_store.cc:276] 0709 13:07:49.412954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.415514 (+  2560us) service_pool.cc:224] Handling call
0709 13:07:49.415831 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.373079 (+957248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.559384 (+186305us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b54004439d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:50.559569 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2199915776}
I20260709 13:07:50.581804 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:50.567987 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 704) took 1340 ms. Trace:
I20260709 13:07:50.563074 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:50.582906 21070 rpcz_store.cc:276] 0709 13:07:49.227729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.550203 (+1322474us) service_pool.cc:224] Handling call
0709 13:07:50.550496 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.567661 (+ 17165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.567927 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.588160 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:50.590832 21036 leader_election.cc:400] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:50.505877 25395 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:50.591235 21036 leader_election.cc:403] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:50.584901 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:50.598790 21072 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:50.580709 21221 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:50.599889 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:50.602674 21221 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:50.604720 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:50.585199 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:50.608633 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:50.612110 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1346) took 1294 ms. Trace:
I20260709 13:07:50.572041 25506 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:50.616515 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:50.583581 25528 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11.
I20260709 13:07:50.583467 25471 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:50.585426 25542 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:50.656138 25421 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.035s	user 0.013s	sys 0.000s
I20260709 13:07:50.656829 25471 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:50.664215 25471 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:50.667637 25537 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.624409 25350 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:50.669279 25350 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:50.668848 25547 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.664338 25565 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:50.670468 25565 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:50.670845 25565 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:50.633644 25554 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
W20260709 13:07:50.634537 25563 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:50.594902 25488 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:50.597929 25400 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:50.639415 25518 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:50.549103 25363 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:50.645641 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:50.646014 25564 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:07:50.623816 25387 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.653335 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:50.737761 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 84561.3us
I20260709 13:07:50.594563 25556 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:50.595022 25447 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:50.597601 25502 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.669711 25350 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:50.632239 25527 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:50.613679 25329 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:07:50.615049 25489 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:50.623626 21072 rpcz_store.cc:276] 0709 13:07:49.317944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.563672 (+1245728us) service_pool.cc:224] Handling call
0709 13:07:50.598595 (+ 34923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.603257 (+  4662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.611920 (+  8663us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:50.606720 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:50.776346 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 169696us
I20260709 13:07:50.658125 25542 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:50.727036 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:50.719870 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:50.728205 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.609977 25335 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:07:50.788647 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:50.746546 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.789830 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:50.790735 21798 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:50.790817 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:50.745636 25421 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:50.751796 25552 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:50.735098 25400 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:50.734783 25518 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:50.755388 25363 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.759872 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:50.762266 25496 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:50.763212 25386 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:50.646631 25395 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:50.772670 21216 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:50.777287 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:50.771818 25487 raft_consensus.cc:2243] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.778435 21792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:07:50.812668 21792 raft_consensus.cc:606] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Received request to transfer leadership to TS 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:50.813155 21792 raft_consensus.cc:614] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:50.778664 25182 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:07:50.779805 25495 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.776s	user 0.012s	sys 0.000s
I20260709 13:07:50.784215 21220 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
W20260709 13:07:50.788693 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.816676 21220 pending_rounds.cc:85] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 6
I20260709 13:07:50.816969 21220 pending_rounds.cc:107] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:07:50.787984 25489 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.793915 25421 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:50.826499 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:50.827459 21072 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:50.804425 25400 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:50.801496 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:50.804713 25518 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:50.753336 21794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 905) took 2466 ms (client timeout 1917 ms). Trace:
I20260709 13:07:50.802158 21799 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:50.802824 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:07:50.776386 25550 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.440s	user 0.004s	sys 0.000s
I20260709 13:07:50.804437 21067 raft_consensus.cc:2376] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15. Re-sending same reply.
I20260709 13:07:50.806113 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:50.844547 21214 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:50.794658 25387 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:50.671926 25357 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:50.817337 21220 raft_consensus.cc:2889] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:50.792779 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:50.751873 21147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2895) took 1512 ms (client timeout 1852 ms). Trace:
W20260709 13:07:50.852245 21147 rpcz_store.cc:269] 0709 13:07:49.239078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.239702 (+   624us) service_pool.cc:224] Handling call
0709 13:07:50.751587 (+1511885us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54001ead54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:50.751792 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452331776}
I20260709 13:07:50.804013 25488 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:50.828655 25489 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:50.837198 21794 rpcz_store.cc:269] 0709 13:07:48.287161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.724910 (+437749us) service_pool.cc:224] Handling call
0709 13:07:50.753049 (+2028139us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54005d5f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:50.753216 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4277484288}
I20260709 13:07:50.808967 25386 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:50.845762 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:50.816746 25449 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:50.841349 21797 raft_consensus.cc:2410] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:07:51.333175 21822 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
W20260709 13:07:51.180517 20967 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.012s	sys 0.019s
W20260709 13:07:51.488780 20967 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.013s	sys 0.019s
I20260709 13:07:51.264869 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:51.254201 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 694) took 2170 ms. Trace:
W20260709 13:07:51.273306 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 480561us
I20260709 13:07:51.490790 21070 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:51.490386 21065 rpcz_store.cc:276] 0709 13:07:49.083637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.006412 (+922775us) service_pool.cc:224] Handling call
0709 13:07:50.006725 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.495295 (+488570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.906575 (+411280us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54003ca854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.254139 (+347564us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1121224064}
I20260709 13:07:51.493779 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:51.493803 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:51.273648 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1348) took 1772 ms (client timeout 1695 ms). Trace:
W20260709 13:07:51.495736 21063 rpcz_store.cc:269] 0709 13:07:49.501231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.534986 (+ 33755us) service_pool.cc:224] Handling call
0709 13:07:50.904139 (+1369153us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54003bef54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.273562 (+369423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891498368}
I20260709 13:07:51.496713 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:51.264972 25136 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:07:51.347704 25329 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.580s	user 0.014s	sys 0.001s
I20260709 13:07:50.526577 21068 rpcz_store.cc:276] 0709 13:07:49.155901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.377216 (+1221315us) service_pool.cc:224] Handling call
0709 13:07:50.377494 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.377689 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:50.378886 (+  1197us) log.cc:844] Serialized 14 byte log entry
0709 13:07:50.502397 (+123511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:50.502470 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:50.502525 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.502600 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.502709 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1404,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":1697,"raft.queue_time_us":92,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":4864,"thread_start_us":120018,"threads_started":1}
W20260709 13:07:51.399497 25565 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.007s
I20260709 13:07:51.508307 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1349) took 1987 ms. Trace:
W20260709 13:07:51.258658 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:51.254330 25570 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.447s	user 0.002s	sys 0.009s
I20260709 13:07:51.260140 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2916) took 1361 ms. Trace:
I20260709 13:07:51.252178 21138 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:51.251454 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 555) took 1502 ms. Trace:
I20260709 13:07:51.511700 21140 rpcz_store.cc:276] 0709 13:07:49.749025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.753366 (+  4341us) service_pool.cc:224] Handling call
0709 13:07:49.753636 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.341535 (+587899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.122403 (+780868us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b540032cdd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.251375 (+128972us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226884992}
I20260709 13:07:51.513801 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:51.261190 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2907) took 1861 ms. Trace:
I20260709 13:07:51.464859 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:51.466342 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:51.466419 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:51.267035 21146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2905) took 1870 ms (client timeout 1531 ms). Trace:
I20260709 13:07:51.467842 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:51.281958 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:51.262179 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.298913 21217 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:51.545140 21142 rpcz_store.cc:276] 0709 13:07:49.399432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.399561 (+   129us) service_pool.cc:224] Handling call
0709 13:07:51.102541 (+1702980us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540036e054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.261119 (+158578us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1377772416}
W20260709 13:07:51.253412 21221 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.650s	user 0.013s	sys 0.002s
W20260709 13:07:51.469465 21212 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.010s	sys 0.029s
I20260709 13:07:51.254218 21214 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:51.264792 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 559) took 1714 ms. Trace:
I20260709 13:07:51.265832 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 555) took 1719 ms. Trace:
I20260709 13:07:51.556746 21219 rpcz_store.cc:276] 0709 13:07:49.547038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.721117 (+174079us) service_pool.cc:224] Handling call
0709 13:07:49.727917 (+  6800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.868851 (+140934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.904991 (+1036140us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54003b5bd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.265773 (+360782us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239695232}
I20260709 13:07:51.556612 21215 rpcz_store.cc:276] 0709 13:07:49.550341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.758909 (+208568us) service_pool.cc:224] Handling call
0709 13:07:50.249788 (+490879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.249970 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:50.249995 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:50.250036 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:50.250072 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.250125 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:50.884924 (+634799us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b5400104350. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.264722 (+379798us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1128185216}
I20260709 13:07:50.870159 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:51.559423 25547 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.560009 21220 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:51.560554 25547 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.560128 21221 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:07:51.562719 21221 pending_rounds.cc:85] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 5
I20260709 13:07:51.563086 21221 pending_rounds.cc:107] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:07:51.563401 21221 raft_consensus.cc:2889] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:51.563656 25513 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.119s	user 0.008s	sys 0.015s
W20260709 13:07:50.972566 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:51.574548 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:50.854460 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 115879us
I20260709 13:07:51.286998 21213 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:51.274807 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:51.262746 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.581574 25544 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:07:51.582062 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 307206us
W20260709 13:07:51.272823 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:51.485342 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.899148 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:51.592032 25575 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.592922 25575 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.593712 21801 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:51.593858 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1351) took 1029 ms. Trace:
I20260709 13:07:51.594261 21067 rpcz_store.cc:276] 0709 13:07:50.564449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.592470 (+1028021us) service_pool.cc:224] Handling call
0709 13:07:51.593017 (+   547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.593429 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:51.593453 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:51.593519 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:51.593549 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.593596 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:51.593712 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:51.274860 25182 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:51.264453 25184 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:51.598146 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:51.598893 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 553) took 1808 ms. Trace:
I20260709 13:07:51.596479 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:51.599365 21138 rpcz_store.cc:276] 0709 13:07:49.743612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.747463 (+  3851us) service_pool.cc:224] Handling call
0709 13:07:50.753936 (+1006473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.511535 (+757599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.514656 (+  3121us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54001ead54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.544505 (+ 29849us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001ead50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.551499 (+  6994us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3294433536}
I20260709 13:07:51.600171 21800 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:51.357018 25266 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:07:51.498284 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:51.262344 21064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1337) took 2183 ms (client timeout 2096 ms). Trace:
W20260709 13:07:51.185060 25313 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.014s	sys 0.037s
I20260709 13:07:51.603545 25182 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:51.604668 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1363) took 1030 ms. Trace:
I20260709 13:07:51.605340 25182 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:07:51.288594 25350 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.010s
I20260709 13:07:51.605159 21063 rpcz_store.cc:276] 0709 13:07:50.574081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.497758 (+923677us) service_pool.cc:224] Handling call
0709 13:07:51.604583 (+106825us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:51.607430 25350 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:51.095544 25361 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.008s	sys 0.008s
I20260709 13:07:51.271446 25363 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:51.297614 25386 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:50.842749 25395 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:51.150171 25396 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.608808 21063 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:51.255225 25398 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:51.611917 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.612121 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1356) took 1046 ms. Trace:
I20260709 13:07:51.612417 25398 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:51.612576 21063 rpcz_store.cc:276] 0709 13:07:50.565961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.607786 (+1041825us) service_pool.cc:224] Handling call
0709 13:07:51.608277 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.611807 (+  3530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.612033 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:51.613612 25533 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:07:51.261832 25399 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:51.615674 25399 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:51.329942 25400 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.575419 25387 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.621932 25400 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:51.279762 25437 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:51.622658 21800 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:51.623418 25437 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:51.624518 21213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1405) took 1640 ms (client timeout 1611 ms). Trace:
W20260709 13:07:51.625211 25498 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.673s	user 0.021s	sys 0.003s
W20260709 13:07:51.625098 21213 rpcz_store.cc:269] 0709 13:07:49.984037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.988640 (+  4603us) service_pool.cc:224] Handling call
0709 13:07:51.623997 (+1635357us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540015c554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.624236 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2947824000}
I20260709 13:07:50.835940 25421 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:51.284328 25468 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.799s	user 0.010s	sys 0.010s
I20260709 13:07:51.261475 25504 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:50.829056 25495 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:51.292357 25526 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.354s	user 0.000s	sys 0.013s
I20260709 13:07:51.656559 25504 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:51.668073 21800 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:51.668257 25504 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:51.669384 25489 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.003s
I20260709 13:07:51.670094 25504 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:50.893296 25515 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.621866 25387 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.254185 25518 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.673477 25515 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:51.673879 25387 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.652078 25547 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:51.657193 25386 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:51.286602 25531 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:51.254312 25534 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:51.683670 25534 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:51.685418 25534 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:51.252633 25562 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.844s	user 0.006s	sys 0.001s
I20260709 13:07:51.495117 21065 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:51.689132 21065 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:07:51.689384 25534 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:51.503060 25329 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:51.504853 25503 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:51.692865 25503 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.508716 25565 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.693585 25503 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.695989 25565 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:50.858609 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 78553.3us
I20260709 13:07:51.462826 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:51.253722 25571 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.381s	user 0.000s	sys 0.011s
I20260709 13:07:51.273470 25574 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:51.508824 21068 rpcz_store.cc:276] 0709 13:07:49.520748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.506744 (+1985996us) service_pool.cc:224] Handling call
0709 13:07:51.507360 (+   616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.507777 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:51.507806 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:51.507863 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:51.507902 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.507986 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:51.508111 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:51.702685 21068 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:51.702965 21141 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:51.499449 25136 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:51.408078 25487 raft_consensus.cc:2272] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:51.526984 21146 rpcz_store.cc:269] 0709 13:07:49.396647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.396775 (+   128us) service_pool.cc:224] Handling call
0709 13:07:50.924203 (+1527428us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54002b43d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.266972 (+342769us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1744074496}
I20260709 13:07:51.511274 21139 rpcz_store.cc:276] 0709 13:07:49.898536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.029503 (+130967us) service_pool.cc:224] Handling call
0709 13:07:50.029799 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:50.400428 (+370629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.083779 (+683351us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54002599d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.260074 (+176295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825323264}
I20260709 13:07:51.716696 21146 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:51.717886 21139 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:51.348781 25441 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.534057 21070 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:51.533411 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:51.718537 25398 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.518098 25036 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.536046 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:51.537235 25433 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.537971 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:51.719235 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 705) took 2219 ms. Trace:
I20260709 13:07:51.721446 21147 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:51.541591 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:51.721632 21070 rpcz_store.cc:276] 0709 13:07:49.499566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.498150 (+1998584us) service_pool.cc:224] Handling call
0709 13:07:51.518010 (+ 19860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.718972 (+200962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.719152 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:51.722944 21145 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:51.542313 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:51.555500 21212 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.011s	sys 0.029s
I20260709 13:07:51.723517 21070 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:51.724156 21144 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:51.720302 25036 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:51.716686 21141 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:51.565140 21142 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:07:51.543335 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:51.726823 25547 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:51.727311 25547 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.727723 25547 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:51.564397 25513 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.730037 25514 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:51.732041 25513 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.726516 25036 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:07:51.585898 25506 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:07:51.726511 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 267252us
I20260709 13:07:51.734340 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:51.603817 25313 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.016s	sys 0.037s
I20260709 13:07:51.259532 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 695) took 2124 ms. Trace:
I20260709 13:07:51.736368 21144 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:07:51.736284 21066 rpcz_store.cc:276] 0709 13:07:49.134838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.297987 (+1163149us) service_pool.cc:224] Handling call
0709 13:07:50.320862 (+ 22875us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:50.429622 (+108760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.142689 (+713067us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54000fa854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.259460 (+116771us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246826752}
I20260709 13:07:51.737465 25437 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:51.605695 21064 rpcz_store.cc:269] 0709 13:07:49.079071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.178985 (+ 99914us) service_pool.cc:224] Handling call
0709 13:07:50.911245 (+1732260us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b540026e154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.262276 (+351031us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432878976}
I20260709 13:07:51.347237 25357 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:51.761400 25499 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:51.761865 25499 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:51.762804 25499 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.764228 25499 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:51.768378 25266 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:51.288523 25471 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.624s	user 0.012s	sys 0.007s
I20260709 13:07:51.778914 25471 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.781584 25471 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.609016 21218 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:51.626026 25498 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.617899 25399 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:51.635855 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 124488us
W20260709 13:07:51.610697 25361 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.603s	user 0.010s	sys 0.017s
I20260709 13:07:51.610210 25350 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:51.657558 21071 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:51.656558 21067 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:51.644425 25544 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:51.799070 25421 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.803849 25544 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.804458 25544 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:51.805665 25421 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:51.806288 25398 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:51.806764 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52064us
I20260709 13:07:51.611092 25363 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:51.809381 25586 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:51.810587 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 696) took 2645 ms. Trace:
I20260709 13:07:51.673110 25501 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.811811 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1355) took 1246 ms. Trace:
I20260709 13:07:51.811750 25501 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.812322 21067 rpcz_store.cc:276] 0709 13:07:50.565634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.626948 (+1061314us) service_pool.cc:224] Handling call
0709 13:07:51.656181 (+ 29233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.811529 (+155348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.811754 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:51.813760 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:51.814508 21067 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:51.670040 25489 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.656410 25468 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.675762 25518 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:51.676009 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2183) took 3742 ms. Trace:
I20260709 13:07:51.815547 25449 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.816200 25449 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.816808 25489 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.815783 21069 rpcz_store.cc:276] 0709 13:07:47.933833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.463602 (+1529769us) service_pool.cc:224] Handling call
0709 13:07:49.487042 (+ 23440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:49.487375 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.487401 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:49.495758 (+  8357us) log.cc:844] Serialized 26 byte log entry
0709 13:07:49.695840 (+200082us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:49.695901 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.695996 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:50.906971 (+1210975us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:51.675459 (+768488us) raft_consensus.cc:1685] finished
0709 13:07:51.675495 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:51.675824 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3558,"raft.queue_time_us":485,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":8200,"spinlock_wait_cycles":5552896,"thread_start_us":133543,"threads_started":1,"wal-append.queue_time_us":279477}
I20260709 13:07:51.818598 25449 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:51.285142 25516 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.799s	user 0.008s	sys 0.011s
I20260709 13:07:51.703789 21215 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:51.704391 21065 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:51.705855 25503 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.706059 21072 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:51.706867 25184 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.410087 25500 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:51.846730 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:51.710953 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:51.662650 25575 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:51.884729 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49927.5us
I20260709 13:07:51.720795 25433 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.732963 25506 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:51.587925 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.889425 25433 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.890905 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:51.891541 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 160333us
I20260709 13:07:51.892854 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1420) took 1081 ms. Trace:
I20260709 13:07:51.893254 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:51.894451 24965 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:51.893325 21212 rpcz_store.cc:276] 0709 13:07:50.811407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.814471 (+  3064us) service_pool.cc:224] Handling call
0709 13:07:50.814845 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.815098 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:51.724091 (+908993us) log.cc:844] Serialized 14 byte log entry
0709 13:07:51.864308 (+140217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:51.883608 (+ 19300us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:51.883742 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.883874 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:51.892666 (+  8792us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":601462,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":11349,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":198,"thread_start_us":986836,"threads_started":2,"wal-append.queue_time_us":123733}
I20260709 13:07:51.587011 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1406) took 1596 ms. Trace:
I20260709 13:07:51.895013 24965 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:51.735543 21796 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:07:51.745134 21145 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:51.745651 21063 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:07:51.748330 21068 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:51.738237 21801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:07:51.719333 25441 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:51.895241 21221 rpcz_store.cc:276] 0709 13:07:49.988427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.989451 (+  1024us) service_pool.cc:224] Handling call
0709 13:07:49.989747 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:50.557626 (+567879us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b5400127120. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42493dc0 00007f7f4251e1c0 00007f7f4251f231 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c
0709 13:07:51.564327 (+1006701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.585159 (+ 20832us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12978432}
I20260709 13:07:51.903213 25515 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:51.755890 21216 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:51.770648 25487 ts_tablet_manager.cc:1916] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:51.896881 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:51.905977 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1419) took 1098 ms. Trace:
I20260709 13:07:51.906044 25513 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:51.906602 25513 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.755825 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:51.907143 25513 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:51.906476 21216 rpcz_store.cc:276] 0709 13:07:50.807963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.810706 (+  2743us) service_pool.cc:224] Handling call
0709 13:07:50.811021 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.904850 (+1093829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.905675 (+   825us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b54002ee5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.905910 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2132880128}
I20260709 13:07:51.771768 21066 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:51.900537 21801 raft_consensus.cc:606] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:51.758939 21064 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:07:51.730912 25514 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:51.913548 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66977.9us
I20260709 13:07:51.793331 25498 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.793545 21794 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:51.915234 25514 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.899860 21065 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:51.800455 21220 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:51.916690 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1421) took 1076 ms. Trace:
I20260709 13:07:51.917037 21220 rpcz_store.cc:276] 0709 13:07:50.840141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.852536 (+ 12395us) service_pool.cc:224] Handling call
0709 13:07:51.916613 (+1064077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:51.809008 25363 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.917397 25544 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.817260 25468 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:51.305476 25542 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.502s	user 0.013s	sys 0.001s
I20260709 13:07:51.928097 25542 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.930305 25542 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:51.793601 21035 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:51.931530 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 138159us
I20260709 13:07:51.935652 21214 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:51.811036 21071 rpcz_store.cc:276] 0709 13:07:49.164842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.402895 (+1238053us) service_pool.cc:224] Handling call
0709 13:07:51.264637 (+861742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.808769 (+544132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.809671 (+   902us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b540026e154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.810305 (+   634us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b540026e150. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.810469 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2814030592}
I20260709 13:07:51.937877 25542 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.938601 25542 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.939733 25514 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:51.941077 21147 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:07:51.941463 21147 pending_rounds.cc:85] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 5
I20260709 13:07:51.941722 21147 pending_rounds.cc:107] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:07:51.941960 21147 raft_consensus.cc:2889] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:51.942476 21794 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:07:51.942875 21794 pending_rounds.cc:85] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 8
I20260709 13:07:51.943147 21794 pending_rounds.cc:107] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:07:51.942998 25498 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:51.943464 21794 raft_consensus.cc:2889] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:51.872711 25136 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.944212 25498 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:51.839047 25184 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.944964 25542 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:51.945112 25498 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:51.946048 25498 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:51.946369 25498 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:51.842514 25449 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:51.843565 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.947341 21801 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:51.947196 25449 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:51.949502 25449 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:51.950099 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:51.950940 21793 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:51.951414 25498 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.952558 25514 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.952895 25498 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:51.953219 25514 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:51.707583 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 103147us
I20260709 13:07:51.955582 25503 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:51.956197 25503 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:51.957362 25503 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.958709 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:51.853037 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:51.827708 25266 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.856734 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:51.855804 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:51.857614 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:51.872684 25496 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.873073 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:51.873658 21138 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:51.880440 25547 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.712322 25182 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:51.896052 24965 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:51.897783 25416 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.899554 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:51.622051 25493 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.913885 21070 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:51.914214 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:51.895481 25575 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260709 13:07:51.913905 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.794055 25574 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:51.791373 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1411) took 1020 ms. Trace:
I20260709 13:07:51.794552 25399 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:51.916133 25516 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:51.664287 25495 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:51.898501 21796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 493) took 2155 ms (client timeout 1647 ms). Trace:
I20260709 13:07:51.900709 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1362) took 1326 ms. Trace:
I20260709 13:07:51.960155 25518 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:07:51.981118 25421 raft_consensus.cc:2764] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:07:51.960618 25589 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:07:51.982136 25575 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:51.982774 25575 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:51.982825 25493 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.984757 21797 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:51.984126 25575 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.985160 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:51.962433 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.985970 25575 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:51.959805 21799 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:51.987021 25416 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.987367 21063 rpcz_store.cc:276] 0709 13:07:50.573790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.645316 (+1071526us) service_pool.cc:224] Handling call
0709 13:07:51.701021 (+ 55705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:51.899780 (+198759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.900428 (+   648us) spinlock_profiling.cc:243] Waited 37.9 ms on lock 0x7b540041cb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.900603 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87096064}
I20260709 13:07:51.988716 21218 rpcz_store.cc:276] 0709 13:07:50.770554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.771332 (+   778us) service_pool.cc:224] Handling call
0709 13:07:50.771620 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.789998 (+1018378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.790926 (+   928us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b540015ca54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:51.791256 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1924936064}
I20260709 13:07:51.989578 21144 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:07:51.990137 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.987812 25493 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:51.991796 21792 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:51.992286 21141 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:51.993111 25547 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:51.950102 25449 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:51.977128 25586 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:51.993844 21143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:51.994688 21143 raft_consensus.cc:606] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:51.918191 25363 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:51.995169 21143 raft_consensus.cc:614] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:51.919183 25515 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.996460 25515 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.814761 25501 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:51.840809 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1352) took 1276 ms. Trace:
I20260709 13:07:51.825884 25386 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:07:51.994125 25586 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:51.946486 25184 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.958894 25503 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:51.998193 21072 rpcz_store.cc:276] 0709 13:07:50.564755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.845466 (+280711us) service_pool.cc:224] Handling call
0709 13:07:51.840585 (+995119us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b540001aed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.840747 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981181312}
W20260709 13:07:51.999032 21796 rpcz_store.cc:269] 0709 13:07:49.742880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.744843 (+  1963us) service_pool.cc:224] Handling call
0709 13:07:51.898238 (+2153395us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b5400380254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:51.898403 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4457635712}
I20260709 13:07:52.006538 21793 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:07:52.008251 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:51.962807 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:52.013224 25386 consensus_queue.cc:1243] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:52.014079 25515 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:52.016423 25496 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.018115 25386 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:52.017946 25506 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.019534 21792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce55364f10c348819e7c63c08633b184"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:07:52.020195 21792 raft_consensus.cc:606] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:52.020565 21792 raft_consensus.cc:614] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:52.020656 25266 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:51.993958 25449 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:52.021533 25506 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:52.023979 25531 consensus_queue.cc:1175] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" responder_term: 10 status { last_received { term: 10 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 10 index: 9 } }
I20260709 13:07:52.024350 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:52.024441 25516 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.025185 21792 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:52.025190 25516 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:52.027458 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 102987us
I20260709 13:07:52.027606 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.028658 25395 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:52.029134 25416 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:52.029912 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:52.030652 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 68061.2us
I20260709 13:07:52.032342 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.033144 21063 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:52.032346 25036 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:52.034376 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:52.034708 25036 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:51.955569 25514 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:52.035168 25574 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:52.035403 25374 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:52.036880 25574 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:07:52.037606 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.037448 25374 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:52.036556 25496 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:52.055459 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:52.047118 21035 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:52.022037 25449 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:51.946319 25136 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:52.050776 25582 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:52.050747 25501 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:52.061406 21063 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:52.044008 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:52.062952 21070 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:52.010963 25542 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.071691 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.071954 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:52.073028 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.038282 25036 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:07:52.081013 21146 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:52.087822 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:52.059789 25416 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:52.097194 25589 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:52.098138 25496 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.099400 21144 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:52.098649 25036 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:52.059598 25184 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:52.119920 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51913.8us
I20260709 13:07:52.053701 25515 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 13:07:52.148244 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 104379us
I20260709 13:07:52.124086 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:52.066322 25513 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.125087 21767 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:52.100235 25437 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:07:52.136880 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49699us
I20260709 13:07:52.161051 21220 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:52.168735 21218 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:52.175482 21141 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:07:52.180920 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.118219 25036 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:52.117861 25504 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:52.158159 25395 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:52.151844 25495 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:52.186005 21214 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:52.122411 25542 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.203156 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:52.204433 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.200696 25447 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
W20260709 13:07:52.200364 25594 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.101s	user 0.001s	sys 0.001s
W20260709 13:07:52.207504 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.201519 25363 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:07:52.199329 25036 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.220999 21218 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:52.205562 25544 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:52.158983 25184 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.203979 21215 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:52.123550 25493 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:07:52.200645 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:52.225940 21145 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:07:52.089233 25531 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:07:52.228560 21220 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:52.212718 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:52.250084 21063 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:52.176059 25386 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:52.223254 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.204974 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:52.237049 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44625.6us
I20260709 13:07:52.179787 25437 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:52.239641 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44686.9us
I20260709 13:07:52.243475 25522 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:52.080603 25582 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.212350 25495 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.227617 25395 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:52.251045 21217 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:52.192852 25504 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:52.260218 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59667.4us
I20260709 13:07:52.248795 25503 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.263166 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:52.211764 25500 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.104099 25586 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.163920 25589 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.205559 25513 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:52.268816 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:52.257063 25386 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.265872 25361 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:52.200898 25441 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:07:52.211556 25496 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:52.260104 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:52.290146 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.264400 25544 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.276266 25515 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:52.302855 25544 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.303372 21795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:07:52.304039 21795 raft_consensus.cc:606] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:52.304409 21795 raft_consensus.cc:614] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:52.305523 25544 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:52.316336 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:52.316740 25589 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:52.316946 25499 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:52.317615 25499 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:52.317257 25437 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.320370 25184 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:52.321086 25496 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:52.321817 25496 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:52.323949 21221 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:07:52.327603 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 67396.6us
I20260709 13:07:52.330046 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.331349 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.332253 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.306362 25395 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.333312 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.334055 21138 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:07:52.307392 21216 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:52.335896 25516 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:52.336491 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:52.305994 25593 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 13:07:52.338551 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44009.1us
W20260709 13:07:52.308415 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45350.2us
I20260709 13:07:52.309152 25487 ts_tablet_manager.cc:1929] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.6
I20260709 13:07:52.309729 25542 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:52.310894 25357 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:52.283617 25518 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:52.312466 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:52.313452 25441 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:07:52.314150 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.315150 25514 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:52.314844 25503 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.315649 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.332430 25547 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:52.308463 25582 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:52.339586 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.322528 25496 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:52.384407 25487 log.cc:1199] T 0d78b35ac92d402db8a5e3fe45be52b5 P 5e5e65640c8b4f5ca7eadc6a14312c55: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/wals/0d78b35ac92d402db8a5e3fe45be52b5
I20260709 13:07:52.396237 25136 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:52.397109 25487 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46800 (request call id 0) took 4368 ms. Trace:
I20260709 13:07:52.285784 25386 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:52.397790 25496 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:52.312402 25495 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:52.399111 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.367038 25513 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.289114 25361 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:52.399657 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.281656 25266 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
W20260709 13:07:52.380504 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 88118.3us
I20260709 13:07:52.302325 25515 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:52.387583 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:52.362929 25350 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:52.283687 25500 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.431047 21138 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:07:52.318928 25499 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.401670 25514 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:52.437572 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.426715 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.439191 21072 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 5: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:52.439618 21070 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:52.438905 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.368919 25357 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:52.442128 21139 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:07:52.410029 25361 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:52.400678 25547 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:52.442648 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.446892 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.455924 21147 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:52.451006 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.367326 25421 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
W20260709 13:07:52.454668 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49612.2us
I20260709 13:07:52.459530 21064 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:52.436712 25350 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.466866 21217 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:07:52.473743 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:52.409874 24965 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:07:52.410077 25513 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.463183 25547 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:52.496665 25416 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:52.397624 25487 rpcz_store.cc:276] 0709 13:07:48.028259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.091783 (+ 63524us) service_pool.cc:224] Handling call
0709 13:07:48.667923 (+576140us) ts_tablet_manager.cc:1151] Deleting tablet 0d78b35ac92d402db8a5e3fe45be52b5
0709 13:07:48.668026 (+   103us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:07:50.922302 (+2254276us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b5400258fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42524362 00007f7f45f0f34c 00007f7f4753260a 00007f7f475632d0 00007f7f4754983d 00007f7f475497e9 00007f7f47549779 00007f7f47549741 00007f7f4754896d 000000000038f354
0709 13:07:51.950367 (+1028065us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:52.309012 (+358645us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:52.396971 (+ 87959us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1345928,"spinlock_wait_cycles":1076728192,"tablet-delete.queue_time_us":1150,"thread_start_us":19855,"threads_started":1}
I20260709 13:07:52.412760 25582 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:07:52.440330 25518 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.511114 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.474946 25350 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:52.515933 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 273592us
W20260709 13:07:52.517382 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.484941 25496 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:52.536377 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.412228 25504 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.540313 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.544368 21065 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:52.492769 25182 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:07:52.528342 25499 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:52.548236 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:52.551122 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:52.557093 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:52.557914 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:52.558024 21794 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:07:52.558775 21799 raft_consensus.cc:2410] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:07:52.558039 25500 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:52.559907 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:52.560495 21801 raft_consensus.cc:2376] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17. Re-sending same reply.
W20260709 13:07:52.561365 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 47739.1us
I20260709 13:07:52.562584 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:52.562880 21797 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:07:52.563796 25605 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:07:52.563476 25495 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:52.534225 25496 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:52.557435 25603 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:07:52.568212 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:52.570429 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 167758us
I20260709 13:07:52.570690 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:52.573177 21795 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:52.552441 25182 consensus_queue.cc:1243] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:07:52.592254 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.595196 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.596180 21065 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:07:52.603176 25500 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:52.603401 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.603890 25500 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:52.604235 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60568.5us
I20260709 13:07:52.604620 21146 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:52.606405 25500 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:52.607451 24965 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:52.607187 25361 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.609051 21140 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:07:52.571767 25496 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:52.612411 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:52.578580 25449 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:52.631044 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.531023 25515 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.620196 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:52.623346 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.618273 24965 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:52.531229 25357 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.636853 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:52.621078 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:52.627031 25498 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.674s	user 0.008s	sys 0.002s
I20260709 13:07:52.647914 21146 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:52.642983 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:52.659021 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.632773 25503 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:52.673009 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:52.670400 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet 0d78b35ac92d402db8a5e3fe45be52b5 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:07:52.670815 25361 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:52.566148 25514 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.682355 25514 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:52.655943 25515 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:52.685876 21063 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:52.595338 25447 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
W20260709 13:07:52.670029 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 58721.4us
I20260709 13:07:52.672575 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:52.676092 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:52.678454 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.679179 25615 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:07:52.700709 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.680017 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:52.680296 25613 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:07:52.652693 25518 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:52.648568 25374 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:07:52.689508 25386 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:52.709053 20955 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 85804.9us
I20260709 13:07:52.716759 21139 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:52.675032 25395 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:52.738304 25518 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.739013 25518 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.741902 25518 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:52.720886 25386 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:52.721632 21795 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:07:52.745260 25495 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:52.745929 25495 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.748366 25495 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:52.749357 25614 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:52.749866 25441 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:52.749989 25614 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:52.750416 25441 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:52.723567 21071 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:52.713706 25503 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:52.724828 25586 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:52.752876 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.753582 25441 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.752110 25503 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.755051 25503 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:52.724967 24965 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.756387 21794 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:07:52.728613 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 91872us
I20260709 13:07:52.729653 25036 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:52.732554 25542 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:52.771799 25542 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:52.772842 25542 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.733786 25437 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:52.773873 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:52.734122 25585 raft_consensus.cc:993] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:52.734527 25513 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:52.718832 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:52.743637 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.775203 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.776337 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:52.776680 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.777390 21145 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:52.778295 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:52.779556 25374 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:52.775231 21068 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:52.781090 25374 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:52.782615 25585 raft_consensus.cc:1081] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:52.768298 25182 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:52.784549 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.784943 25357 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:52.785257 21139 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:52.786003 25623 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:52.788586 21795 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:52.791069 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:52.779654 25513 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:52.807991 21068 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:52.790722 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:52.810940 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:52.819161 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:52.819873 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:52.815726 21792 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:52.835181 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:52.838146 21147 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:52.838624 21071 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:52.787168 25623 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:52.845356 21794 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:52.833328 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:52.846884 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.848068 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:52.773885 25542 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:52.859364 21064 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:52.856288 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.860399 21217 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:52.853381 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:52.857964 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:52.751176 25614 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.873652 21220 raft_consensus.cc:2368] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:52.874845 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:52.877831 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:52.879040 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:52.845028 25421 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:52.850246 21065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:60302
I20260709 13:07:52.887638 21065 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:52.888134 21065 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:52.833396 25182 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:52.833899 25498 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.856478 25386 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:52.832978 25500 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:52.893700 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.902343 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.858490 25449 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:52.872582 25499 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:52.884541 25501 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:52.911768 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.903265 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.924533 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:52.915479 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:52.906744 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.925206 21216 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:52.875948 25515 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:52.888798 25585 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:52.931134 21215 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:52.894220 25182 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:52.861171 25623 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:52.917925 21217 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:52.925843 21220 raft_consensus.cc:2410] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 11 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:07:52.903875 25416 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:52.892691 25421 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:52.845706 25374 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:07:52.924202 25501 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:52.943504 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 47112.5us
I20260709 13:07:52.885689 25399 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:07:52.942963 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.965765 21221 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
W20260709 13:07:52.945374 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 365423us
W20260709 13:07:52.951540 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 332022us
I20260709 13:07:52.945195 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.969755 21215 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:52.924577 25386 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:52.936720 25416 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.916692 25498 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.920399 25513 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:07:52.969135 24965 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:52.942632 25421 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:52.986908 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:52.988057 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.945600 25500 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:52.990424 21217 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:52.921651 25496 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.929586 25547 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:52.993607 21215 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:52.975065 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.945344 25501 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:53.001868 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 86483.7us
I20260709 13:07:53.012825 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:52.993202 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:52.987018 25416 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:53.026376 21221 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:07:53.023914 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 48395.3us
I20260709 13:07:52.931017 25495 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:07:52.989665 24965 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.032725 21220 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:52.992072 25361 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:53.033613 21142 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:52.985069 25632 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:53.034286 25632 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.032487 25629 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:07:53.034734 25632 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:53.039003 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.044262 21215 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:07:53.045238 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.045809 21215 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:07:53.046733 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:52.985788 25628 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:07:52.982051 25627 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:07:53.050035 21213 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:53.051574 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:53.039994 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 95734.2us
I20260709 13:07:53.042171 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.058640 21217 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:52.991009 25503 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.047268 21215 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:53.032680 24965 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:53.042424 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:53.036027 25374 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:53.036746 25361 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:53.091085 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.092855 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:53.094854 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.097724 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:53.098462 21213 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:53.062172 25499 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:53.041464 25449 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
W20260709 13:07:53.120114 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.169s (SENT) [suppressed 41 similar messages]
I20260709 13:07:53.134979 21217 pending_rounds.cc:85] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 4
I20260709 13:07:53.138592 21217 pending_rounds.cc:107] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:07:53.138983 21217 raft_consensus.cc:2889] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:53.142838 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.144090 21219 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 8: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:53.038650 25182 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.150501 21220 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:53.151319 21212 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:53.152012 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:53.157632 25622 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:07:53.103044 25374 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:53.162079 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.163784 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:53.164674 21071 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:53.166378 21072 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:53.167933 21067 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:53.114461 25629 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:53.171675 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:53.172659 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:53.180480 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:53.181038 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:53.181602 21146 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:53.181522 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:53.184366 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:53.185003 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 128607us
I20260709 13:07:53.185853 21139 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:07:53.186754 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.172763 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:53.187700 21219 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:53.188578 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:53.183725 21144 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:53.189548 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:53.184203 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:53.190833 21141 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:53.189551 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:53.193166 21138 raft_consensus.cc:2410] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:53.194036 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:53.055951 25623 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.195871 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:53.064040 25386 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.201385 21069 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:53.196780 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:53.196581 25395 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:53.225420 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:53.199765 25515 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:53.286890 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:53.216038 25350 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.682s	user 0.004s	sys 0.007s
I20260709 13:07:53.212450 25182 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:53.191610 25361 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:53.372330 25626 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:07:53.226984 25499 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.407769 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:53.534068 21796 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:07:53.393930 25586 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.001s
I20260709 13:07:53.416563 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:53.412391 25547 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:53.285306 25496 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:53.363747 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:53.380563 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.394381 25498 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:53.222429 25623 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:53.497521 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.475890 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.447165 21141 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:07:53.454377 21146 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:53.507125 21213 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:07:53.481727 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.498692 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.191093 25629 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.513291 21219 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:07:53.578022 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.578673 21219 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
W20260709 13:07:53.337725 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.515178 21216 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:53.528755 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:53.532168 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.540293 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:53.581416 21071 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:53.512785 25644 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:53.549314 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.584403 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.584762 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.585067 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.587191 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:53.550356 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:53.588502 21068 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:53.472746 25495 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:53.534469 25639 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 13:07:53.595553 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.557286 21065 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.000s
I20260709 13:07:53.590468 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:53.533879 25500 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:53.539707 25638 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.591799 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:07:53.557711 25630 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.522s	user 0.001s	sys 0.010s
I20260709 13:07:53.605414 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:53.594102 21064 consensus_meta.cc:287] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.728s	user 0.009s	sys 0.001s
I20260709 13:07:53.614313 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:53.615178 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.615599 21212 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:53.616598 21064 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:53.617347 21070 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:53.598853 21065 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.607172 21213 raft_consensus.cc:2376] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11. Re-sending same reply.
I20260709 13:07:53.473855 25515 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.419363 25350 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.583180 25496 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.658823 25496 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.600708 25644 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.661269 25363 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:53.662211 25547 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.662094 25363 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.662838 25547 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.666043 25361 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:53.668062 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.576354 25498 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.605736 25495 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.517982 25421 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:07:53.638062 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 517928us
I20260709 13:07:53.671522 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:53.676905 21065 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:53.505502 25374 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.683459 21219 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:53.612391 25386 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.692560 21214 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:53.696812 25648 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 13:07:53.697701 25416 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.001s
I20260709 13:07:53.698184 25416 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:53.698969 25642 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:07:53.702167 25585 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.757s	user 0.004s	sys 0.006s
I20260709 13:07:53.702652 25585 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:53.703459 21794 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:53.703737 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:53.705019 21068 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:53.706643 21068 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:53.708037 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:53.719569 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.720569 21216 raft_consensus.cc:2410] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:53.721467 21213 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:53.614852 25638 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.726285 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:53.726812 21800 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:53.732806 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:53.733705 21066 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
W20260709 13:07:53.758208 21221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.003s	sys 0.009s
W20260709 13:07:53.760742 21221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.004s	sys 0.010s
I20260709 13:07:53.760888 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:53.683146 25498 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.767272 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.727797 25585 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.738044 25136 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:53.772053 21216 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:53.761255 25496 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:53.793865 25416 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:53.795254 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.796200 21216 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
W20260709 13:07:53.797844 25501 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.754s	user 0.008s	sys 0.000s
I20260709 13:07:53.798384 25501 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.778053 25627 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:53.801579 25632 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.753s	user 0.003s	sys 0.007s
I20260709 13:07:53.802187 25632 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.802796 25644 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:53.803552 21220 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:53.804576 25421 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:53.750428 25449 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:07:53.807713 25449 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:53.808110 25449 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:53.808961 25449 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.810320 25449 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:53.709395 25386 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.801486 25495 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:53.777112 25036 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.008s	user 0.013s	sys 0.018s
I20260709 13:07:53.811776 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.820554 21139 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
W20260709 13:07:53.829102 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 68411.4us
I20260709 13:07:53.804458 25136 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.843827 21215 raft_consensus.cc:2368] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:53.784049 25582 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:53.812767 25437 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:53.870496 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:53.871488 21217 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 5: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:53.860349 24965 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.769s	user 0.007s	sys 0.003s
I20260709 13:07:53.780216 25585 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:53.880204 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.880975 21216 raft_consensus.cc:2376] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14. Re-sending same reply.
I20260709 13:07:53.884284 21071 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:53.860086 25582 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:53.877705 25515 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:53.861405 25495 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:07:53.919962 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.925163 21216 raft_consensus.cc:2368] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:53.858810 25136 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.864204 25468 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:07:53.886679 25522 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:53.928161 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.934871 21216 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:53.936969 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:53.885779 24965 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:53.939600 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.939932 25447 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:53.940304 25447 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:53.941098 24965 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:07:53.941212 25447 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.942811 25447 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:53.943400 25136 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:53.932217 25522 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:53.946035 25363 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:53.945935 25522 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:53.946321 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.947620 21138 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:53.940464 21217 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:53.947671 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.949883 21212 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:07:53.950497 21220 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:53.951002 21212 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:53.941020 25660 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:07:53.955129 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:53.955900 21216 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:53.960515 25585 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:53.970461 25182 consensus_meta.cc:287] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.003s
I20260709 13:07:53.924912 25350 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:53.971760 25632 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:53.972267 25498 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:53.917511 25624 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.199s
W20260709 13:07:53.976773 25542 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.104s	user 0.007s	sys 0.004s
W20260709 13:07:53.977562 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.949453 25629 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:53.982609 25357 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.197s	user 0.008s	sys 0.003s
I20260709 13:07:54.036580 21218 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:07:53.984167 25651 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:07:53.985399 25652 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.037722 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.037796 25652 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.038522 21216 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:53.986680 25441 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:53.926431 25437 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:07:53.987792 25522 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:54.042367 25498 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.989022 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 514) took 1169 ms. Trace:
I20260709 13:07:54.042999 25498 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.043040 21798 rpcz_store.cc:276] 0709 13:07:52.819574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.820060 (+   486us) service_pool.cc:224] Handling call
0709 13:07:52.859742 (+ 39682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:53.035965 (+176223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:53.035999 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:53.083240 (+ 47241us) log.cc:844] Serialized 26 byte log entry
0709 13:07:53.199620 (+116380us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:53.199665 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.988594 (+788929us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400700154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:53.988642 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:53.988686 (+    44us) raft_consensus.cc:1685] finished
0709 13:07:53.988699 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:53.988941 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8498,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":6282,"spinlock_wait_cycles":373666560,"thread_start_us":40256,"threads_started":1,"wal-append.queue_time_us":429068}
I20260709 13:07:53.992118 25386 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:53.990540 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2733) took 1050 ms. Trace:
I20260709 13:07:54.009351 21217 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:54.045471 25498 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:54.045310 21214 rpcz_store.cc:276] 0709 13:07:52.940030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.945808 (+  5778us) service_pool.cc:224] Handling call
0709 13:07:53.990343 (+1044535us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b54004349d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:53.990478 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1556911872}
I20260709 13:07:54.001313 21767 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:54.016336 21220 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:07:54.019726 21145 raft_consensus.cc:2368] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10.
I20260709 13:07:54.026369 21069 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:54.047933 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47827us
I20260709 13:07:54.053820 25363 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.054453 25363 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.056002 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:54.056633 21795 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:54.056912 21795 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:54.057638 21795 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.058291 25363 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:54.058593 21795 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:54.060045 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.028510 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 180078us
I20260709 13:07:53.983713 25501 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:54.039876 21067 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:54.042479 25652 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:54.026432 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.063738 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:54.094715 21071 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:54.060791 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:54.062932 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:54.062026 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:54.065716 25437 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:54.066458 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:54.010149 25515 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.096820 25515 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.099725 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 943) took 1223 ms. Trace:
I20260709 13:07:54.100145 21794 rpcz_store.cc:276] 0709 13:07:52.876791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.885181 (+  8390us) service_pool.cc:224] Handling call
0709 13:07:52.885563 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:53.703941 (+818378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.099486 (+395545us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54005401d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:54.099663 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880426240}
I20260709 13:07:54.105293 21216 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:54.007655 25582 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.106423 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:54.069011 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.062348 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:54.107075 21069 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:54.107390 21141 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:07:54.107777 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:54.092762 25136 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.108397 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:54.108667 25136 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.110903 25136 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:54.111078 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.068651 21219 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:54.084012 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:54.086410 21215 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:54.113011 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:54.095976 21797 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:54.108992 21141 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:54.063030 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:54.114745 21140 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:54.128669 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.129395 21063 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:54.129582 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.114990 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:54.048651 25498 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:54.133602 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:54.159083 21072 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
W20260709 13:07:54.160355 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 434108us
I20260709 13:07:54.161554 25515 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:54.164105 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:54.165453 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52527.2us
I20260709 13:07:54.164448 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:54.163951 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:54.166648 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:54.167938 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:54.168926 21140 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:54.171975 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.178539 21144 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:54.181779 21216 raft_consensus.cc:2410] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:07:54.114751 25437 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:54.183822 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:54.185065 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.137840 25665 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:54.188164 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:54.134915 25629 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:54.186822 25495 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:54.135852 25493 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:54.197999 21141 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:54.204384 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:54.206128 21212 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 6.
I20260709 13:07:54.206892 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.207536 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:54.208982 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44615.3us
I20260709 13:07:54.174240 25501 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.210278 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:07:54.210312 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.210796 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51771.5us
I20260709 13:07:54.210908 21138 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:54.208886 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:54.201735 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.212442 21767 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:54.212519 21141 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:54.135092 25184 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:07:54.199654 25665 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.195417 25449 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:54.214597 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:54.217391 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.216398 25495 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:54.200222 25629 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.157143 25586 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:07:54.199375 25498 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:54.213449 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:54.252959 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45503us
I20260709 13:07:54.205513 25585 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:54.159157 25669 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:07:54.206719 25667 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.209041 25615 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.205600 25182 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.254552 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:54.208274 25493 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.267217 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:54.219260 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.218393 25386 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:54.312372 21141 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:07:54.312721 25386 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.312798 21215 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:54.268814 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.313280 25386 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:54.313592 21216 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:07:54.314069 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.314527 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.314880 21215 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:54.315451 21216 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:54.249790 25595 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:54.316342 25595 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.316922 25595 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:54.268422 21212 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:54.279819 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:54.267819 25665 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:54.268945 25361 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.591s	user 0.008s	sys 0.003s
W20260709 13:07:54.285486 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 72139.3us
I20260709 13:07:54.320065 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:54.283912 25585 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.321040 25585 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:54.321455 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:54.322788 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:54.323479 25518 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:54.323937 25518 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.324287 25518 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:54.290416 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2937) took 1110 ms. Trace:
I20260709 13:07:54.293440 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.268770 25449 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.264005 25614 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:07:54.300355 25668 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:07:54.306668 21221 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:54.309508 21138 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:07:54.270831 25671 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:54.307650 25184 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:54.271535 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:54.319186 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.278820 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:54.407786 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.408452 21069 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
W20260709 13:07:54.408557 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 129869us
I20260709 13:07:54.409694 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:54.323387 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:54.216292 25501 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.344033 25493 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:54.326157 25595 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.351560 25636 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:54.353013 25534 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:54.350780 25518 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:54.430095 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.353426 25615 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:54.362572 25670 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:07:54.311118 25667 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.276681 25629 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:54.438550 25595 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:54.439218 25636 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:54.440310 25636 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:54.443645 25671 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:54.444813 25518 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:54.445554 25534 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:54.446645 25534 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.450999 21215 raft_consensus.cc:2368] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15.
W20260709 13:07:54.369478 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.372347 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.452344 21214 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:54.452966 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.453253 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.453892 21214 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:54.454519 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.455605 21068 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:54.455863 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.414582 25184 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.456503 21066 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
W20260709 13:07:54.460579 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.377348 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.379500 25182 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:54.380867 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 176881us
I20260709 13:07:54.381538 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 868) took 1841 ms. Trace:
I20260709 13:07:54.470875 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.469347 21800 rpcz_store.cc:276] 0709 13:07:52.539723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.539844 (+   121us) service_pool.cc:224] Handling call
0709 13:07:54.381333 (+1841489us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002eaed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:54.381459 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2701642496}
I20260709 13:07:54.383966 25638 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:54.384744 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 2942) took 1201 ms. Trace:
I20260709 13:07:54.385241 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:54.478215 21219 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:54.284865 25441 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.479526 25441 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.481925 25441 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:54.301309 25495 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:54.389638 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1439) took 1361 ms. Trace:
I20260709 13:07:54.482658 21218 rpcz_store.cc:276] 0709 13:07:53.028691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.030161 (+  1470us) service_pool.cc:224] Handling call
0709 13:07:54.389412 (+1359251us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b5400040e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:54.389567 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2238142336}
I20260709 13:07:54.483119 25495 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.485828 21218 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:54.486681 21140 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:54.477967 21139 rpcz_store.cc:276] 0709 13:07:53.183369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.196546 (+ 13177us) service_pool.cc:224] Handling call
0709 13:07:54.384619 (+1188073us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540036d654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:54.384696 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422089600}
I20260709 13:07:54.491678 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.493036 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:54.493323 25681 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:07:54.493775 21066 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:54.394204 21795 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:07:54.331481 21145 rpcz_store.cc:276] 0709 13:07:53.179916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.180114 (+   198us) service_pool.cc:224] Handling call
0709 13:07:54.290146 (+1110032us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b54002b43d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:54.290317 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927972224}
W20260709 13:07:54.402875 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 133939us
I20260709 13:07:54.406092 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:54.408702 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.498322 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.415562 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:54.500523 21145 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:54.423365 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:54.403832 25449 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:54.419732 25395 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:54.426980 25677 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:54.427111 25447 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:54.427560 25645 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:54.429097 25501 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:54.437529 25514 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:54.454984 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.372241 25498 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.456568 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.573084 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:54.573841 21068 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:54.455892 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.457711 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.374440 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:54.461694 21217 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:54.487145 21218 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:54.500893 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:54.501637 21221 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 5.
I20260709 13:07:54.502419 21147 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:54.576534 21138 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:54.503054 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 87574.6us
W20260709 13:07:54.502611 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.504693 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.579510 21215 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:54.467638 25184 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:54.523339 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:54.525350 21765 leader_election.cc:336] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.672s (SENT)
I20260709 13:07:54.520252 25395 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:54.498215 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3205033560000174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 13:07:54.586933 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3106033560000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 13:07:54.587316 21036 connection.cc:256] RPC call timeout handler was delayed by -1.209782333000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 13:07:54.587584 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2034823330000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
I20260709 13:07:54.533396 21213 raft_consensus.cc:2376] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11. Re-sending same reply.
I20260709 13:07:54.588577 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:54.589200 21069 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:54.539152 25574 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.544615 21219 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:07:54.545578 25585 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.545365 25437 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.547472 25386 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.550638 25629 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.826761 25629 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:54.827507 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.543027 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:54.829520 21064 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:54.830536 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:54.831094 21064 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:54.836402 25547 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:54.519758 25638 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:54.837196 25638 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.837605 25638 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:54.581444 21065 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:54.843559 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.844362 21065 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:54.852928 25386 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:54.582259 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:54.503756 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:54.855755 21066 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:54.856619 21068 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:07:54.583642 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:54.584821 21187 connection.cc:256] RPC call timeout handler was delayed by -1.02226538299999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 13:07:54.583559 21139 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
W20260709 13:07:54.586104 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 211765us
W20260709 13:07:54.855931 25665 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.001s
W20260709 13:07:54.679122 25649 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.095s	user 0.003s	sys 0.001s
W20260709 13:07:54.702802 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.883836 21140 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:54.881361 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:54.885687 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:54.886782 25629 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.887527 25629 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.883364 25665 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:54.893045 25665 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:54.895390 25386 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:54.890398 25629 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:54.898216 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.898926 25645 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.899477 25645 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:54.900923 25449 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.901633 21147 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:54.898936 21213 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:54.906622 25184 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.908993 25184 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:54.914165 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1380) took 1780 ms. Trace:
I20260709 13:07:54.914636 21070 rpcz_store.cc:276] 0709 13:07:53.134054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.134248 (+   194us) service_pool.cc:224] Handling call
0709 13:07:53.617185 (+482937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:53.617723 (+   538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.913926 (+1296203us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54003ca350. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:54.914096 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109951744}
I20260709 13:07:54.919656 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:54.920457 21800 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:54.921216 25493 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.921648 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:54.922567 21794 raft_consensus.cc:2410] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:07:54.924536 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:54.919027 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:54.928313 21797 raft_consensus.cc:2368] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7.
I20260709 13:07:54.929080 25665 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:54.929453 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:54.929595 25665 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:54.930040 21798 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:54.930878 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 405531us
W20260709 13:07:54.931447 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.932061 25503 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:54.933233 25503 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:54.933394 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:54.933503 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:54.934307 25266 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:54.934669 25266 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:54.935031 25266 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:54.939805 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:54.940565 25136 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:54.940860 21795 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:54.941057 25136 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.941490 25136 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:54.942499 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:54.930610 25665 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:54.943570 21796 raft_consensus.cc:2410] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:07:54.944535 25504 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:54.949213 21798 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:54.950574 25449 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:54.928896 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:54.951187 25493 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:54.952445 21066 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:54.952677 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:54.954284 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:54.676257 21217 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:07:54.679168 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 244576us
I20260709 13:07:54.660657 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:54.960943 21216 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:54.960783 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:54.727335 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:54.709126 25652 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.750470 25615 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:54.660247 25673 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:07:54.592584 25515 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:07:54.685791 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.589483 25395 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:54.772771 21035 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:54.824385 25667 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:54.842120 25182 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:54.867431 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:07:54.954977 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.955835 21069 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:07:54.959458 21064 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:54.753077 25585 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:54.961740 21071 raft_consensus.cc:2376] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12. Re-sending same reply.
I20260709 13:07:54.806326 21138 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:07:54.666908 25574 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:54.962076 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 103567us
I20260709 13:07:54.951917 25363 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:54.964015 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:54.991768 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:54.992606 21801 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:07:54.993083 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.959117 25518 raft_consensus.cc:2764] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:07:54.966361 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:54.964362 21796 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:54.952757 25501 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:54.980731 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:54.984637 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 211928us
I20260709 13:07:54.985986 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:54.965029 25632 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:54.968603 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 314442us
W20260709 13:07:55.004859 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1854823330000954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 13:07:55.005273 21036 connection.cc:256] RPC call timeout handler was delayed by -1.174682333000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 13:07:55.005591 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1719823330000736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 13:07:55.005882 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1476823330000343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 13:07:55.012929 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:55.013830 21140 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:07:55.013876 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:55.016902 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:54.991976 25363 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.020546 25363 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:55.002430 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:55.028328 21767 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:55.029297 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:55.030304 21071 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:55.031422 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:55.032162 21064 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:55.034183 21065 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:55.035807 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.042466 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:55.043102 21138 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:55.044643 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:55.038174 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:55.046059 21147 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:55.048497 21800 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:55.051358 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0837823330000447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
W20260709 13:07:55.051921 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0693823330000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
I20260709 13:07:55.053807 21216 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:55.004997 25624 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:55.027233 25585 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:55.005170 25693 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:54.976366 25652 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.020680 25184 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:07:55.079605 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.048969 25656 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:55.065953 21064 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:54.985291 25266 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.026393 25493 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:07:55.064303 25518 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:55.064872 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:55.098815 21801 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:55.070981 25501 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:55.086967 25693 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:55.092521 25449 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:55.107192 25518 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.131765 25518 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:55.134763 25498 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:55.135653 25498 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:55.136874 25498 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.071034 25632 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:55.165552 25632 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:55.166525 25632 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:55.138245 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.006227 25645 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.146944 25518 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.172096 25518 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:55.175633 25645 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:55.158524 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 291210us
W20260709 13:07:55.178455 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.180457 25518 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:55.181840 25705 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:55.182322 25705 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:55.159672 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 164446us
W20260709 13:07:55.161950 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 145170us
I20260709 13:07:55.194410 21138 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:07:55.194713 25656 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:55.195233 25585 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:55.199096 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 145963us
I20260709 13:07:55.183449 25705 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.209367 25705 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:55.212438 25495 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:55.213079 25495 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.215540 25495 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.216060 25693 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.218874 25416 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:55.121802 25361 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:55.226346 21140 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
W20260709 13:07:55.201480 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 162310us
W20260709 13:07:55.203405 24965 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.002s	sys 0.023s
I20260709 13:07:55.197127 25656 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.205958 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:55.208562 21765 leader_election.cc:336] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.988s (SENT)
I20260709 13:07:55.257875 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.228966 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.260022 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.261291 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:55.262267 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 271836us
I20260709 13:07:55.263134 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:55.247334 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:55.243693 24965 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.002s	sys 0.024s
I20260709 13:07:55.226099 25416 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:55.269977 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64131.1us
I20260709 13:07:55.275750 25624 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:55.277104 25624 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.205801 25632 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 13:07:55.286571 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 78034.8us
I20260709 13:07:55.244627 25495 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:07:55.290525 25632 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:55.290983 25632 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:55.297009 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.299813 25574 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.269129 25669 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:55.310421 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47395.2us
I20260709 13:07:55.239740 25449 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:07:55.312777 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:55.280916 25184 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:55.268183 25722 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.305910 25693 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:55.319005 25669 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.363826 25669 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:55.319677 25638 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.374642 25638 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:55.377848 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60907.4us
I20260709 13:07:55.378491 25184 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:55.378778 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.379781 25705 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:55.380275 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:55.379913 25184 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.381160 25705 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:55.381651 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.382519 25705 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:55.323073 25574 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:55.319401 21795 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.283075 25645 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:55.323966 21147 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:55.326875 25582 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.328300 21793 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEARNER]: Advancing to term 10
W20260709 13:07:55.328744 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.332849 21143 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:55.337170 25416 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.338064 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:55.335611 25363 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.339013 25632 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:55.343580 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46640.7us
I20260709 13:07:55.408320 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.409689 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:55.410445 21147 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:55.351686 25585 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.352922 25652 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:55.284261 25706 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:55.354993 25266 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.286860 25687 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:55.403828 25036 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.420205 25036 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.405889 21212 raft_consensus.cc:2410] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:07:55.410611 21139 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:07:55.411689 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.414757 25447 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:55.401927 25727 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
W20260709 13:07:55.401793 25717 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 13:07:55.426492 21142 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:55.427879 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.429422 25363 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.421940 25687 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:55.421083 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 136621us
I20260709 13:07:55.423089 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.424445 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1456) took 1220 ms. Trace:
I20260709 13:07:55.424929 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:55.458037 21212 rpcz_store.cc:276] 0709 13:07:54.204528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.319769 (+115241us) service_pool.cc:224] Handling call
0709 13:07:55.337889 (+1018120us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:07:55.424359 (+ 86470us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53802752}
I20260709 13:07:55.459998 21141 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
W20260709 13:07:55.439258 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.458801 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.468592 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.473466 21792 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:55.474095 21793 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:55.422290 25449 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:55.480535 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:55.479017 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56012.8us
W20260709 13:07:55.482889 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 55099.5us
I20260709 13:07:55.483891 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.484193 25678 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:55.485108 25515 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:55.485374 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.486490 25515 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:55.488301 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:55.488976 21063 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.490196 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:55.492233 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:55.492579 25678 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:55.493642 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.433096 25687 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:55.433271 25705 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:55.500820 21214 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:55.434530 25706 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:55.508860 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1451) took 1373 ms. Trace:
I20260709 13:07:55.512827 21214 rpcz_store.cc:276] 0709 13:07:54.135599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.500160 (+1364561us) service_pool.cc:224] Handling call
0709 13:07:55.500598 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.508602 (+  8004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.508803 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:55.504035 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 515) took 1299 ms. Trace:
I20260709 13:07:55.506870 21146 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:55.486781 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.515570 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.507862 21795 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:55.468359 25629 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.518088 25515 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:55.518812 25515 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:55.518654 25629 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.519399 25515 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.520926 25629 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.472232 25632 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:55.523592 25705 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:55.525274 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2930) took 2364 ms. Trace:
I20260709 13:07:55.525676 21142 rpcz_store.cc:276] 0709 13:07:53.160680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.160828 (+   148us) service_pool.cc:224] Handling call
0709 13:07:53.192055 (+ 31227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.434235 (+2242180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.524991 (+ 90756us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b540042f754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:55.525186 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5138190080}
I20260709 13:07:55.526839 21064 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:55.528894 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:55.529968 25731 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:07:55.513792 21797 rpcz_store.cc:276] 0709 13:07:54.204978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.205393 (+   415us) service_pool.cc:224] Handling call
0709 13:07:55.503769 (+1298376us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54003bbd54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:55.503955 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507822336}
I20260709 13:07:55.472394 25518 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:55.495594 25645 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:07:55.529856 25711 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:55.521564 25395 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.002s
I20260709 13:07:55.530930 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:55.480093 25677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.009s	sys 0.024s
I20260709 13:07:55.166057 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:55.510296 25136 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.000s
I20260709 13:07:55.498967 25437 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.433938 25447 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.536037 21063 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:55.540848 25395 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.541987 25636 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:55.542532 25677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.009s	sys 0.025s
I20260709 13:07:55.540398 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.552171 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2242) took 1090 ms. Trace:
I20260709 13:07:55.527179 25722 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:55.575106 21065 rpcz_store.cc:276] 0709 13:07:54.461618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.844925 (+383307us) service_pool.cc:224] Handling call
0709 13:07:54.845227 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.034610 (+189383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.551854 (+517244us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54000fb254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:55.552102 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434151808}
I20260709 13:07:55.535625 25728 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:55.580855 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.583500 21144 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:55.645699 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2491) took 1144 ms. Trace:
W20260709 13:07:55.641157 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61895.5us
W20260709 13:07:55.622279 25730 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.194s	user 0.001s	sys 0.001s
I20260709 13:07:55.615592 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1445) took 1543 ms. Trace:
I20260709 13:07:55.653090 25361 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:55.640432 25728 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:55.652899 21220 rpcz_store.cc:276] 0709 13:07:54.072284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.072429 (+   145us) service_pool.cc:224] Handling call
0709 13:07:54.072769 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:54.100107 (+ 27338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:54.100136 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:54.100633 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:07:54.217963 (+117330us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:54.217992 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.615139 (+1397147us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b540034b2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:55.615191 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:55.615241 (+    50us) raft_consensus.cc:1685] finished
0709 13:07:55.615252 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:55.615528 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3149,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":61984000,"thread_start_us":116935,"threads_started":1,"wal-append.queue_time_us":328673}
I20260709 13:07:55.654349 25728 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:55.555542 25136 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.577415 25677 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:55.585026 25669 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.585840 25711 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:55.556077 25629 raft_consensus.cc:2764] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:07:55.587713 25671 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:55.597283 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 625151us
I20260709 13:07:55.644049 25693 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:55.676718 21063 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:07:55.676741 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:55.677647 21069 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:55.678359 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:55.678776 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:55.646060 21144 rpcz_store.cc:276] 0709 13:07:54.500835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.501462 (+   627us) service_pool.cc:224] Handling call
0709 13:07:54.501903 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.643564 (+1141661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.645454 (+  1890us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540030b454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:55.645640 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2486953344}
I20260709 13:07:55.583099 25395 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.672685 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.641357 25632 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:55.637836 25416 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:55.669744 25678 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:55.615239 25574 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:07:55.605166 25733 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:07:55.639568 25447 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.677016 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:55.607758 25706 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.663724 25705 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.675829 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:55.680490 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:55.682214 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.700395 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:55.688807 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.702558 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:55.694027 21069 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:07:55.695784 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:55.703694 21064 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.696159 21065 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:07:55.704164 21065 pending_rounds.cc:85] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 5
I20260709 13:07:55.704380 21065 pending_rounds.cc:107] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:07:55.704588 21065 raft_consensus.cc:2889] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:55.702323 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:55.705766 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:55.719667 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:55.721585 25363 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:55.722167 25363 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.722555 25363 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:55.693110 25518 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:55.723778 25518 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.724191 25518 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:55.727571 25582 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:55.727821 25677 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.675009 25711 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:07:55.731657 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.732620 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 239050us
I20260709 13:07:55.706355 25395 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:55.736613 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 218755us
I20260709 13:07:55.738409 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:55.684692 25614 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:07:55.741374 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:55.739234 21065 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:55.740563 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:55.710207 25534 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:55.734789 25636 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:55.717752 25632 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:55.742594 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:55.679814 25671 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.744829 21070 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:55.748054 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.739673 25582 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:55.736758 25395 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:55.710479 24965 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:07:55.744174 25609 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.778198 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:55.707353 25705 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.730873 25504 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:07:55.769788 21064 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:55.767879 25522 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:55.741618 25656 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:07:55.789347 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:55.790098 21064 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:55.778863 21065 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:55.791772 25678 raft_consensus.cc:2764] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:07:55.791811 25514 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.794513 25514 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.744315 25449 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:55.796031 25449 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.796903 25514 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.766506 25503 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:55.799642 25503 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:55.799939 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65862.3us
I20260709 13:07:55.800793 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.800659 25503 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.802093 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:55.803071 25735 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:55.803540 25735 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:55.803972 25735 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:55.805444 25136 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.806432 25705 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:55.807333 25363 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.809518 25363 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.776590 25395 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:55.740293 25645 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:55.815693 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 113735us
I20260709 13:07:55.823850 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:55.819043 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:55.833321 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:55.838035 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:55.839028 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.832165 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:55.791929 25447 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.787556 25522 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:55.845768 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:55.843792 21065 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:55.849022 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:55.788815 25582 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:55.811151 25518 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.865410 21220 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:55.870647 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:55.866957 21218 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:55.871372 21217 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:55.872025 21221 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:55.874440 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:55.844259 25705 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:07:55.876052 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 92099.7us
W20260709 13:07:55.878649 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.880260 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2943) took 2666 ms. Trace:
I20260709 13:07:55.880697 21143 rpcz_store.cc:276] 0709 13:07:53.214320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.223210 (+  8890us) service_pool.cc:224] Handling call
0709 13:07:53.223641 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.403618 (+2179977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.880016 (+476398us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b540027a454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:55.880194 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4849710976}
I20260709 13:07:55.884687 21795 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:55.866671 25669 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.869560 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:55.886443 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:55.865784 25614 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:55.864161 25740 raft_consensus.cc:2764] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:07:55.787595 25609 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.904124 21795 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:55.841007 25645 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.864343 25515 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.924189 25036 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:55.867599 25447 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.900430 25534 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:55.905406 25363 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:55.905184 25504 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.900978 25705 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:55.905339 25737 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:55.934233 21065 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:55.938771 21065 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:55.940325 21216 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:55.939904 21065 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.941569 21065 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:55.917634 25678 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:55.905867 25624 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:07:55.944931 21220 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:55.947455 25735 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.947984 25745 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:55.962378 25735 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.956054 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:55.943989 25728 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.921692 25449 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:07:55.948740 25744 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:55.952306 25687 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.966576 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:55.967465 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:55.967957 25687 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.968696 25136 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:55.969261 25136 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.955698 24965 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:55.969730 25728 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:55.970592 25652 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:55.969738 25136 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:55.863967 25711 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.971972 21071 raft_consensus.cc:2410] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:07:55.952840 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.970158 24965 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:55.972961 24965 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:55.973809 24965 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.969096 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:55.971031 25652 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:55.975142 25582 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:55.964074 25745 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:55.976400 25652 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:55.956387 25514 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:55.977918 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:55.941120 25363 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.978789 25624 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:55.979120 25363 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:55.979378 25624 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.979815 25624 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:55.983212 25609 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:55.978582 21220 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:56.010403 21797 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:55.909444 25614 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.018898 25136 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.021132 25447 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:55.914223 25522 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:55.941306 25504 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:56.060937 21220 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
W20260709 13:07:56.032939 25741 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
W20260709 13:07:56.038664 25725 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.618s	user 0.003s	sys 0.004s
I20260709 13:07:56.033519 25652 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:56.077517 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.099331 21220 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:56.103353 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:56.123983 25742 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:07:56.090533 25522 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:56.135200 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.138917 25732 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:56.139549 25732 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:56.140617 25732 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.142234 25732 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:56.144538 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.145561 21215 raft_consensus.cc:2410] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:07:56.150434 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:56.151502 21068 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:56.152943 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:56.154320 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:56.155993 25749 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:07:56.155903 21066 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:56.146994 21215 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:07:56.155097 21068 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:07:56.170617 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 280334us
I20260709 13:07:56.100148 25518 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:56.086023 25705 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.177732 25514 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.213814 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:56.218135 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:56.218894 21070 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:56.171309 25693 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:07:56.230176 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 353492us
I20260709 13:07:56.263226 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:56.264336 21063 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
W20260709 13:07:56.266014 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.266718 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:56.217567 25645 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:56.289942 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.300843 21218 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:07:56.313104 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.314043 21218 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
W20260709 13:07:56.325944 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.283100 25749 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:07:56.217196 25514 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:56.336323 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:56.338783 21220 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:56.340445 21068 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:56.342046 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:56.320782 25645 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.342330 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:56.327873 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 1.13876e+06us
I20260709 13:07:56.343403 21065 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:56.345964 25624 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:56.346563 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.355978 21140 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:56.356256 21217 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:56.351979 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:56.354754 21220 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:07:56.371388 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:56.367830 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:56.372756 21144 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:56.374462 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.375097 21220 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:56.377395 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:56.378965 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:56.380223 21068 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:56.380955 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.380687 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:56.340562 25614 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:56.343397 25504 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:56.382927 25441 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:56.385419 25671 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:56.386528 25036 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:56.387248 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:56.393247 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.397404 25755 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:07:56.400068 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.403156 21144 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
W20260709 13:07:56.404368 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.012s	sys 0.021s
I20260709 13:07:56.381299 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:56.413702 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.012s	sys 0.021s
I20260709 13:07:56.414068 21142 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:56.413098 21214 raft_consensus.cc:2376] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14. Re-sending same reply.
I20260709 13:07:56.421904 21220 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:56.430568 21144 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:07:56.431811 21140 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:56.365374 25645 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:56.418393 25614 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:56.437817 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 592070us
I20260709 13:07:56.444199 21140 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:56.446894 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:56.447976 21141 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:56.441444 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.450699 21214 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:56.451762 21142 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:07:56.450196 21217 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:56.453049 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.456614 21217 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:56.466362 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.467938 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.468748 21217 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:56.469581 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.470185 21217 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:07:56.471686 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.472630 21214 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:56.473528 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.474079 21214 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:56.448138 21140 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:56.495697 21142 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:56.498106 21036 leader_election.cc:400] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:56.471102 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.504434 21217 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:56.508122 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.509483 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.510102 21219 raft_consensus.cc:2376] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14. Re-sending same reply.
I20260709 13:07:56.510557 25503 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.511199 25503 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.513777 25503 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:56.516337 21217 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
W20260709 13:07:56.518177 25751 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.499s	user 0.003s	sys 0.000s
I20260709 13:07:56.519408 25136 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:56.520534 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44272.7us
I20260709 13:07:56.521390 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:56.522730 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:56.523917 25763 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:56.525432 25624 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:56.525933 25754 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:56.530400 25363 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.003s
I20260709 13:07:56.530865 25363 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:56.532617 25758 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:56.533035 25363 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:56.534853 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:56.537199 25758 raft_consensus.cc:2764] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:07:56.539774 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:56.542438 25504 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.542755 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.559250 21143 raft_consensus.cc:2410] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:07:56.543063 25504 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.548146 21036 leader_election.cc:400] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:56.577435 25504 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:56.543475 25758 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:56.578215 25503 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:56.543942 25514 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:56.553851 21146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2479) took 2492 ms (client timeout 1725 ms). Trace:
W20260709 13:07:56.549376 25395 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.709s	user 0.010s	sys 0.000s
W20260709 13:07:56.579434 21146 rpcz_store.cc:269] 0709 13:07:54.061677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.061806 (+   129us) service_pool.cc:224] Handling call
0709 13:07:56.553626 (+2491820us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400279554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:56.553738 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3244170112}
I20260709 13:07:56.579972 25395 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:56.577603 21036 leader_election.cc:403] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:07:56.552925 25718 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.582856 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:56.568099 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.572165 21067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2234) took 2118 ms (client timeout 1646 ms). Trace:
W20260709 13:07:56.588181 21067 rpcz_store.cc:269] 0709 13:07:54.453231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.454593 (+  1362us) service_pool.cc:224] Handling call
0709 13:07:56.571816 (+2117223us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54003c9e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:56.572036 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3432097536}
I20260709 13:07:56.591308 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:56.592235 21214 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 7.
I20260709 13:07:56.582903 25718 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.614614 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:56.614149 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:56.613562 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 206257us
I20260709 13:07:56.548012 21220 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:07:56.564815 25747 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.548s	user 0.001s	sys 0.000s
I20260709 13:07:56.621874 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:56.566757 25752 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 13:07:56.609758 21070 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:56.627259 21185 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:56.582583 25395 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:56.545460 25645 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:56.622816 21142 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:56.593650 25758 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:56.633503 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:56.634640 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 51856.6us
I20260709 13:07:56.583777 25732 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:56.639909 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:56.642334 21067 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:07:56.626008 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:56.643911 21070 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:07:56.649369 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:56.647467 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:56.627956 25514 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.630636 25363 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:56.631160 25746 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:56.652137 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:56.657840 25624 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:56.675257 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.675613 21070 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:56.623826 25762 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:07:56.648845 25645 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:56.631481 25636 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:07:56.656603 25504 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:56.621176 25503 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.678740 25395 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:56.692492 25363 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:56.690197 25732 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:56.730509 21070 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:07:56.697960 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.698628 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:56.715114 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.678682 25624 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.690807 25645 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:56.760210 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 110960us
I20260709 13:07:56.736042 25759 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:56.674268 25514 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.737825 25732 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:56.768386 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.769186 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.702991 25754 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:56.769706 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1404) took 1092 ms. Trace:
I20260709 13:07:56.737237 25504 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:56.771535 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:56.739318 25395 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:56.772686 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.787087 21214 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:56.743726 25624 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:56.750000 25645 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:56.776506 21072 rpcz_store.cc:276] 0709 13:07:55.677661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.677793 (+   132us) service_pool.cc:224] Handling call
0709 13:07:55.697076 (+ 19283us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:55.703859 (+  6783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:55.703906 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:55.704631 (+   725us) log.cc:844] Serialized 52 byte log entry
0709 13:07:55.902628 (+197997us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:55.902781 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.903045 (+   264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:56.563994 (+660949us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:07:56.761936 (+197942us) raft_consensus.cc:1685] finished
0709 13:07:56.761960 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:56.769610 (+  7650us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5402,"raft.queue_time_us":69657,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":8486,"spinlock_wait_cycles":51225472,"thread_start_us":174609,"threads_started":1,"wal-append.queue_time_us":289388}
I20260709 13:07:56.803028 25629 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:56.803612 25629 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:56.804081 25629 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:56.806730 25770 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.328s	user 0.002s	sys 0.005s
I20260709 13:07:56.775460 25732 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:56.831619 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 70080.2us
W20260709 13:07:56.860632 21036 leader_election.cc:336] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.531s (SENT)
I20260709 13:07:56.861022 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:56.774190 25759 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:56.885874 21036 connection.cc:256] RPC call timeout handler was delayed by -1.9111536950000527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31800000000000006s for a response
W20260709 13:07:56.886314 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8588477870000588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.215s for a response
I20260709 13:07:56.887490 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:56.888384 25718 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:56.888577 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:56.888989 25718 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:56.889456 25718 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:56.891736 25718 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:56.894594 21214 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:56.896499 25514 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:56.897635 25395 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:56.898308 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.898906 21214 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:56.911607 21214 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:56.914543 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:56.915264 21214 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:56.916150 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:56.917366 25515 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:56.797919 25363 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:56.920325 25758 raft_consensus.cc:2764] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:07:56.922145 25624 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:56.923123 25503 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:56.925339 25624 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:56.907586 25746 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.927908 25746 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:56.930855 25746 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:56.934814 21214 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:56.966545 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:56.971253 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:56.938133 25624 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:56.979828 21214 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:07:56.907964 25504 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:57.004725 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 151455us
I20260709 13:07:57.004760 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:57.020393 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 132968us
I20260709 13:07:56.996730 25624 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.028465 25788 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:57.029312 25788 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:57.029673 25788 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:57.030014 25788 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:57.033684 25708 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.034335 25708 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.038209 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:57.038985 21218 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:57.047262 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:57.017280 25754 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:57.048246 25503 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:57.048856 25503 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:57.049058 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:57.050556 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:57.051373 21212 raft_consensus.cc:1217] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 12.7->[14.8-14.8]   Dedup: 14.8->[]
I20260709 13:07:57.052747 21212 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:07:57.053879 21143 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:57.054889 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:57.055029 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:57.056658 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:57.056743 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:57.057750 21145 raft_consensus.cc:2410] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:07:57.058621 21139 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:57.060089 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:57.066170 21218 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:07:57.068459 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:57.067700 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:57.070930 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:57.071820 21218 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 9: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:57.072680 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:57.123378 21141 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:57.072609 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:57.123582 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:57.124193 21147 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:57.049832 25503 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.074170 25754 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:57.083042 25758 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:07:57.083853 25784 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:07:57.088224 21138 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:57.100461 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:57.134716 21139 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:57.100970 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:57.102066 25783 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:07:57.101970 21145 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:07:57.125763 25706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.108s	user 0.037s	sys 0.026s
I20260709 13:07:57.105057 25624 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:57.106031 25759 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:57.107985 25708 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:57.119014 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:57.120602 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:57.128311 21147 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:57.157994 21143 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:57.158615 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.159404 21068 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:57.135888 21218 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:07:57.102670 24965 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.110972 25645 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:57.140663 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 134715us
I20260709 13:07:57.141435 21141 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
W20260709 13:07:57.142915 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 138271us
I20260709 13:07:57.178167 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:57.146747 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.176715 21767 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:57.179570 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.175125 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:57.150908 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:57.154844 21070 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:57.155469 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.183863 21069 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:57.155789 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.184788 21071 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:57.129846 25754 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:57.189271 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:57.189855 21219 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:57.176918 21068 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:57.185899 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.190804 21065 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:57.191411 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:57.192581 21218 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:07:57.157766 25706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.038s	sys 0.026s
I20260709 13:07:57.201110 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.206259 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:57.206326 21069 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:07:57.211354 21069 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:57.209813 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:57.194542 25754 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:57.212693 21217 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:57.194428 25708 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:57.196244 25785 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:57.215811 21218 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:57.215986 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:57.163714 25732 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.225329 21218 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:07:57.219269 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 133091us
I20260709 13:07:57.231065 21219 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:07:57.235386 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.236258 21072 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:57.185168 24965 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.200464 25624 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:07:57.243093 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:57.244688 25036 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:57.245208 25036 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:57.225080 25785 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.246003 25036 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:57.245240 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67142.8us
I20260709 13:07:57.253417 21221 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:57.254089 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:57.259600 25799 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:57.262036 25788 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:57.264168 25791 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:07:57.265242 25732 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:57.266167 25447 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.266801 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 881) took 1615 ms. Trace:
I20260709 13:07:57.266711 25447 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.267161 21793 rpcz_store.cc:276] 0709 13:07:55.651698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.651922 (+   224us) service_pool.cc:224] Handling call
0709 13:07:55.667708 (+ 15786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:55.668083 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:55.683774 (+ 15691us) log.cc:844] Serialized 14 byte log entry
0709 13:07:56.839792 (+1156018us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b18003240a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:07:57.194728 (+354936us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:57.195197 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:07:57.195386 (+   189us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:57.195419 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.195486 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:57.266520 (+ 71034us) raft_consensus.cc:1685] finished
0709 13:07:57.266609 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.266745 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15510,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":1389,"raft.queue_time_us":39529,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":4068,"spinlock_wait_cycles":6462720,"thread_start_us":348728,"threads_started":2,"wal-append.queue_time_us":1340525}
I20260709 13:07:57.268929 25447 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:57.266177 25800 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:07:57.247474 25708 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:57.255540 25785 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:57.270784 25800 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:57.247494 25629 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.278551 25629 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:57.281973 25708 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.282562 25708 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.286418 21071 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:57.290331 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:57.298682 21072 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:57.306556 21221 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:07:57.312288 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:57.313552 21141 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:57.312722 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:57.314574 21143 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:07:57.314728 21221 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:07:57.312922 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:57.319981 21221 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:57.324707 21220 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:07:57.329772 21220 pending_rounds.cc:85] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 4
I20260709 13:07:57.330107 21220 pending_rounds.cc:107] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:07:57.330408 21220 raft_consensus.cc:2889] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:57.333284 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:57.358114 21221 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:57.358817 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:57.360357 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:57.361873 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 130442us
I20260709 13:07:57.364586 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2767) took 1026 ms. Trace:
I20260709 13:07:57.365114 21212 rpcz_store.cc:276] 0709 13:07:56.338056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.576309 (+238253us) service_pool.cc:224] Handling call
0709 13:07:57.051177 (+474868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:57.053082 (+  1905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.364349 (+311267us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54004434d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:57.364513 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091145600}
I20260709 13:07:57.363863 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:57.369593 21072 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:07:57.370131 21072 pending_rounds.cc:85] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 6
I20260709 13:07:57.370569 21072 pending_rounds.cc:107] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:07:57.370162 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:57.370923 21072 raft_consensus.cc:2889] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:57.284729 25801 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 13:07:57.378064 21143 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:57.379187 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:57.379748 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 182660us
I20260709 13:07:57.382736 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.385098 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:57.363696 25708 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:57.386065 21141 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:57.386791 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:57.393723 21072 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:57.395766 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.413211 21068 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:57.414171 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1469) took 1578 ms. Trace:
I20260709 13:07:57.415782 21216 rpcz_store.cc:276] 0709 13:07:55.835751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.848966 (+ 13215us) service_pool.cc:224] Handling call
0709 13:07:55.874590 (+ 25624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.940812 (+ 66222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.413798 (+1472986us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b540013bd54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:57.414083 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150738688}
I20260709 13:07:57.417742 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:57.419661 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:57.379941 25758 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:57.378122 25629 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:57.433760 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.434422 21069 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:57.435272 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:57.437206 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:57.438371 25793 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:57.438778 25793 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:57.439824 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:57.439675 25793 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.440678 25793 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:57.444073 25711 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:57.444496 25711 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:57.445318 25711 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.446581 25711 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:57.438783 21797 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:57.463817 21065 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:07:57.443123 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 590) took 1126 ms. Trace:
I20260709 13:07:57.480180 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:07:57.484587 21792 raft_consensus.cc:2368] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7.
I20260709 13:07:57.485849 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:57.483984 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:57.488791 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:57.427591 25737 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:57.501470 21069 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:57.476501 21215 rpcz_store.cc:276] 0709 13:07:56.317201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.541750 (+224549us) service_pool.cc:224] Handling call
0709 13:07:56.542137 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:56.542551 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:56.542579 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:56.543059 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 13:07:56.939586 (+396527us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:57.031809 (+ 92223us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.442503 (+410694us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b18003783a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287 00007f7f42537208
0709 13:07:57.442695 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.442769 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:57.442813 (+    44us) raft_consensus.cc:1685] finished
0709 13:07:57.442922 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.443062 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142171,"prepare.queue_time_us":96558,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":431630,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":22481,"spinlock_wait_cycles":82192640,"thread_start_us":348099,"threads_started":1,"wal-append.queue_time_us":525721}
W20260709 13:07:57.505704 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 172535us
I20260709 13:07:57.506877 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:57.508877 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:57.509892 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:57.491389 25758 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:57.510802 21794 raft_consensus.cc:2410] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:07:57.510684 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:57.511130 25736 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.511723 25736 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.449157 25793 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:57.434572 25645 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:57.512290 25374 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:57.515520 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:57.525717 21141 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:57.526535 21798 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:57.529435 25518 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:57.529963 25518 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.530321 25518 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:57.533987 21070 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:07:57.535207 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.521751 25744 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:57.537742 25744 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.538746 25518 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.540023 25744 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:57.519733 25515 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:57.541283 25798 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:07:57.518240 25652 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:57.542682 25652 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.543066 25652 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:57.547959 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:57.549085 25518 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:57.550473 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:57.508253 25395 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.551615 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.552417 21072 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:57.552054 25395 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.552775 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.553251 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.553843 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.554287 25395 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:57.554345 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:57.554854 25796 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.331s	user 0.001s	sys 0.001s
W20260709 13:07:57.551232 25804 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:07:57.555080 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:57.555694 25797 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
W20260709 13:07:57.556758 21066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1403) took 1880 ms (client timeout 1500 ms). Trace:
I20260709 13:07:57.557176 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.551940 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.488325 25504 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.558228 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:57.557171 21066 rpcz_store.cc:269] 0709 13:07:55.676677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.676800 (+   123us) service_pool.cc:224] Handling call
0709 13:07:57.556552 (+1879752us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b54000fad54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:07:57.556652 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058422144}
I20260709 13:07:57.512967 25624 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:57.554600 21064 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:57.555131 21072 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:57.561653 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:57.563328 21066 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:57.563990 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:57.513993 21069 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:57.568511 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.520319 25756 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:57.569840 21066 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:07:57.573086 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.573951 21066 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:57.578701 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.579581 21066 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:07:57.573261 25518 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.581817 25518 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.574602 21069 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:07:57.512917 25809 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:07:57.587798 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 168241us
I20260709 13:07:57.594596 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:57.590258 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:57.591373 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62451.4us
I20260709 13:07:57.561739 25799 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:57.603533 21793 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:57.604261 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:57.605558 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:57.561338 25374 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:57.608522 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:57.609318 21141 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:57.609834 21794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:07:57.616531 21794 raft_consensus.cc:606] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:57.616894 21794 raft_consensus.cc:614] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:57.617060 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:57.587240 25708 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.612222 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.591120 25744 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.623485 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:57.625020 21220 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:57.635890 21214 raft_consensus.cc:2376] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6. Re-sending same reply.
I20260709 13:07:57.626150 21072 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:57.586630 25788 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:57.637910 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:57.603518 25645 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:57.639191 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:57.639873 21216 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:57.639631 25645 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.640479 25812 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:57.640966 25812 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.641372 25812 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:57.641912 25645 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:57.615314 25737 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:57.642982 25737 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:57.644088 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:57.644690 25737 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:57.645102 21215 raft_consensus.cc:2368] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:57.645578 25737 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:57.592391 25624 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:57.646667 25624 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.628367 25708 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.565289 25789 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:07:57.570995 25503 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:07:57.560400 25811 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
W20260709 13:07:57.650537 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 47839.6us
I20260709 13:07:57.673703 21220 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:57.650647 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 45326.3us
I20260709 13:07:57.560392 25504 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.679777 21221 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:07:57.680233 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:57.683853 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.600970 25787 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.652724 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:57.564006 25793 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:07:57.660933 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:57.650336 25762 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.694444 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:57.679750 25747 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:57.669684 25788 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:57.715189 25788 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:57.715741 25788 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:57.715457 25374 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.668673 25708 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:57.606263 24965 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:57.676874 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:57.723984 21212 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:57.651315 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 589) took 1334 ms. Trace:
I20260709 13:07:57.601719 25515 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:07:57.595384 25754 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.603441 25801 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:07:57.590864 25756 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.684180 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:07:57.735896 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 59089.9us
W20260709 13:07:57.736390 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52286.9us
I20260709 13:07:57.737289 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:57.738031 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:57.695039 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:57.739588 21214 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:57.669768 25706 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:07:57.627261 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.740693 21064 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:57.741150 21110 leader_election.cc:400] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 6 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 6. Current term is 7.
I20260709 13:07:57.742074 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:57.660305 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 148391us
I20260709 13:07:57.701004 25736 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:57.691480 25518 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:57.746099 25518 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:57.707695 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:57.711776 25814 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:07:57.706952 21793 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:57.679062 25503 consensus_queue.cc:1243] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:07:57.697587 21221 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:57.686003 25737 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:57.725513 21217 rpcz_store.cc:276] 0709 13:07:56.316856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.517344 (+200488us) service_pool.cc:224] Handling call
0709 13:07:56.517668 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.215390 (+697722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.651003 (+435613us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54004439d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:57.651226 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597921024}
I20260709 13:07:57.668048 25744 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.686713 25629 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:57.741966 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.799372 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.821188 21072 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:07:57.688620 25441 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:07:57.797621 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:57.808563 21793 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:57.714954 25785 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.821188 21064 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:07:57.751994 24965 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:57.751758 25363 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:57.718175 25787 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.714548 25652 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.819854 25737 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:57.722303 25818 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
I20260709 13:07:57.804064 25708 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:57.718137 25762 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:57.837612 21793 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.871450 21793 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:57.751627 25522 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:07:57.818903 25756 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:57.821954 25820 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:57.876732 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:57.901311 21072 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:07:57.857075 24965 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:57.898053 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 286653us
W20260709 13:07:57.905835 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.910396 21216 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:57.912205 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:57.955538 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 337348us
I20260709 13:07:57.957932 25809 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:57.956758 21214 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:57.875962 25819 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:57.898079 25818 consensus_queue.cc:1243] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:07:57.909708 25820 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.930661 25708 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:57.971812 25809 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.928290 24965 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:57.982513 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 356921us
W20260709 13:07:57.974092 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 373579us
W20260709 13:07:57.950466 25711 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.003s
I20260709 13:07:57.996587 21072 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:57.949968 25801 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:58.004550 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:07:58.005087 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 260183us
I20260709 13:07:57.929910 25363 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:07:58.005772 25799 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:07:57.990893 25816 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:07:58.005501 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 283999us
I20260709 13:07:57.966431 25819 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:58.033063 25799 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:57.950156 25793 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:07:57.959751 25522 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:07:58.038102 25522 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:58.041190 25522 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:58.020264 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 184217us
I20260709 13:07:58.020058 21793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:07:58.043255 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:58.043654 21793 raft_consensus.cc:606] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:58.034621 25819 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.051167 21141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:58.051532 25819 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:58.051991 21141 raft_consensus.cc:606] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:57.990789 25820 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:58.008370 25708 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:57.991806 25818 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:07:58.019450 25830 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.080260 25785 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:58.020241 25504 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:57.995774 25812 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.019884 25736 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:07:58.065049 25374 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:58.065191 25795 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:57.995505 25629 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.073796 25441 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:07:58.094636 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 86820.6us
I20260709 13:07:58.105573 21185 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:58.076620 25754 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:58.089794 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 129915us
I20260709 13:07:58.021463 25645 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:07:58.064203 25835 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:07:58.115298 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 106259us
I20260709 13:07:58.113355 21793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:07:58.119787 21793 raft_consensus.cc:606] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:58.124205 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:58.094241 25374 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:58.128545 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 124095us
I20260709 13:07:58.110337 25395 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:58.111300 25636 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:58.107816 25441 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:58.112404 25808 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:58.110075 25711 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:07:58.138427 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.122428 25744 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:58.116798 25818 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:58.112136 25838 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:07:58.143524 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 401491us
I20260709 13:07:58.150668 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:58.125372 25821 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.127211 25652 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:58.133307 25827 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.151151 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:58.088743 25830 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:58.144801 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 98810.7us
I20260709 13:07:58.140086 25808 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:58.165998 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:58.125442 25831 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:07:58.110538 25795 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.174254 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1470) took 2324 ms. Trace:
I20260709 13:07:58.152740 25636 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:58.175148 21213 rpcz_store.cc:276] 0709 13:07:55.844510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.846170 (+  1660us) service_pool.cc:224] Handling call
0709 13:07:55.872566 (+ 26396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.936187 (+ 63621us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:56.167109 (+230922us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.026446 (+859337us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b180030c4c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f45f143c7 00007f7f425269a7 00007f7f425360bf 00007f7f42536038
0709 13:07:57.026760 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.069419 (+ 42659us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:57.069478 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:57.069550 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.169022 (+1099472us) spinlock_profiling.cc:243] Waited 63.3 ms on lock 0x7b540034cbd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:58.169086 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:58.169204 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33826,"prepare.queue_time_us":808679,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":69,"raft.queue_time_us":51359,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":3247,"spinlock_wait_cycles":157706496,"thread_start_us":433585,"threads_started":2,"wal-append.queue_time_us":309511,"wal-append.run_cpu_time_us":2036,"wal-append.run_wall_time_us":1576955}
I20260709 13:07:58.150382 25846 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:58.177172 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:58.156903 25838 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:58.145061 25711 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.171151 25736 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:58.190186 25850 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:58.182179 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:07:58.168298 25827 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.171361 25762 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:58.154145 25744 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.145823 25708 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.152566 25818 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.175403 25754 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:07:58.156399 25821 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.233894 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1443) took 1045 ms. Trace:
I20260709 13:07:58.234437 21071 rpcz_store.cc:276] 0709 13:07:57.188108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.189698 (+  1590us) service_pool.cc:224] Handling call
0709 13:07:57.189970 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.287018 (+ 97048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.233601 (+946583us) spinlock_profiling.cc:243] Waited 96.2 ms on lock 0x7b54002ac6d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:58.233823 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":221181184}
I20260709 13:07:58.242482 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:58.243647 21797 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:58.237677 25787 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:58.172331 25441 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.220819 25762 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.247087 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:58.248083 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:58.215794 25736 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:58.256732 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 79630.2us
I20260709 13:07:58.220513 25853 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:58.258659 21797 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:07:58.267414 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:58.177879 25374 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:58.269237 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:58.266479 25788 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.003s
I20260709 13:07:58.250174 25652 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:58.250612 25835 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:07:58.279436 25824 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.137s	user 0.000s	sys 0.002s
W20260709 13:07:58.280936 25847 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:07:58.281569 25821 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:58.268764 25787 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:58.245551 25808 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.258262 25840 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 13:07:58.281909 25829 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:07:58.282778 25843 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:07:58.268347 25838 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:58.283319 25822 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:07:58.272491 25812 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:58.268970 25441 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:58.232613 25744 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.270147 25853 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.283736 25736 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.266347 25762 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.282611 25858 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:58.283459 25834 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:07:58.313094 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2777) took 1366 ms. Trace:
I20260709 13:07:58.260214 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:58.365064 21219 rpcz_store.cc:276] 0709 13:07:56.946388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.239850 (+293462us) service_pool.cc:224] Handling call
0709 13:07:57.240521 (+   671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:57.455510 (+214989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:57.455543 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:57.490199 (+ 34656us) log.cc:844] Serialized 26 byte log entry
0709 13:07:57.717996 (+227797us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:57.764158 (+ 46162us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.801340 (+ 37182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.802390 (+  1050us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:58.312862 (+510472us) raft_consensus.cc:1685] finished
0709 13:07:58.312882 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:58.313029 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":205630,"prepare.queue_time_us":4019,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":6631,"raft.queue_time_us":324300,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":115,"spinlock_wait_cycles":14596864,"thread_start_us":202911,"threads_started":1,"wal-append.queue_time_us":556367}
W20260709 13:07:58.370903 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 110797us
W20260709 13:07:58.299482 25839 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:07:58.298180 25837 raft_consensus.cc:1064] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
W20260709 13:07:58.301442 25841 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:07:58.313503 25652 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:58.299037 25808 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:58.307662 25374 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:58.297408 25849 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.308167 25818 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:58.298174 25787 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:58.302480 25835 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:58.302115 25711 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:58.312552 25830 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:58.373327 25756 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:58.384331 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:58.387682 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 219411us
I20260709 13:07:58.270502 25636 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.395529 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:58.384807 25837 raft_consensus.cc:1068] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:07:58.613772 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 216537us
I20260709 13:07:58.379552 25652 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:07:58.468106 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.389600 25787 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:58.625844 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.033s	sys 0.067s
I20260709 13:07:58.392416 25835 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:58.628180 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.033s	sys 0.067s
W20260709 13:07:58.628675 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 509218us
W20260709 13:07:58.648346 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 497222us
I20260709 13:07:58.647274 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:07:58.651095 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:58.374495 25858 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:58.652980 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 765) took 1034 ms. Trace:
I20260709 13:07:58.318692 25744 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:58.653537 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 487512us
I20260709 13:07:58.664335 21069 rpcz_store.cc:276] 0709 13:07:57.619112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.621183 (+  2071us) service_pool.cc:224] Handling call
0709 13:07:57.670111 (+ 48928us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:57.691255 (+ 21144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:57.691332 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:57.700262 (+  8930us) log.cc:844] Serialized 26 byte log entry
0709 13:07:58.028229 (+327967us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:58.028449 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.028550 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:58.652693 (+624143us) raft_consensus.cc:1685] finished
0709 13:07:58.652723 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:58.652898 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20691,"raft.queue_time_us":119316,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":36597,"thread_start_us":248245,"threads_started":1,"wal-append.queue_time_us":399670}
I20260709 13:07:58.666424 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:58.623569 25837 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:58.625102 25850 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:07:58.617301 25795 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:58.630242 25835 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.308737 25827 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:58.634037 25787 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:58.703305 25826 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:58.703794 25826 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:07:58.703954 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.707001 25624 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:58.664363 25711 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 13:07:58.708544 25820 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.000s
I20260709 13:07:58.708616 25838 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:58.709074 25820 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.296805 25788 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.622450 25821 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:07:58.689075 25848 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:58.629849 25819 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:07:58.696122 25853 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:58.696900 25036 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:58.698385 24965 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.688s	user 0.008s	sys 0.003s
I20260709 13:07:58.707471 25624 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:58.616523 25818 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:07:58.389071 25849 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.736172 25624 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.737076 25849 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:58.633464 25706 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:58.738072 25624 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:58.738281 25706 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:58.738945 25706 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:58.718422 25865 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:58.686594 25652 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.711143 25820 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:58.718894 25856 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.719839 25867 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:58.720220 25819 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.716150 25848 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.625849 25846 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:07:58.715971 25795 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:07:58.650450 25860 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.340s
I20260709 13:07:58.726300 25858 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:58.753492 25848 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:58.759382 25652 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:58.763360 25852 consensus_queue.cc:1175] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "46c1676121b64b7d9bcf2227cc639dd3" responder_term: 13 status { last_received { term: 11 index: 7 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:58.764490 25819 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:07:58.766641 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.727896 25861 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:58.719843 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1452) took 1148 ms. Trace:
I20260709 13:07:58.767475 25861 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.767412 21066 rpcz_store.cc:276] 0709 13:07:57.571801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.580488 (+  8687us) service_pool.cc:224] Handling call
0709 13:07:57.580842 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.581111 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:57.593857 (+ 12746us) log.cc:844] Serialized 14 byte log entry
0709 13:07:58.708149 (+1114292us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b1800203980. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f45f143c7 00007f7f425269a7 00007f7f425360bf 00007f7f42536038
0709 13:07:58.708346 (+   197us) log.cc:844] Serialized 14 byte log entry
0709 13:07:58.719493 (+ 11147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:58.719546 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:58.719596 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.719659 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:58.719774 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1075465,"prepare.queue_time_us":1150289,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":15537,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":37399680}
I20260709 13:07:58.711153 25788 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:58.745187 25865 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.770035 25861 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:58.771394 25865 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:58.752591 25867 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.772100 25867 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:58.728423 25762 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:58.742293 25850 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.766670 25819 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:58.793959 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 398468us
W20260709 13:07:58.797973 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 178931us
I20260709 13:07:58.796546 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1435) took 1643 ms. Trace:
I20260709 13:07:58.786576 25833 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.828944 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:58.799220 25732 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:58.816028 21067 rpcz_store.cc:276] 0709 13:07:57.153400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.153526 (+   126us) service_pool.cc:224] Handling call
0709 13:07:57.153846 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.154062 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:58.615002 (+1460940us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b180036ee60. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f45f143c7 00007f7f425269a7 00007f7f425360bf
0709 13:07:58.615167 (+   165us) log.cc:844] Serialized 14 byte log entry
0709 13:07:58.796026 (+180859us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:58.796108 (+    82us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:58.796183 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.796292 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:58.796464 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":209944,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":2343,"raft.queue_time_us":3912,"spinlock_wait_cycles":9469184,"thread_start_us":359934,"threads_started":2}
I20260709 13:07:58.740680 24965 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:07:58.841677 25846 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:58.851635 21036 leader_election.cc:400] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:58.852355 21036 leader_election.cc:403] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:07:58.807902 25788 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:58.840299 25860 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.839716 25865 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.888091 25853 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:58.888000 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:58.891435 21220 raft_consensus.cc:2376] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13. Re-sending same reply.
I20260709 13:07:58.892488 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:58.785212 25856 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.885581 25836 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.874120 25863 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:58.853123 25762 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.883605 25865 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.876586 25827 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:07:58.876353 25861 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.924870 25861 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:58.925709 25865 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:58.922065 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:58.840785 25867 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.927130 21217 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:58.927848 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:58.841204 25833 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:58.941802 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:58.874526 25818 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.968830 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:58.968849 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:58.877415 25788 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.020742 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.017966 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:59.033056 21068 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:07:59.033793 21221 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
W20260709 13:07:58.955910 21187 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.600s (SENT) [suppressed 17 similar messages]
W20260709 13:07:58.940505 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.980827 21212 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:58.883874 25849 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:59.028420 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 362204us
W20260709 13:07:59.026918 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 374717us
I20260709 13:07:59.053392 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:58.942126 21798 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:58.980429 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:58.939807 21797 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:59.029695 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:58.877980 25860 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.023307 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.025410 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.022851 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.064075 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.064584 21067 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:59.022315 21144 raft_consensus.cc:606] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:07:59.022858 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.971794 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:59.066406 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.067039 21795 raft_consensus.cc:2368] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:07:59.035714 21221 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:59.028105 25808 raft_consensus.cc:993] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:59.025852 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:58.889861 25831 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.056308 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:59.056430 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:58.885785 25759 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.922577 25737 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:07:59.037147 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 68447.2us
W20260709 13:07:59.063859 25441 consensus_meta.cc:287] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.750s	user 0.011s	sys 0.002s
I20260709 13:07:59.086978 21795 pending_rounds.cc:85] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 11
I20260709 13:07:59.087389 21795 pending_rounds.cc:107] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:07:59.087622 21795 raft_consensus.cc:2889] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:59.088627 21219 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:07:59.051440 25504 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:59.071051 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.074790 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.094601 21071 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:07:59.077741 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 425313us
I20260709 13:07:58.927451 25853 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.068449 25821 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.069397 25495 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:59.089056 25737 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:59.027976 25846 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:59.096320 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:59.096684 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:59.051333 25818 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.098706 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.069535 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1448) took 1503 ms. Trace:
I20260709 13:07:59.110489 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:59.110922 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:59.076632 25820 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:59.115491 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:59.136754 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:59.137284 21067 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:58.969437 25762 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.072517 25809 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:59.085621 25816 raft_consensus.cc:1081] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:07:59.127750 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:59.128132 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:59.119801 25861 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:59.097831 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.085825 25817 raft_consensus.cc:476] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Not starting pre-election -- already a leader
I20260709 13:07:59.109876 25736 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:59.135001 21065 rpcz_store.cc:276] 0709 13:07:57.566001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.567428 (+  1427us) service_pool.cc:224] Handling call
0709 13:07:57.600833 (+ 33405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.601035 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:58.711581 (+1110546us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.068999 (+357418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:59.069075 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:59.069143 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.069234 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.069375 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1126843,"prepare.queue_time_us":1123499,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":7526,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9598,"thread_start_us":357052,"threads_started":1}
I20260709 13:07:59.135335 21071 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:07:59.089282 25759 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.092868 25865 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:58.919694 25836 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:59.127640 25374 consensus_meta.cc:287] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.000s
I20260709 13:07:59.140789 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.132838 25819 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:59.129444 21797 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:07:59.154924 21217 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:07:59.085448 25831 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.155927 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:07:59.144789 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.122094 25495 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.162257 21069 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:07:58.969579 25863 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.203524 25863 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.204478 25848 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.167878 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:59.206669 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:59.199854 25819 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.209216 25819 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.210400 25833 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.193182 25865 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.212400 25818 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.212934 25865 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.135943 25820 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.116837 25504 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.178149 21218 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:59.135802 25447 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:59.187610 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46937us
W20260709 13:07:59.171576 25746 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.001s	sys 0.022s
I20260709 13:07:59.188202 25036 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:59.215072 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:59.223066 25036 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.223503 25036 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:59.154544 25363 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:59.193046 21035 leader_election.cc:400] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:07:59.123155 25441 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:59.194209 21071 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
W20260709 13:07:59.194729 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 238827us
I20260709 13:07:59.126372 25821 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.200352 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:59.253422 21035 leader_election.cc:403] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:59.254266 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:59.136678 25846 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.064731 25849 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.226078 21800 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:07:59.213522 25848 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.261822 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:59.257951 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.251582 21220 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
W20260709 13:07:59.211195 25787 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.000s
I20260709 13:07:59.262449 21792 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:59.215898 25495 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:59.261408 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:07:59.268846 21217 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:59.228173 25820 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:59.175132 25736 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:59.258029 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 65139.1us
I20260709 13:07:59.276340 21066 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:07:59.188289 24965 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:07:59.239624 25746 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.001s	sys 0.022s
I20260709 13:07:59.158802 25809 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.267659 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:59.263113 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:59.254936 25441 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.292368 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.292934 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.292419 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:59.251896 25875 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.266975 25867 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.293617 21141 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:59.266024 25865 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:07:59.305238 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.263645 25363 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.266803 25818 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:59.281764 25515 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.195478 25706 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.314653 21145 raft_consensus.cc:2410] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:07:59.315868 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 48286.3us
I20260709 13:07:59.329460 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:59.316431 21138 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:07:59.283382 25856 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.191542 25629 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:07:59.318809 25363 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:59.324647 21141 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:07:59.199822 25737 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.305403 25441 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:59.318915 25818 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:59.354844 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.356361 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.312187 21217 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:07:59.271178 25787 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.369961 25849 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.314654 24965 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:07:59.371197 25787 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.312901 25788 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:07:59.372270 25887 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:07:59.372437 25736 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.373428 25836 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.373925 25629 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:59.374313 25629 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:07:59.375187 25629 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.375702 25853 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.376442 25629 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:59.377043 25821 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:59.377390 25652 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.000s
I20260709 13:07:59.312849 25875 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.378858 25504 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.380777 25875 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.327622 25636 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:59.381764 25636 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:59.327936 25867 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:59.382256 25636 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:59.382622 25867 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.383862 25759 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.384850 25867 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.351127 25036 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.362236 25860 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.363976 25762 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.364324 25706 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.365495 25809 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.344029 25858 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:59.367722 21214 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:07:59.412233 25036 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.368192 25846 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.389766 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:07:59.382269 25835 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.665s	user 0.007s	sys 0.003s
I20260709 13:07:59.397066 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:59.395392 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.400939 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.327790 25711 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:59.402503 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 763) took 1790 ms. Trace:
I20260709 13:07:59.347906 25819 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:59.390483 21221 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:59.417313 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 196619us
I20260709 13:07:59.418699 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:07:59.391750 25882 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:07:59.420096 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:59.452056 21063 rpcz_store.cc:276] 0709 13:07:57.612214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.612681 (+   467us) service_pool.cc:224] Handling call
0709 13:07:57.612948 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.613148 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:59.055686 (+1442538us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.402120 (+346434us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:59.402200 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:59.402262 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.402326 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.402452 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1098605,"prepare.queue_time_us":1111612,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":58767,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":7959,"thread_start_us":667489,"threads_started":3}
I20260709 13:07:59.329777 25515 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.455456 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:59.463773 21214 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:59.412299 25878 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:07:59.455899 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.373440 25818 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:59.457176 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61926.1us
I20260709 13:07:59.462540 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.412043 25858 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:59.489506 21067 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:07:59.491074 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.491544 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.469645 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:07:59.494972 21797 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:07:59.471120 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:59.471489 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.495782 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.496464 21138 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:07:59.493355 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:59.497838 21067 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:59.468767 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.492323 25858 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:59.497769 21214 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:07:59.498724 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.500330 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:07:59.504099 21138 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:59.458871 25835 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.458338 25711 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.508174 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:07:59.493463 25846 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:59.509763 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.509542 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:59.514119 21214 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
W20260709 13:07:59.514518 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.515055 21067 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 8.
I20260709 13:07:59.517066 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.517216 21767 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:07:59.520119 21767 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:07:59.515760 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 292974us
I20260709 13:07:59.501845 25833 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:59.489012 25788 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 13:07:59.525070 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 262028us
I20260709 13:07:59.523234 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:59.532608 21213 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 19: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:07:59.508399 25809 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:59.523247 21067 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:59.535096 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:07:59.557420 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.540555 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.511479 25759 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:59.489624 25878 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:07:59.574577 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.561702 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.572453 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.579157 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.503830 25789 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:07:59.512835 25361 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:07:59.545943 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.579438 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.526664 25816 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:59.527534 25833 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:07:59.588510 25833 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:59.589020 25897 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:07:59.589406 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:07:59.590029 25899 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:07:59.576315 25759 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:59.591979 25759 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:59.594051 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:59.594661 21142 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:59.597128 25515 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.597325 25495 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:07:59.597931 25809 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:59.598532 25809 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:59.599385 25361 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.599452 25809 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.529371 25846 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.602129 21142 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:07:59.602260 25846 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.604684 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.605580 21064 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:07:59.606696 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:59.607669 21072 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:07:59.608522 25711 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.609412 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.575891 25736 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:59.610035 25736 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.610406 25736 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:59.610522 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.611173 21142 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:07:59.612737 25835 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.617535 25441 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.618059 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.618896 21145 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:59.575750 25836 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:59.621366 21142 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:59.562013 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.556713 25762 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:59.627806 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65894us
I20260709 13:07:59.555963 25860 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.523365 25905 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:59.590071 21067 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:59.639498 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.528570 25831 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.640257 21142 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:07:59.685019 25865 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.685472 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.685573 25865 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.686698 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.638130 25816 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:07:59.665741 25894 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:59.666862 25821 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.667965 25819 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.641101 25762 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:07:59.641563 25624 consensus_meta.cc:287] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.903s	user 0.011s	sys 0.003s
I20260709 13:07:59.641238 25904 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:59.665833 25495 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.687259 21142 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:59.687927 25860 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.691867 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.692934 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.693246 25860 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:07:59.588500 25896 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:07:59.670835 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.528168 25856 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:07:59.695909 25856 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:59.695950 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.696389 25856 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:07:59.697155 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.697220 25856 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.697989 21064 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:07:59.619750 25836 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.699294 25836 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:59.688412 25894 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.701668 25894 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.688649 25821 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.702881 25624 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:59.703496 25624 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.704566 25821 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.688874 25819 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.705539 25624 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.684092 25441 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.706887 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.707513 21064 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:07:59.669505 25833 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.688163 25816 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.709293 21138 raft_consensus.cc:2410] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:07:59.693871 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.714742 21067 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:07:59.732116 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:07:59.733912 21142 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
W20260709 13:07:59.730181 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 229917us
I20260709 13:07:59.732796 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.734833 21067 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:07:59.729236 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:59.735770 21064 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:59.690637 25904 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:07:59.690428 25762 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:07:59.738148 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:59.738867 21067 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:59.739595 25819 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.740250 25826 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.679077 25736 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.726341 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:59.826937 21068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.011s	sys 0.027s
I20260709 13:07:59.903353 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.738482 25904 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:59.903687 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 976) took 1006 ms. Trace:
I20260709 13:07:59.803753 25759 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:07:59.904129 21064 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
W20260709 13:07:59.832015 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 89809.7us
W20260709 13:07:59.907532 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 181297us
W20260709 13:07:59.904948 21068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.011s	sys 0.030s
I20260709 13:07:59.905388 21794 rpcz_store.cc:276] 0709 13:07:58.897299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.897967 (+   668us) service_pool.cc:224] Handling call
0709 13:07:58.898310 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.071698 (+173388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.071744 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:59.071786 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:59.071831 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.743618 (+671787us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b540060efd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:59.903439 (+159821us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.903608 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398204160}
I20260709 13:07:59.910836 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:07:59.912943 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:59.915660 21142 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:07:59.918200 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:07:59.918859 21036 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:07:59.919184 21036 leader_election.cc:403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:07:59.919955 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:07:59.921713 21067 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:07:59.923072 25036 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:59.923871 25036 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:59.924265 25036 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:07:59.924611 25036 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:59.926525 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:07:59.917330 25789 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:59.928056 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:59.928382 25789 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:59.929682 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:07:59.930497 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:07:59.912380 25820 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.007s
I20260709 13:07:59.961098 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:07:59.961637 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:07:59.961840 21067 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:59.921595 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2797) took 1028 ms. Trace:
I20260709 13:07:59.963788 21215 rpcz_store.cc:276] 0709 13:07:58.894099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.894231 (+   132us) service_pool.cc:224] Handling call
0709 13:07:59.015634 (+121403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.200480 (+184846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.200512 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:59.201008 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:07:59.386699 (+185691us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:59.386739 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.764720 (+377981us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400442fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:07:59.764794 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:59.764865 (+    71us) raft_consensus.cc:1685] finished
0709 13:07:59.764886 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.921501 (+156615us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16376,"raft.queue_time_us":403,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":424212736,"thread_start_us":185296,"threads_started":1,"wal-append.queue_time_us":332956}
I20260709 13:07:59.911998 25894 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:59.916052 25788 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:07:59.985639 25788 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:07:59.930991 25495 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:00.003579 25495 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:00.004269 25495 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.004413 25858 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.005594 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.010215 21213 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:08:00.007007 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:07:59.918366 25636 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.535s	user 0.004s	sys 0.007s
I20260709 13:08:00.020673 21068 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:00.021937 25629 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.000s
I20260709 13:08:00.022537 25629 raft_consensus.cc:2764] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:08:00.022975 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.023226 25629 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:00.023746 25629 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.024362 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.024994 21138 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:59.979769 25820 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.028374 25759 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:00.029317 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2793) took 1137 ms. Trace:
I20260709 13:08:00.029834 25736 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.030311 25820 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:07:59.923225 25819 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.912328 25821 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:00.054059 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:59.999028 25915 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:07:59.916929 25809 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:08:00.063791 25036 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.064412 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2278) took 1043 ms. Trace:
I20260709 13:08:00.070355 21065 rpcz_store.cc:276] 0709 13:07:59.021261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.174979 (+153718us) service_pool.cc:224] Handling call
0709 13:07:59.183868 (+  8889us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.195146 (+ 11278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.195185 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:59.213452 (+ 18267us) log.cc:844] Serialized 26 byte log entry
0709 13:07:59.509101 (+295649us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:59.509141 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.509208 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:00.064183 (+554975us) raft_consensus.cc:1685] finished
0709 13:08:00.064203 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.064356 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":7147,"thread_start_us":295258,"threads_started":1,"wal-append.queue_time_us":527901}
I20260709 13:08:00.071309 21138 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:00.072640 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.057585 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.083434 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.077939 21801 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:07:59.936558 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.019330 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2792) took 1131 ms. Trace:
I20260709 13:08:00.087255 25036 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.017424 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.037671 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.038758 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.040388 21798 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
W20260709 13:08:00.040845 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.041517 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.029716 21218 rpcz_store.cc:276] 0709 13:07:58.891751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.892232 (+   481us) service_pool.cc:224] Handling call
0709 13:08:00.029103 (+1136871us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b540015ca54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:00.029237 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654878592}
I20260709 13:08:00.026141 25629 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:07:59.991994 25865 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.099510 21218 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:59.982214 25894 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:00.057715 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.104274 21147 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:00.086581 21219 rpcz_store.cc:276] 0709 13:07:58.887913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.891346 (+  3433us) service_pool.cc:224] Handling call
0709 13:08:00.019039 (+1127693us) spinlock_profiling.cc:243] Waited 60.4 ms on lock 0x7b54002ec2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:00.019242 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138966272}
I20260709 13:07:59.994021 21067 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:00.106933 21035 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:00.107235 25858 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.088791 21065 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:00.108491 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 767) took 1047 ms. Trace:
I20260709 13:08:00.108841 21065 rpcz_store.cc:276] 0709 13:07:59.061144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.084546 (+1023402us) service_pool.cc:224] Handling call
0709 13:08:00.088468 (+  3922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:00.108243 (+ 19775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.108442 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:00.088210 21794 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:00.095499 21215 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:08:00.060123 25818 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
W20260709 13:08:00.069970 25916 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:08:00.052285 25923 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
W20260709 13:08:00.097934 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 178039us
I20260709 13:08:00.058653 25732 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.121985 25732 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:00.123255 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.127202 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:00.127529 25926 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:00.128188 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:00.128079 25926 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.128944 25902 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:00.130398 25926 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.130903 25926 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:00.131496 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.114892 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.132929 21142 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:00.132840 25904 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.133936 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.134111 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.135416 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.130064 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:00.108330 25894 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.076877 25821 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.118578 21065 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:08:00.050784 25636 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.062695 25819 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.061340 25836 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.124155 25732 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.146991 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.119377 25818 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.105548 25833 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.144618 25758 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.135082 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.153550 21142 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:08:00.110878 21215 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:08:00.095606 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.157056 21138 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:00.113535 21147 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:00.159191 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.107925 25754 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:00.135074 25904 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.145370 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.145854 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:00.174852 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.144722 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 771) took 1068 ms. Trace:
I20260709 13:08:00.118072 25788 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.148917 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.175987 21065 rpcz_store.cc:276] 0709 13:07:59.076051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.114028 (+1037977us) service_pool.cc:224] Handling call
0709 13:08:00.116566 (+  2538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:00.142441 (+ 25875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.144452 (+  2011us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:00.153661 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:00.146106 25447 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:00.137324 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.139750 25919 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:08:00.180279 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.177882 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.178236 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:00.176534 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:00.177021 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47030.1us
I20260709 13:08:00.182653 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:00.162968 25754 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.140373 25926 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.184211 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.184530 21138 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:00.186671 25925 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:00.187471 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.188582 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.188655 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.189630 21142 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:08:00.190768 21142 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:08:00.192826 25495 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:00.199147 21070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2259) took 2667 ms (client timeout 1600 ms). Trace:
I20260709 13:08:00.199302 25447 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:00.199769 25447 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:00.199658 21070 rpcz_store.cc:269] 0709 13:07:57.531528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.534971 (+  3443us) service_pool.cc:224] Handling call
0709 13:08:00.198962 (+2663991us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540041d554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:00.199080 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3724891904}
I20260709 13:08:00.200299 25447 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:00.200999 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.201290 25636 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.185653 25788 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:08:00.203509 25788 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:00.203873 25788 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:00.204308 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:00.204507 25878 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.204677 25788 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.205468 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.206043 25788 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.180125 25927 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:00.207052 21070 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:08:00.208230 21142 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:08:00.212044 21139 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:08:00.212589 25818 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.213474 25927 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.213979 25927 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.214735 25926 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.218693 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.168830 25758 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.219372 21146 raft_consensus.cc:2376] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13. Re-sending same reply.
W20260709 13:08:00.220202 25363 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.834s	user 0.009s	sys 0.002s
I20260709 13:08:00.220333 25836 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:00.220665 21146 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:08:00.221158 25758 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.186017 25754 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:00.221863 21146 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:00.196452 25833 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:00.223668 25833 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:00.224116 25833 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:00.224400 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 774) took 1049 ms. Trace:
I20260709 13:08:00.224911 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.224793 21065 rpcz_store.cc:276] 0709 13:07:59.175152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.223659 (+1048507us) service_pool.cc:224] Handling call
0709 13:08:00.223943 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:00.224107 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:00.224130 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:00.224172 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:00.224206 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.224259 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.224353 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:00.213577 21139 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:08:00.230510 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.231405 21794 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:00.252063 21139 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:08:00.256755 21139 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:00.261170 25852 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.261844 25852 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.263528 25931 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:00.263935 21146 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:08:00.264295 25931 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.264834 25931 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.220674 25363 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.269372 25363 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.226910 25821 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.267076 25931 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.271710 25931 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.272338 25931 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.274576 25931 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.290050 25894 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.294591 21068 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:00.299431 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.300050 21793 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:00.305555 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.307248 21139 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:08:00.307682 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.312934 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.313875 21797 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:00.316655 21067 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:08:00.320822 21139 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:08:00.325013 21793 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:00.327414 21139 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:08:00.328099 21139 pending_rounds.cc:85] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 7
I20260709 13:08:00.328436 21139 pending_rounds.cc:107] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:08:00.328742 21139 raft_consensus.cc:2889] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:00.322715 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:00.306329 25925 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:00.326179 25902 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:00.345772 25889 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:00.345999 25495 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.346454 25889 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:00.331851 25374 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:08:00.347734 25852 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.347544 25889 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.350035 25894 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.355984 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:00.365746 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 183141us
I20260709 13:08:00.368014 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:00.370537 21065 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:00.372314 21065 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:08:00.373998 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.370420 25852 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 13:08:00.379837 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 52068.3us
I20260709 13:08:00.380566 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 773) took 1233 ms. Trace:
I20260709 13:08:00.381052 21070 rpcz_store.cc:276] 0709 13:07:59.147344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.201939 (+1054595us) service_pool.cc:224] Handling call
0709 13:08:00.202204 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:00.207530 (+  5326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.380326 (+172796us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b540041c654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:00.380512 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10408832}
I20260709 13:08:00.380993 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.383011 21793 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:00.328828 25927 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.393090 21070 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:00.399828 21793 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:00.402266 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.400714 25925 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.410173 25927 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.392557 25833 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.411393 21065 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:08:00.411878 21800 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:00.413158 25833 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:00.415686 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 768) took 1350 ms. Trace:
I20260709 13:08:00.416085 21064 rpcz_store.cc:276] 0709 13:07:59.065171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.111073 (+1045902us) service_pool.cc:224] Handling call
0709 13:08:00.135596 (+ 24523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:00.135942 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:00.135971 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:00.141349 (+  5378us) log.cc:844] Serialized 26 byte log entry
0709 13:08:00.229653 (+ 88304us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:00.229691 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.229756 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:00.415460 (+185704us) raft_consensus.cc:1685] finished
0709 13:08:00.415481 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.415625 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1583,"raft.queue_time_us":4341,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4456,"thread_start_us":56093,"threads_started":1,"wal-append.queue_time_us":165151}
I20260709 13:08:00.417693 21796 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:08:00.391316 25919 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 13:08:00.427934 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60440.8us
I20260709 13:08:00.381346 25754 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.434091 21064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:60302
I20260709 13:08:00.434873 21064 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:00.435372 21064 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:00.435606 21797 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:00.439561 25933 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:08:00.441460 21066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2269) took 1423 ms (client timeout 1794 ms). Trace:
I20260709 13:08:00.442119 25936 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:08:00.432425 25708 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:08:00.444164 25934 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:00.445268 25754 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.446748 21069 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:08:00.441885 21066 rpcz_store.cc:269] 0709 13:07:59.018365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.023037 (+  4672us) service_pool.cc:224] Handling call
0709 13:08:00.441299 (+1418262us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54000db954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:00.441405 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488941312}
I20260709 13:08:00.448365 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.448843 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.449394 21794 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:00.459821 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.459697 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.432111 25624 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:00.465500 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.464088 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.466724 21139 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:00.467495 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.464828 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.469389 21142 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:00.470749 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.473529 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.418797 25447 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.476665 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.478662 21142 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.476938 21064 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:00.484440 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.472324 25852 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:00.495775 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:00.496800 21066 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:00.500221 25447 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.477257 25624 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:00.505580 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.506816 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:00.512396 21794 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:00.512621 21066 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:00.469120 25927 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:08:00.514030 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:00.470700 25754 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:08:00.471952 25495 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:08:00.472669 25894 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:08:00.520761 21035 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:00.432781 25902 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.536085 25821 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:00.553954 25927 consensus_queue.cc:1243] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:08:00.543352 21066 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:00.546069 25495 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:00.547762 25894 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.548313 25926 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:00.549118 25930 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.548979 25624 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.496584 25852 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:00.552554 25940 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:08:00.518926 25818 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:08:00.556087 21035 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:00.558426 25927 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:00.557595 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:00.557793 21142 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:00.521317 25447 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:00.569240 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:00.564229 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:00.557443 21064 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.557195 25495 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:00.569233 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:00.573349 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.559047 25926 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:00.572811 25818 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:00.575805 25732 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:00.577669 25732 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:00.577256 25926 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.576180 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:60302 (request call id 778) took 1016 ms. Trace:
W20260709 13:08:00.580019 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.567911 25927 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.584940 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1504) took 1499 ms. Trace:
I20260709 13:08:00.585393 21221 rpcz_store.cc:276] 0709 13:07:59.085304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.087204 (+  1900us) service_pool.cc:224] Handling call
0709 13:07:59.087486 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.457346 (+369860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.584691 (+1127345us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b5400102554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:00.584879 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633823488}
I20260709 13:08:00.572381 25889 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:00.569763 25447 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.570609 25878 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:00.561661 25894 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.579712 21064 rpcz_store.cc:276] 0709 13:07:59.556410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.417420 (+861010us) service_pool.cc:224] Handling call
0709 13:08:00.572265 (+154845us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:00.560406 25930 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.585507 25926 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:00.589080 25927 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:00.598531 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 2970) took 1013 ms. Trace:
I20260709 13:08:00.598623 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:00.590459 25447 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:00.600859 25894 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.602211 21138 rpcz_store.cc:276] 0709 13:07:59.585317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.193829 (+608512us) service_pool.cc:224] Handling call
0709 13:08:00.194115 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:00.311688 (+117573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:00.311719 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:00.333089 (+ 21370us) log.cc:844] Serialized 26 byte log entry
0709 13:08:00.391905 (+ 58816us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:00.391950 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.392047 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:00.598284 (+206237us) raft_consensus.cc:1685] finished
0709 13:08:00.598333 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.598477 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66581,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"thread_start_us":162481,"threads_started":2,"wal-append.queue_time_us":126634}
I20260709 13:08:00.602193 21064 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:00.590782 25878 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:00.594053 25889 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:00.609807 25878 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:00.606761 25495 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.596659 25816 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:00.576362 25852 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.580613 25732 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.613224 25495 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:00.623442 25942 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:08:00.628010 21139 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:00.634126 21064 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:08:00.635150 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1466) took 1056 ms. Trace:
I20260709 13:08:00.635579 21064 rpcz_store.cc:276] 0709 13:07:59.578821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.592631 (+1013810us) service_pool.cc:224] Handling call
0709 13:08:00.602086 (+  9455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:00.634686 (+ 32600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.635083 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:00.636466 21146 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:08:00.638185 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.617715 25816 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:00.689165 25948 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:08:00.690286 25816 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.685359 25852 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:00.700254 25930 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.704321 21064 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:00.705049 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1468) took 1117 ms. Trace:
I20260709 13:08:00.705407 21064 rpcz_store.cc:276] 0709 13:07:59.587312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.637470 (+1050158us) service_pool.cc:224] Handling call
0709 13:08:00.704012 (+ 66542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:00.704808 (+   796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.704998 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:00.684501 25840 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:08:00.686519 25889 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.726130 25819 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.727701 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.730016 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2280) took 1708 ms. Trace:
W20260709 13:08:00.731149 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.730456 21071 rpcz_store.cc:276] 0709 13:07:59.021861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.254312 (+232451us) service_pool.cc:224] Handling call
0709 13:07:59.254623 (+   311us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:59.282649 (+ 28026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.282694 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:59.283267 (+   573us) log.cc:844] Serialized 52 byte log entry
0709 13:07:59.532041 (+248774us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:59.620170 (+ 88129us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.620932 (+   762us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.621209 (+   277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.729626 (+1108417us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b540001b3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:00.729677 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:00.729811 (+   134us) raft_consensus.cc:1685] finished
0709 13:08:00.729827 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.729959 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":60682,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":80,"raft.queue_time_us":13706,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":22889,"spinlock_wait_cycles":70032128,"thread_start_us":204773,"threads_started":1,"wal-append.queue_time_us":695606}
I20260709 13:08:00.735754 21142 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:00.713138 25930 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:08:00.742225 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.743127 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2518) took 1454 ms. Trace:
I20260709 13:08:00.743944 21064 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:00.749169 25447 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:00.750634 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 1.18892e+06us
I20260709 13:08:00.750887 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1514) took 1577 ms. Trace:
I20260709 13:08:00.743489 21143 rpcz_store.cc:276] 0709 13:07:59.288614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.289665 (+  1051us) service_pool.cc:224] Handling call
0709 13:08:00.742055 (+1452390us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:08:00.743059 (+  1004us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7184640}
I20260709 13:08:00.752985 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:00.754395 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:00.751185 21216 rpcz_store.cc:276] 0709 13:07:59.173782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.207961 (+ 34179us) service_pool.cc:224] Handling call
0709 13:07:59.208244 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.208445 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:59.294980 (+ 86535us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.535125 (+240145us) log.cc:844] Serialized 14 byte log entry
0709 13:08:00.216557 (+681432us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b180046e160. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:00.750536 (+533979us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:00.750592 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:00.750644 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.750724 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.750844 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1216516,"prepare.queue_time_us":167875,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":25958,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7363,"spinlock_wait_cycles":8884480,"thread_start_us":324322,"threads_started":2,"wal-append.queue_time_us":436292}
I20260709 13:08:00.740603 25361 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:00.758782 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:00.765034 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32968 (request call id 22) took 2588 ms. Trace:
I20260709 13:08:00.759690 25706 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:00.765450 21144 rpcz_store.cc:276] 0709 13:07:58.176911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.177214 (+   303us) service_pool.cc:224] Handling call
0709 13:08:00.764849 (+2587635us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b540042f754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42516853 00007f7f474e460a 00007f7f40a6a7d4 00007f7f40a6a762 00007f7f40a6a691 00007f7f40a6a60c 00007f7f40a698c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:00.764953 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1890745728}
I20260709 13:08:00.767063 25706 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:00.767556 25706 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:00.765472 25503 raft_consensus.cc:2764] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:08:00.770220 25756 raft_consensus.cc:1064] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
W20260709 13:08:00.770651 25756 raft_consensus.cc:1068] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:00.771400 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:00.772753 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:00.772887 25515 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.760157 25833 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:00.773367 25515 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.757668 25816 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:00.759380 25711 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:00.774611 25711 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:00.775004 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 766) took 1714 ms. Trace:
I20260709 13:08:00.771077 25756 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.775467 21068 rpcz_store.cc:276] 0709 13:07:59.060826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.051588 (+990762us) service_pool.cc:224] Handling call
0709 13:08:00.051911 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:00.295182 (+243271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.774766 (+479584us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400019fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:00.774948 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557725440}
I20260709 13:08:00.775745 25756 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.776226 25515 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:00.776830 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:00.777927 25756 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:00.778355 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:00.780409 25447 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.791903 25826 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.795496 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2537) took 1305 ms. Trace:
I20260709 13:08:00.796125 21145 rpcz_store.cc:276] 0709 13:07:59.490293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.492239 (+  1946us) service_pool.cc:224] Handling call
0709 13:08:00.795246 (+1303007us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400279554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:00.795421 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280941312}
I20260709 13:08:00.798525 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.798705 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:00.799363 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2523) took 1508 ms (client timeout 1732 ms). Trace:
I20260709 13:08:00.799540 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:00.799763 21140 rpcz_store.cc:269] 0709 13:07:59.290512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.290692 (+   180us) service_pool.cc:224] Handling call
0709 13:08:00.798371 (+1507679us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:08:00.799276 (+   905us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11029376}
I20260709 13:08:00.800719 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:00.801342 25827 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:00.801745 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:00.801872 25827 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:00.803213 25827 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.734807 25878 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.808830 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.809760 21792 raft_consensus.cc:2410] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:08:00.811483 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.812503 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.813048 21063 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:00.777927 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.813840 21068 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:00.814133 21066 raft_consensus.cc:2410] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:08:00.812522 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.775015 25711 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.829032 25904 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:00.782757 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 210623us
I20260709 13:08:00.787089 25933 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:00.787823 25935 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:00.788735 25864 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:00.815884 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:00.835088 21068 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:00.816990 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:00.835908 21185 connection.cc:256] RPC call timeout handler was delayed by -1.2718652639999846s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 13:08:00.836613 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.817555 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.818037 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.810336 20966 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 9 to 10, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 10 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:00.840004 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.819375 21792 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 19: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:00.819692 25759 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:00.820008 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.820230 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.821681 25746 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:00.822377 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:00.825709 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.845077 21066 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:00.845433 21213 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:08:00.826421 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.846570 21215 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
W20260709 13:08:00.810346 25921 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.836s	user 0.002s	sys 0.000s
I20260709 13:08:00.811096 25737 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:00.830034 21035 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:00.847829 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.850831 25732 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:00.852986 25754 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:00.853538 25754 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.857096 25894 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:00.857620 25894 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:00.859817 25812 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:00.860687 25812 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:00.861012 25812 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:00.861809 21068 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:08:00.862131 21794 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:00.863631 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:00.864207 21795 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:00.864480 21795 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:00.865173 21795 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.866039 21795 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:00.867694 25732 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:00.868738 25732 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.830530 21217 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:00.873908 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.876580 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.877234 21215 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:00.878014 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.879108 25812 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.879591 21068 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:08:00.880511 25812 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.881381 21068 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:00.831029 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:00.883963 25754 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:00.884913 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:00.759642 25908 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.066s	user 0.004s	sys 0.000s
I20260709 13:08:00.831240 21216 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:08:00.889290 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.889904 21216 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:08:00.890697 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:00.892607 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1511) took 1778 ms. Trace:
I20260709 13:08:00.892975 21212 rpcz_store.cc:276] 0709 13:07:59.115971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.207527 (+ 91556us) service_pool.cc:224] Handling call
0709 13:07:59.207855 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:59.307049 (+ 99194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.307095 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:59.307761 (+   666us) log.cc:844] Serialized 52 byte log entry
0709 13:07:59.472042 (+164281us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:59.506144 (+ 34102us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.508198 (+  2054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.508285 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:00.692944 (+1184659us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b1800420640. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:00.746749 (+ 53805us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:00.748437 (+  1688us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b1800420640. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f463f058f 00007f7f3f42f571 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd
0709 13:08:00.892404 (+143967us) raft_consensus.cc:1685] finished
0709 13:08:00.892425 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:00.892546 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121819,"prepare.queue_time_us":38982,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":83093,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":34674,"spinlock_wait_cycles":51934080,"thread_start_us":130994,"threads_started":1,"wal-append.queue_time_us":250141}
I20260709 13:08:00.897434 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:00.898567 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.891206 21216 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:00.905593 25816 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:00.907099 25816 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:00.907811 25816 consensus_queue.cc:1243] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:08:00.908252 25816 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:00.908557 25816 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:00.909314 25816 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:00.910478 25816 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.911535 25926 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:00.912395 25926 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.912849 25926 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.914958 25926 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.875065 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.920629 21217 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:08:00.921470 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.923686 21217 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:00.933140 25894 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.838214 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:00.839892 21138 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:00.941298 21071 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
W20260709 13:08:00.942559 21185 connection.cc:256] RPC call timeout handler was delayed by -1.0837652640000215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:08:00.943318 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 127460us
I20260709 13:08:00.944868 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:00.951732 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:00.953364 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.954197 21142 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:00.957540 25936 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:00.958098 25936 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:00.963577 25820 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.964138 25820 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:00.943066 21185 connection.cc:256] RPC call timeout handler was delayed by -1.0243652640000391s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
W20260709 13:08:00.969951 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.954998 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:00.972605 21212 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:08:00.835011 25935 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:00.982635 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:00.983922 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.835216 25933 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:00.985934 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.930913 21795 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:08:00.842279 25874 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:00.992529 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.993037 21213 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:00.842064 25836 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:00.980806 25762 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:00.844975 25636 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:00.838742 25904 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:01.021363 21792 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:00.997201 21063 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:01.023965 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:01.024780 21063 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:00.998257 21066 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
W20260709 13:08:00.998987 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.998538 21071 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:08:01.001294 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:01.015880 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:00.994894 21035 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:01.008942 21216 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
W20260709 13:08:01.021449 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 76700.2us
I20260709 13:08:01.029728 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:00.974505 25833 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:00.975751 25820 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:00.967851 25746 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:01.049847 21794 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:01.050406 21794 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:01.034430 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:01.052394 21070 raft_consensus.cc:2410] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:08:01.019817 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:01.049863 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:01.058243 25789 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:01.059523 25503 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:01.038866 21216 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:08:01.039636 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.996150 25756 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.061821 25756 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:01.062619 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:00.995879 25515 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.063901 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.064189 25515 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:01.064962 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:01.065557 21216 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:01.066227 25515 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:00.935458 25759 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.049063 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:01.032873 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.067356 21215 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:01.051399 21794 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.068846 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:01.069008 21185 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:01.069319 21794 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:00.843124 25864 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.070578 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:00.942729 25927 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.035905 25878 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:01.035262 25762 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:01.051502 25863 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:01.073575 25759 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:01.073747 25905 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:01.075619 25936 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:01.076138 25933 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:01.052596 25833 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.031587 25874 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:01.038525 25826 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:01.071942 21213 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:01.094862 21216 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:01.063930 25756 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:01.071211 25789 consensus_queue.cc:1175] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" responder_term: 15 status { last_received { term: 12 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: [14.12-14.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:01.022604 25873 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:08:01.124527 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:01.106799 21185 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:01.080425 25853 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:01.023262 25836 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.031363 25636 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.118496 21795 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:01.122865 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:01.134887 21216 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:01.135411 21794 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:08:01.135763 21794 pending_rounds.cc:85] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 11
I20260709 13:08:01.135969 21794 pending_rounds.cc:107] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:08:01.136178 21794 raft_consensus.cc:2889] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:01.073297 25855 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:08:01.146214 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.092917 25795 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.136925 21035 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:01.132844 25919 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:01.150782 25919 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:01.151829 25923 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:01.151747 25919 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.152406 25923 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:01.153271 25923 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.137941 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:01.156734 21140 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:01.156935 25873 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:01.157460 25873 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:01.157925 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:01.158376 25873 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:01.159227 21140 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:01.133183 25933 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:01.159931 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:01.121263 25863 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:01.160302 25762 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.161484 25853 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:01.162567 25795 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.088047 25935 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:01.166975 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:01.244110 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.244407 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.245563 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.171365 25706 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:01.175187 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2274) took 2155 ms. Trace:
I20260709 13:08:01.247706 21072 rpcz_store.cc:276] 0709 13:07:59.020067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.025583 (+  5516us) service_pool.cc:224] Handling call
0709 13:08:01.174949 (+2149366us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b540019d2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:01.175112 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226686336}
I20260709 13:08:01.175405 21068 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:08:01.178972 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:01.249230 21146 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:01.249181 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:01.180877 25636 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:01.182497 25864 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:01.183751 21069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 777) took 1652 ms (client timeout 2105 ms). Trace:
I20260709 13:08:01.183315 25874 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.184595 21794 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:08:01.263176 21794 pending_rounds.cc:85] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 8
I20260709 13:08:01.263490 21794 pending_rounds.cc:107] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:08:01.263721 21794 raft_consensus.cc:2889] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:01.186223 21216 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:01.260221 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:01.265053 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:01.266625 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.187958 25821 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:08:01.192281 21220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2795) took 2300 ms (client timeout 1925 ms). Trace:
I20260709 13:08:01.192802 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.193025 21214 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:08:01.199158 25836 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:01.135193 25826 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:01.190243 25711 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:01.203704 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.206615 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:01.207515 25856 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:01.160723 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53997.8us
I20260709 13:08:01.228164 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2529) took 1888 ms. Trace:
W20260709 13:08:01.237407 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 77567.5us
I20260709 13:08:01.238260 21063 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:01.238870 21070 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:08:01.239532 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 994) took 1021 ms. Trace:
W20260709 13:08:01.251637 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.251271 21072 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:08:01.259635 21069 rpcz_store.cc:269] 0709 13:07:59.531140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.700929 (+169789us) service_pool.cc:224] Handling call
0709 13:08:00.204154 (+503225us) spinlock_profiling.cc:243] Waited 7.02 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:08:01.183550 (+979396us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400214654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:01.183693 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571783296}
I20260709 13:08:01.265187 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:01.189436 25833 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:01.268824 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.319171 21065 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:08:01.251662 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2801) took 1858 ms. Trace:
I20260709 13:08:01.324791 21218 rpcz_store.cc:276] 0709 13:07:59.393551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.101267 (+707716us) service_pool.cc:224] Handling call
0709 13:08:00.101549 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:00.767566 (+666017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:00.767604 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:00.768113 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 13:08:00.998956 (+230843us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:01.251043 (+252087us) log.cc:844] Serialized 14 byte log entry
0709 13:08:01.251373 (+   330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.251442 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:01.251482 (+    40us) raft_consensus.cc:1685] finished
0709 13:08:01.251494 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:01.251606 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":665585,"raft.queue_time_us":666828,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":9746,"thread_start_us":445812,"threads_started":2,"wal-append.queue_time_us":366286}
I20260709 13:08:01.320899 21800 rpcz_store.cc:276] 0709 13:08:00.218595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.232469 (+ 13874us) service_pool.cc:224] Handling call
0709 13:08:00.257438 (+ 24969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:00.412289 (+154851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.239298 (+827009us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540060efd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:01.239472 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354776448}
W20260709 13:08:01.300495 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 242713us
I20260709 13:08:01.229408 25855 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:01.303378 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:01.321554 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:01.303648 21220 rpcz_store.cc:269] 0709 13:07:58.892312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.927607 (+ 35295us) service_pool.cc:224] Handling call
0709 13:08:01.192060 (+2264453us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54001caa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:01.192189 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":742973184}
I20260709 13:08:01.282558 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:01.311393 21212 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:01.320173 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:01.323863 21215 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:01.328495 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.323135 21072 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:08:01.324644 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:01.304263 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.329075 21220 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 7.
I20260709 13:08:01.247073 25759 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:01.330928 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:01.332036 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49544.1us
I20260709 13:08:01.332185 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.319588 21141 rpcz_store.cc:276] 0709 13:07:59.341732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.346656 (+  4924us) service_pool.cc:224] Handling call
0709 13:07:59.349216 (+  2560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.394283 (+ 45067us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:59.422306 (+ 28023us) log.cc:844] Serialized 14 byte log entry
0709 13:07:59.528899 (+106593us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:59.528954 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:59.529011 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.227824 (+1698813us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b54004236d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:01.227893 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:01.228072 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27482,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":86,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":101812992,"thread_start_us":105771,"threads_started":1,"wal-append.queue_time_us":342757,"wal-append.run_cpu_time_us":9794,"wal-append.run_wall_time_us":1287180}
W20260709 13:08:01.337396 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.337487 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:01.341934 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.342860 21213 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:01.343990 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:01.345368 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.267640 25859 raft_consensus.cc:2764] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:08:01.346787 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.347746 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:01.326973 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.355365 25795 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:01.360909 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.332242 25817 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.281728 25821 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:01.282025 25873 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:01.373649 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:01.385627 21216 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:08:01.388823 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48243.8us
I20260709 13:08:01.389008 21798 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:08:01.389299 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 218042us
I20260709 13:08:01.389102 21220 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:01.379848 21221 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:01.347172 25904 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:01.374639 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:01.382094 25817 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:01.370820 25953 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:08:01.306197 25495 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:01.391269 21221 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:01.382766 21069 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:08:01.357769 25932 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:01.346784 25878 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:01.402026 21216 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:08:01.402401 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:01.413542 21801 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:01.405490 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:01.390782 25853 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:01.391057 25835 raft_consensus.cc:2764] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:08:01.417160 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:01.417997 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:01.229182 25746 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.384078 25863 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.371989 25836 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:08:01.446300 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:01.449009 21216 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:01.456137 21218 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:08:01.406926 25495 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.454519 21069 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:01.385811 25833 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:08:01.371363 25756 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:08:01.457540 21216 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:08:01.461975 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 81108.3us
I20260709 13:08:01.412533 25878 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:01.391162 25636 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:08:01.467067 21069 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:01.404629 25821 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.468955 25898 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:01.469808 25821 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:01.469959 25903 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:08:01.472165 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 124470us
I20260709 13:08:01.473256 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:01.474942 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:01.450557 25902 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:01.475920 21069 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:01.481448 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:01.458971 25927 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:01.461584 25495 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:01.551879 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1486) took 1093 ms. Trace:
I20260709 13:08:01.552438 21063 rpcz_store.cc:276] 0709 13:08:00.458167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.037829 (+579662us) service_pool.cc:224] Handling call
0709 13:08:01.237926 (+200097us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:01.320363 (+ 82437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.551544 (+231181us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b540026eb50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:01.551793 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459228544}
I20260709 13:08:01.487317 21069 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:01.444165 25855 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:01.562647 21215 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:01.488704 25903 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:01.489413 25898 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:01.490661 21801 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:08:01.568336 21801 pending_rounds.cc:85] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 9
I20260709 13:08:01.568639 21801 pending_rounds.cc:107] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:08:01.568887 21801 raft_consensus.cc:2889] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:01.491700 21144 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:08:01.492723 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:01.495705 21141 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:08:01.474730 25858 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:01.490917 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:01.573544 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 82738.4us
I20260709 13:08:01.574572 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:01.498412 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:01.499825 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:01.575691 21140 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:01.508401 25759 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:01.509641 25960 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:08:01.511026 25875 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:01.515865 25706 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:01.516489 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:01.466741 25878 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:01.527815 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.530061 25853 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:01.473585 25636 consensus_queue.cc:1243] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:08:01.533183 25711 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:01.584599 25956 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:08:01.577600 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.444519 25826 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.578954 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:01.666170 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 87319.3us
I20260709 13:08:01.667439 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:01.581212 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:01.668905 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 87770.9us
I20260709 13:08:01.670214 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:01.480923 25864 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 13:08:01.496578 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:01.572899 25858 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:01.612354 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:01.587455 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 71167.7us
I20260709 13:08:01.577104 25930 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:01.733352 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:01.804152 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.580647 25833 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.086s
W20260709 13:08:01.691264 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:01.828987 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.659106 25853 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:01.583925 25789 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.089s
W20260709 13:08:01.811206 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 137377us
W20260709 13:08:01.673007 25950 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:08:01.744967 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:01.725919 21220 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:01.676090 25903 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:01.830034 25036 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:08:01.796222 25503 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:08:01.890196 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 424363us
I20260709 13:08:01.898267 21220 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:01.861268 25859 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:01.590051 25875 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:01.675611 25711 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:01.674335 25902 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:01.716660 25816 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:01.825840 25706 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:08:01.906214 21185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.012s	sys 0.056s
W20260709 13:08:01.920845 21185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.012s	sys 0.056s
W20260709 13:08:01.909435 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 164561us
I20260709 13:08:01.919009 21219 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:01.926893 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:01.929919 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:01.921228 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 524854us
I20260709 13:08:01.956420 21220 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:01.878906 25963 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:01.891990 25363 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:01.905236 25930 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:01.831089 25852 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
W20260709 13:08:01.922776 25905 consensus_meta.cc:287] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.799s	user 0.010s	sys 0.010s
I20260709 13:08:01.952147 21219 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:08:01.983053 21217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.015s	sys 0.034s
I20260709 13:08:01.755931 25858 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:01.960839 21220 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:08:01.984786 21217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.015s	sys 0.034s
I20260709 13:08:01.991183 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2812) took 1164 ms. Trace:
I20260709 13:08:01.991816 21217 rpcz_store.cc:276] 0709 13:08:00.825972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.924384 (+ 98412us) service_pool.cc:224] Handling call
0709 13:08:00.924612 (+   228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:00.924811 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:01.393383 (+468572us) log.cc:844] Serialized 14 byte log entry
0709 13:08:01.989749 (+596366us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:01.989801 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:01.989851 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.989916 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:01.990028 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56361,"prepare.queue_time_us":439373,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"thread_start_us":918893,"threads_started":2}
I20260709 13:08:01.969826 25930 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:01.995754 25758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.027s	sys 0.027s
I20260709 13:08:01.983861 25905 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:01.988453 25856 consensus_meta.cc:287] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.656s	user 0.011s	sys 0.006s
W20260709 13:08:01.986630 25933 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.003s
W20260709 13:08:01.999300 25758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.027s	sys 0.027s
I20260709 13:08:01.962427 25756 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:01.962992 25963 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:01.996129 25515 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:08:01.985215 25835 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.064s
I20260709 13:08:02.017575 25789 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:02.033896 25835 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.000s	sys 0.065s
I20260709 13:08:02.022406 25756 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:02.066465 25972 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:08:02.066844 25930 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:02.069294 25973 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:08:01.985823 25853 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.076910 25875 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.081070 25963 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.056648 25983 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:02.083391 25756 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:02.085213 25817 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:02.086421 25968 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.276s	user 0.004s	sys 0.003s
W20260709 13:08:02.091102 25873 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.707s	user 0.014s	sys 0.010s
W20260709 13:08:02.116492 25874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.011s	sys 0.018s
W20260709 13:08:02.146946 25987 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:02.119375 25889 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:02.156533 25889 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:02.165140 25889 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:02.167956 25878 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.005s
I20260709 13:08:02.168632 25878 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.171383 25878 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.167078 25889 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:02.228652 21067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.008s	sys 0.021s
W20260709 13:08:02.229650 21067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.008s	sys 0.021s
I20260709 13:08:01.869522 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1492) took 1064 ms. Trace:
I20260709 13:08:02.231531 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1494) took 1426 ms. Trace:
I20260709 13:08:02.232239 21066 rpcz_store.cc:276] 0709 13:08:00.804693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.306318 (+501625us) service_pool.cc:224] Handling call
0709 13:08:01.306554 (+   236us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:01.306765 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:01.306787 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:01.332829 (+ 26042us) log.cc:844] Serialized 26 byte log entry
0709 13:08:01.416686 (+ 83857us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:01.416740 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.416843 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:01.869247 (+452404us) raft_consensus.cc:1685] finished
0709 13:08:01.869274 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:01.869446 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":33659,"threads_started":1,"wal-append.queue_time_us":261794}
I20260709 13:08:02.232189 21067 rpcz_store.cc:276] 0709 13:08:00.805215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.310411 (+505196us) service_pool.cc:224] Handling call
0709 13:08:01.310767 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:01.348862 (+ 38095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:01.348906 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:01.349532 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:08:02.230470 (+880938us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:02.230626 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.230781 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:02.230906 (+   125us) raft_consensus.cc:1685] finished
0709 13:08:02.230933 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:02.231321 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37623,"raft.queue_time_us":45049,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":841333,"threads_started":1,"wal-append.queue_time_us":707811}
W20260709 13:08:01.845710 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.236519 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 789) took 1295 ms. Trace:
I20260709 13:08:02.237161 21070 rpcz_store.cc:276] 0709 13:08:00.973154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.332813 (+359659us) service_pool.cc:224] Handling call
0709 13:08:01.333090 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:01.347734 (+ 14644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:01.347782 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:01.371085 (+ 23303us) log.cc:844] Serialized 26 byte log entry
0709 13:08:01.618157 (+247072us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:01.618229 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.618353 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:02.235788 (+617435us) raft_consensus.cc:1685] finished
0709 13:08:02.235839 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:02.236286 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22715,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":88,"thread_start_us":246677,"threads_started":1,"wal-append.queue_time_us":624825}
W20260709 13:08:02.240646 21147 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.009s	sys 0.046s
W20260709 13:08:02.241209 21147 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.010s	sys 0.047s
I20260709 13:08:02.241437 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1489) took 1778 ms. Trace:
W20260709 13:08:02.244222 21215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.002s	sys 0.013s
W20260709 13:08:02.244836 21215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.002s	sys 0.013s
I20260709 13:08:02.246721 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1536) took 1110 ms. Trace:
W20260709 13:08:01.879711 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:02.248404 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:01.889955 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.989328 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:02.253904 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 290769us
I20260709 13:08:02.254405 21215 rpcz_store.cc:276] 0709 13:08:01.135869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.567926 (+432057us) service_pool.cc:224] Handling call
0709 13:08:01.568319 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:01.568718 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:01.568745 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:01.577868 (+  9123us) log.cc:844] Serialized 52 byte log entry
0709 13:08:02.245863 (+667995us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:02.246007 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.246179 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:02.246279 (+   100us) raft_consensus.cc:1685] finished
0709 13:08:02.246298 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:02.246590 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99343,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":43685,"thread_start_us":576791,"threads_started":1,"wal-append.queue_time_us":532820}
W20260709 13:08:02.258623 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:02.262596 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.971180 25363 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.267984 25363 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:02.267912 21213 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
W20260709 13:08:01.990185 21216 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.025s	sys 0.061s
W20260709 13:08:02.269168 21216 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.026s	sys 0.061s
I20260709 13:08:02.269965 25363 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.271574 25363 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:02.275810 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.016775 25374 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.275930 25363 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:02.305464 21767 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.005s	sys 0.005s
W20260709 13:08:02.122232 25874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.011s	sys 0.019s
W20260709 13:08:02.324504 21767 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.005s	sys 0.006s
I20260709 13:08:02.241972 21071 rpcz_store.cc:276] 0709 13:08:00.463066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.037628 (+574562us) service_pool.cc:224] Handling call
0709 13:08:01.078163 (+ 40535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:01.078583 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:01.078624 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:01.079131 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:08:01.403348 (+324217us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:01.403387 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.403448 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:02.071564 (+668116us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:02.240673 (+169109us) raft_consensus.cc:1685] finished
0709 13:08:02.240719 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:02.241202 (+   483us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7050,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":7307,"spinlock_wait_cycles":49033600,"thread_start_us":321796,"threads_started":1,"wal-append.queue_time_us":508667}
I20260709 13:08:02.333606 25374 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.064205 25789 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.091895 25860 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:08:02.355988 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:02.356503 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 737571us
I20260709 13:08:02.357402 25927 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.357863 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.358340 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.359122 21147 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:02.363394 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.363458 25898 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:02.363581 21139 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 9.
I20260709 13:08:02.367996 25995 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:02.358390 25927 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.368903 25995 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.370826 25927 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.371591 25995 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.372740 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:02.373420 25995 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.374082 25706 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:08:02.374586 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 72276.8us
I20260709 13:08:02.374702 25706 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:02.373047 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.021s	sys 0.016s
I20260709 13:08:02.375454 25706 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:02.375671 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.021s	sys 0.016s
W20260709 13:08:02.376863 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 580794us
I20260709 13:08:02.374464 25995 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.379465 25995 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:02.381486 25878 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.382470 25878 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.385350 25878 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:02.387723 25889 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:02.390008 25981 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:02.391068 25981 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.394201 25981 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:02.393913 25889 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:02.394905 25711 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:02.396298 25980 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:02.397116 25980 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:02.403728 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:02.410820 25860 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:02.413767 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.413174 25860 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:08:02.414354 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.444737 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:02.446136 25853 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:02.446857 25853 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:02.446969 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:02.465489 25789 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:02.467249 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:02.468289 21036 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.021s	sys 0.029s
W20260709 13:08:02.468732 21036 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.021s	sys 0.029s
W20260709 13:08:02.469249 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 610670us
I20260709 13:08:02.470433 25817 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.471588 25817 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.472980 25860 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:02.476085 25853 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.476792 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1481) took 2025 ms. Trace:
I20260709 13:08:02.477334 21068 rpcz_store.cc:276] 0709 13:08:00.451559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.898538 (+446979us) service_pool.cc:224] Handling call
0709 13:08:01.175118 (+276580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:01.248788 (+ 73670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.476328 (+1227540us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b540019d2d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:02.476601 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634635008}
I20260709 13:08:02.480773 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1516) took 3260 ms. Trace:
I20260709 13:08:02.481460 21214 rpcz_store.cc:276] 0709 13:07:59.220191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.533477 (+313286us) service_pool.cc:224] Handling call
0709 13:08:01.192230 (+1658753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:01.304527 (+112297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.479966 (+1175439us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54001caa50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:02.480507 (+   541us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3713627904}
I20260709 13:08:02.482884 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:02.485673 25979 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:02.398888 25980 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.486662 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:02.492080 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49859.8us
I20260709 13:08:02.492843 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:02.474989 25817 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.513084 25746 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:02.514067 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2824) took 1531 ms. Trace:
I20260709 13:08:02.514636 25816 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:02.514533 21212 rpcz_store.cc:276] 0709 13:08:00.982947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.034103 (+ 51156us) service_pool.cc:224] Handling call
0709 13:08:02.513501 (+1479398us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54004434d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:02.513802 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577547520}
I20260709 13:08:02.515486 25816 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:02.521220 25706 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.522751 25934 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:02.495240 25833 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:08:02.519016 25826 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.004s	sys 0.016s
I20260709 13:08:02.501544 25995 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.503602 25789 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:02.522027 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3002) took 1920 ms. Trace:
I20260709 13:08:02.513113 25925 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 13:08:02.543192 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 47911us
I20260709 13:08:02.557001 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1463) took 3023 ms. Trace:
W20260709 13:08:02.556114 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69733.7us
I20260709 13:08:02.555756 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:02.556022 21145 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:02.545459 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1521) took 1509 ms. Trace:
I20260709 13:08:02.558986 21139 rpcz_store.cc:276] 0709 13:08:00.601818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.637919 (+ 36101us) service_pool.cc:224] Handling call
0709 13:08:02.521696 (+1883777us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b540030aa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:02.521875 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3929362176}
I20260709 13:08:02.557914 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:02.583706 21218 rpcz_store.cc:276] 0709 13:08:01.036481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.390524 (+354043us) service_pool.cc:224] Handling call
0709 13:08:01.390858 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:01.458460 (+ 67602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.544824 (+1086364us) spinlock_profiling.cc:243] Waited 58.8 ms on lock 0x7b5400101dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:02.545269 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135284480}
W20260709 13:08:02.550871 25826 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.005s	sys 0.016s
I20260709 13:08:02.588958 21146 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:02.563971 21065 rpcz_store.cc:276] 0709 13:07:59.533604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.412285 (+878681us) service_pool.cc:224] Handling call
0709 13:08:00.412602 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:01.324409 (+911807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.556410 (+1232001us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b540026f054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:02.556727 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2083837312}
I20260709 13:08:02.580842 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:02.534920 25999 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.069s	user 0.001s	sys 0.001s
I20260709 13:08:02.598635 21143 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:08:02.589296 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1535) took 1464 ms. Trace:
I20260709 13:08:02.535555 25706 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:02.557016 25789 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.611658 25930 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:02.614264 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.615358 25789 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:02.618736 26007 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:08:02.619266 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2833) took 1582 ms. Trace:
I20260709 13:08:02.619701 21221 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:02.619799 21213 rpcz_store.cc:276] 0709 13:08:01.037259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.345093 (+307834us) service_pool.cc:224] Handling call
0709 13:08:02.618949 (+1273856us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b54000bbdd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:02.619181 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2119210752}
I20260709 13:08:02.535627 25856 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:08:02.621757 21219 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:02.622479 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1542) took 1444 ms. Trace:
I20260709 13:08:02.540318 25933 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:08:02.622913 21221 rpcz_store.cc:276] 0709 13:08:01.177653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.825362 (+647709us) service_pool.cc:224] Handling call
0709 13:08:02.619241 (+793879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:02.620260 (+  1019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.622223 (+  1963us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b54000bbdd0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:02.622385 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1824622848}
I20260709 13:08:02.624432 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1552) took 1159 ms. Trace:
I20260709 13:08:02.624984 21219 rpcz_store.cc:276] 0709 13:08:01.464970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.969628 (+504658us) service_pool.cc:224] Handling call
0709 13:08:01.987626 (+ 17998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:02.622393 (+634767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.624004 (+  1611us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b540034b2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:02.624360 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1456721664}
I20260709 13:08:02.548959 25878 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:08:02.558660 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:02.541607 25816 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.555065 25995 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.608734 21072 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:02.544072 25994 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 13:08:02.581929 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0003593980000005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 13:08:02.548760 25758 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:02.588912 25817 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.627022 25826 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:02.600838 21216 rpcz_store.cc:276] 0709 13:08:01.125173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.461554 (+336381us) service_pool.cc:224] Handling call
0709 13:08:01.461898 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:02.269932 (+808034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:02.269992 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:02.271144 (+  1152us) log.cc:844] Serialized 26 byte log entry
0709 13:08:02.457170 (+186026us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:02.457482 (+   312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.460677 (+  3195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:02.558860 (+ 98183us) raft_consensus.cc:1685] finished
0709 13:08:02.558880 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:02.589214 (+ 30334us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52923,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":952089,"threads_started":2,"wal-append.queue_time_us":241712}
I20260709 13:08:02.667824 25856 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.668848 25856 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:08:02.671520 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 91112.4us
I20260709 13:08:02.671864 25706 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:02.672369 25706 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.672817 25706 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:02.672915 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 47243.6us
I20260709 13:08:02.673072 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:02.675588 21792 raft_consensus.cc:2393] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 15: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:02.676503 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 788) took 1826 ms. Trace:
I20260709 13:08:02.676970 21072 rpcz_store.cc:276] 0709 13:08:00.850456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.329849 (+479393us) service_pool.cc:224] Handling call
0709 13:08:01.330110 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:02.675342 (+1345232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.676151 (+   809us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003bf954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:02.676419 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2939452160}
I20260709 13:08:02.678491 25995 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.680455 25363 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:02.554394 25852 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 13:08:02.652055 25826 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:02.687810 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.685367 25515 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:02.550307 25833 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.113s
W20260709 13:08:02.693347 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 47145us
I20260709 13:08:02.701726 25503 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:02.702513 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.705760 25878 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.706519 25878 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.708551 21143 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:02.708768 25925 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:02.709127 25878 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.683136 25817 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.709385 25925 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.711829 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.712551 21147 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:02.715072 25927 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.715740 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:02.715896 25927 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:02.718539 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.719358 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:02.720064 21071 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:02.720010 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:02.721097 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:02.718449 20967 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 224) took 35642 ms (client timeout 15000 ms). Trace:
W20260709 13:08:02.687172 25756 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.009s
I20260709 13:08:02.671542 25758 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.736074 21066 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:02.685410 25852 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:02.711654 25817 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:02.744473 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.680495 25874 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:02.697796 25515 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:02.756979 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:02.726176 25789 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.686750 25826 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:02.723233 20967 rpcz_store.cc:269] 0709 13:07:27.075768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.076016 (+   248us) service_pool.cc:224] Handling call
0709 13:08:02.718113 (+35642097us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:51.412920 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:51.570828 (+157908us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b5c001e9c8c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f4870e331 00007f7f4870e221 00007f7f48708f6c 00007f7f45f66d69 00007f7f45f520a6 00007f7f45f51d58 00007f7f45f569f1 00007f7f45f569a9 00007f7f45f56939 00007f7f45f56901 00007f7f45f55b2d
0709 13:07:51.571064 (+   236us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:51.571139 (+    75us) write_op.cc:435] Acquired schema lock
0709 13:07:51.571186 (+    47us) tablet.cc:662] Decoding operations
0709 13:07:51.665559 (+ 94373us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:51.665653 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:51.665694 (+    41us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:07:51.772400 (+106706us) tablet.cc:701] Row locks acquired
0709 13:07:51.772447 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:07:51.772702 (+   255us) write_op.cc:270] Start()
0709 13:07:51.772974 (+   272us) write_op.cc:276] Timestamp: P: 1783602471772605 usec, L: 0
0709 13:07:51.773035 (+    61us) op_driver.cc:348] REPLICATION: starting
0709 13:07:51.804288 (+ 31253us) log.cc:844] Serialized 9128 byte log entry
0709 13:07:52.398590 (+594302us) op_driver.cc:464] REPLICATION: finished
0709 13:07:52.670544 (+271954us) write_op.cc:301] APPLY: starting
0709 13:07:52.670661 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:53.053569 (+382908us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:53.053611 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:54.072023 (+1018412us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:54.110468 (+ 38445us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:07:54.110533 (+    65us) write_op.cc:312] APPLY: finished
0709 13:07:54.111078 (+   545us) log.cc:844] Serialized 204 byte log entry
0709 13:07:54.268565 (+157487us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:54.286016 (+ 17451us) write_op.cc:454] Released schema lock
0709 13:07:54.286105 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:54.290284 (+  4179us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:54.290313 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2262380,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":268208,"apply.run_cpu_time_us":104527,"apply.run_wall_time_us":1619710,"cfile_cache_hit":90,"cfile_cache_hit_bytes":94095,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":19312,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":10,"num_ops":23,"prepare.queue_time_us":786495,"prepare.run_cpu_time_us":22262,"prepare.run_wall_time_us":513713,"raft.queue_time_us":115125,"raft.run_cpu_time_us":37620,"raft.run_wall_time_us":236248,"replication_time_us":625350,"spinlock_wait_cycles":13929856,"thread_start_us":1302554,"threads_started":5,"wal-append.queue_time_us":488134,"wal-append.run_cpu_time_us":21150,"wal-append.run_wall_time_us":2293717}]]}
I20260709 13:08:02.767359 21072 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:02.767774 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.768608 25919 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:02.769517 25762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.225s	user 0.009s	sys 0.019s
I20260709 13:08:02.769258 20966 catalog_manager.cc:5697] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:02.744915 25852 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:02.772547 25789 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.774497 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:02.775319 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.777417 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.777716 25706 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.840656 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:02.779506 21067 raft_consensus.cc:2410] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:08:02.842501 25706 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:02.843376 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.844033 21147 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:02.845054 25441 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:02.781209 21144 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:02.781965 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:02.779635 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:02.847546 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.787786 25856 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.788611 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1005) took 1611 ms. Trace:
I20260709 13:08:02.796108 25816 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.797163 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.702524 25503 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.746057 25756 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.689566 25995 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:08:02.802541 25036 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.804208 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:02.805225 25925 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.809648 25933 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:02.810643 25927 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:02.811381 25495 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.212s	user 0.012s	sys 0.008s
W20260709 13:08:02.817273 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 322263us
W20260709 13:08:02.821983 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56240.7us
I20260709 13:08:02.765152 25889 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:02.822235 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.827621 21141 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:08:02.817219 25762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.283s	user 0.010s	sys 0.020s
I20260709 13:08:02.829430 25515 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.832069 25789 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:02.832932 25374 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:02.834035 25978 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.842s	user 0.004s	sys 0.013s
W20260709 13:08:02.840090 25858 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.841s	user 0.012s	sys 0.006s
I20260709 13:08:02.839854 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:02.851744 21145 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:02.849102 21794 rpcz_store.cc:276] 0709 13:08:01.177368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.177523 (+   155us) service_pool.cc:224] Handling call
0709 13:08:01.177829 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:01.264214 (+ 86385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.741995 (+1477781us) spinlock_profiling.cc:243] Waited 6.3 ms on lock 0x7b54003bb854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:02.788423 (+ 46428us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14493056}
I20260709 13:08:02.864362 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:02.858866 25036 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.848439 25441 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:02.877880 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 133426us
I20260709 13:08:02.877040 21138 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
W20260709 13:08:02.882261 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:02.883919 26009 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:08:02.884618 25852 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.885744 25852 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:02.889079 25503 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:02.891527 20966 catalog_manager.cc:5697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:02.904289 20966 catalog_manager.cc:5697] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:02.907505 25925 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:02.908486 25925 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.910476 25826 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:02.907644 21072 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:02.879209 25816 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.911043 25925 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:02.876700 25878 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:02.873692 25495 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:02.930867 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 126958us
W20260709 13:08:02.928622 26015 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:02.929971 25756 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:02.906445 20966 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 10 to 11, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 11 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:02.943819 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:02.944355 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.944437 20966 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 8 to 9, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 9 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:02.931515 26010 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.946259 25878 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.946527 26010 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.926457 25374 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.948869 25878 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:02.932113 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:02.949488 26014 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:02.949322 25374 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.933478 25036 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:02.892884 25515 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:02.950349 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.951762 25935 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:02.932837 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:02.955766 25816 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:02.957589 25758 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:02.881834 25858 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:02.962302 21794 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:08:02.963443 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.964144 25746 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:02.930966 21036 leader_election.cc:336] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.077s (SENT)
I20260709 13:08:02.968173 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.969614 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:02.970160 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:02.970656 21144 raft_consensus.cc:2410] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 20 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:08:02.971387 21070 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:02.972077 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:02.943626 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:02.974766 21795 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:02.941737 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:02.964138 21143 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:03.002511 21071 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:03.015218 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:03.020900 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44240us
W20260709 13:08:03.026593 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 107979us
W20260709 13:08:03.043901 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 73875.2us
I20260709 13:08:03.021329 25758 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:03.048385 21767 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:03.028829 25858 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:03.050092 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.055733 25836 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:08:03.056631 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:03.056921 25878 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.034829 25995 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:03.074333 25789 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:03.074892 25789 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:03.079622 21795 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:03.076792 21767 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:03.018688 25762 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.049s
W20260709 13:08:03.082321 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.074517 25995 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:03.083675 25836 consensus_queue.cc:1243] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:08:03.083916 21792 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 9: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:03.070071 25984 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:03.075340 25789 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:02.946918 20966 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 8 to 12, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.079201 25817 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.093149 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:03.093281 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:03.097474 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.079407 25374 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:03.086652 25833 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:03.075727 25706 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:03.076637 25363 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 13:08:03.107052 26023 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:08:03.103668 25878 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:03.105825 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 56759.5us
W20260709 13:08:03.105967 26013 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.139s	user 0.001s	sys 0.002s
I20260709 13:08:03.105410 21794 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:03.088670 26010 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:03.094733 25927 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:03.121261 25816 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:03.121119 25817 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:03.122176 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65642us
I20260709 13:08:03.104421 25833 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.151511 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.127597 25706 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:03.152590 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67774.3us
I20260709 13:08:03.141268 25898 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:03.153096 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:03.153353 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:03.153641 21216 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:03.154603 21218 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:03.155330 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.156314 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.157188 25817 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:03.163759 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.135581 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.174364 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.178411 21146 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:03.173585 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.174580 21800 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:08:03.178447 21072 raft_consensus.cc:2368] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14.
I20260709 13:08:03.173698 21799 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:03.182420 21066 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:03.171795 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.191749 21221 raft_consensus.cc:2410] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:08:03.192567 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:03.179267 21065 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:03.200465 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.202621 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.203495 21067 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:03.163259 25833 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:03.127941 25984 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:03.172596 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.206033 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:03.211517 21070 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:08:03.164628 25898 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.160012 26026 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:03.174350 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.175812 26022 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:03.214982 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.172385 25758 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:03.219015 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:03.214383 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.217296 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:03.178887 25995 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:03.168036 25495 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:03.184098 25817 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:03.220999 21143 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:03.222980 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:03.233217 25935 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:03.233716 25935 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:03.234068 25935 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:03.236079 21801 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:03.229651 25816 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:03.237424 25816 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.237789 25816 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:03.238692 26022 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:03.241350 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:03.241968 21138 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:03.256278 21792 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:03.174451 25441 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:03.264535 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.244158 26026 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:03.277647 26032 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:03.283381 25395 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.284036 25395 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:03.284626 20966 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 9 to 10, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 10 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:03.288178 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:03.295073 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:03.296972 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 104593us
I20260709 13:08:03.251741 25817 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:03.289248 20966 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.301551 20966 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 9 to 13, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 13 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.303014 21142 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:03.305949 25898 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:03.298715 25935 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:03.308504 25706 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:03.309638 25935 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:03.311015 20966 catalog_manager.cc:5697] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 7 to 8, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } }
I20260709 13:08:03.318109 21794 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:03.320065 21138 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:03.312846 20966 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 6 to 7, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 7 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.258121 25758 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:03.364863 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:03.388943 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.401736 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.401337 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:03.402899 21216 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: [14.6-14.6]
I20260709 13:08:03.388391 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.387832 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.387099 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:03.403903 21798 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:03.336661 25878 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:03.412317 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.414441 21143 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:08:03.417074 21221 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:03.419214 21070 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:03.419719 20966 catalog_manager.cc:5697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:03.421268 20966 catalog_manager.cc:5697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 4 to 9, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:03.386739 25984 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:03.355778 25706 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.396799 25758 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:03.430230 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:03.435384 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 230135us
I20260709 13:08:03.437095 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.437547 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:03.438017 21216 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:08:03.437259 20966 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.344549 25789 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:03.441373 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:03.350186 26029 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 13:08:03.439252 20966 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 6 to 8, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 8 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.443507 26032 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:03.444033 20966 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.417681 26022 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:03.446149 20966 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.414569 25889 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.449278 21796 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
W20260709 13:08:03.453754 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.462814 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.467769 21216 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:03.463011 21035 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:03.468643 21072 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:08:03.468863 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 64266us
W20260709 13:08:03.464018 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59385.5us
I20260709 13:08:03.464468 25706 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:03.450747 25833 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:03.466295 25789 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:03.470654 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:03.480027 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:03.414830 25495 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:08:03.484015 21072 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:08:03.487089 21221 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:03.470604 25930 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:03.405543 25826 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:08:03.485481 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.490420 21068 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:03.474252 25927 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:03.496328 26034 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:08:03.496908 26011 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.553s	user 0.004s	sys 0.006s
I20260709 13:08:03.499209 25706 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:03.471553 25889 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:03.499718 21215 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:03.500197 20966 catalog_manager.cc:5697] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 8 to 9, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:08:03.502548 25852 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.005s
I20260709 13:08:03.503366 26032 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:03.503568 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.507733 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:03.518540 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.524354 25826 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:03.510502 25927 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:03.531240 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 58093.5us
I20260709 13:08:03.537497 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.557183 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.558099 21216 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:08:03.555034 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.472129 26026 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:03.534574 25927 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:03.562247 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.560976 21215 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:03.555580 25853 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:03.578403 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.564433 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:03.561431 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 91005.7us
I20260709 13:08:03.585427 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.586848 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.587173 21035 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:03.588999 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:03.563686 25833 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:03.578629 25853 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:03.591768 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.626696 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.597262 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.640584 21216 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:08:03.640722 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53585.9us
I20260709 13:08:03.641485 21035 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:03.643978 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 83233.2us
I20260709 13:08:03.662809 26047 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:03.637118 21215 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:03.663340 21218 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 6.
I20260709 13:08:03.593014 26032 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:03.677889 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:03.670948 25919 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.854s	user 0.014s	sys 0.003s
W20260709 13:08:03.680665 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.663484 26047 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.663205 25853 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:03.684998 25853 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:03.676311 25878 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:08:03.690171 25994 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:03.696236 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.698029 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.697219 21797 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:03.669370 25495 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:03.699767 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:03.699158 21215 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:03.701407 21221 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:03.735775 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:03.698979 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:03.737418 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.680128 26032 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:03.746069 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.746512 21068 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:03.748934 25963 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.749320 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:03.749614 25963 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:03.669838 20966 catalog_manager.cc:5697] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 10 to 12, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.752487 20966 catalog_manager.cc:5697] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:08:03.754292 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 74041.5us
I20260709 13:08:03.755631 21801 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:03.756054 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:03.728751 25935 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:08:03.755465 25878 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:03.693805 26047 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:03.774574 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.778729 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.777386 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:03.778826 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:03.695719 25934 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:08:03.760100 26032 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:03.781427 21212 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:03.728447 25930 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:08:03.798276 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:03.784989 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 90462.6us
W20260709 13:08:03.787091 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 335198us
I20260709 13:08:03.728614 25852 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:08:03.758725 25863 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:08:03.799481 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 107752us
I20260709 13:08:03.776530 25889 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:03.819528 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:03.789374 26045 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:08:03.826826 21212 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:03.754628 20966 catalog_manager.cc:5697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.818666 26022 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:03.822690 25816 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.005s
I20260709 13:08:03.836982 21111 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:03.838575 21212 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:03.841425 21216 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:03.842696 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:03.844323 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:03.782897 25984 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:08:03.854096 25495 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:03.854458 26057 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:03.857414 21213 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:03.861325 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:03.888410 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:03.889029 21216 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:03.862454 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:03.823201 26029 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:08:03.896574 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.899024 21065 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:08:03.855935 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 100009us
I20260709 13:08:03.900478 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:03.883988 26045 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:03.904472 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:03.913616 21071 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:03.904183 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.921638 21216 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:03.927017 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:03.927978 21064 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:03.904983 26026 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:08:03.936115 26026 consensus_queue.cc:1243] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:08:03.849644 26022 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:03.941252 26022 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:03.950124 21066 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:03.848555 25816 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:03.951238 21070 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:03.897168 26057 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.888260 26054 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:08:03.965822 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.883441 25994 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:08:03.976593 21794 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:03.967418 21213 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:03.977720 21213 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:03.960637 21072 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:03.953941 25930 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:08:03.980638 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 248044us
I20260709 13:08:03.953734 26028 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:04.030716 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:03.980165 26048 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.191s	user 0.001s	sys 0.001s
I20260709 13:08:03.969278 26057 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:04.034502 21213 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.058712 25441 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:04.072875 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 106974us
I20260709 13:08:04.027966 25930 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:04.073756 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:04.079396 21213 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:04.083873 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:04.085258 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:04.081038 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 176634us
W20260709 13:08:04.081837 25927 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.503s	user 0.005s	sys 0.004s
I20260709 13:08:04.099997 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:04.081192 25930 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:04.067960 26053 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:08:04.093338 25919 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:04.091883 26028 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.094419 26017 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:04.087623 26022 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:04.092067 25789 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:04.088061 25706 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:08:04.160178 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 85688us
W20260709 13:08:04.163494 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 67200us
W20260709 13:08:04.165400 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 335252us
I20260709 13:08:04.144562 25495 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.175467 25759 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
W20260709 13:08:04.177089 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 146498us
I20260709 13:08:04.178694 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:04.179766 25934 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.180348 25934 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.155611 25878 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:04.158754 26028 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.185801 26028 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.158953 26017 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.189687 26017 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:04.148278 25919 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:04.204303 21801 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:04.107868 25927 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:04.222800 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 157762us
I20260709 13:08:04.224900 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:04.199694 25833 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:08:04.232402 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65922.1us
I20260709 13:08:04.208423 25919 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:04.248296 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:04.248400 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:04.169410 25789 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.255734 25789 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:04.257457 21067 raft_consensus.cc:2393] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:04.257977 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:04.261497 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:04.249250 21140 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:04.265636 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:04.232903 25878 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:04.235567 25833 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:04.268738 21139 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:08:04.266288 21144 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:08:04.269487 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:04.262408 21147 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:08:04.245537 26068 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:04.283411 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 117635us
I20260709 13:08:04.284737 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:04.285979 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:04.264570 25816 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:04.294644 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:04.296756 21140 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:04.298202 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:04.270246 25878 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.275502 25833 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:04.303395 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:04.288146 25817 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.334052 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:04.341399 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1015) took 1184 ms. Trace:
I20260709 13:08:04.341943 21797 rpcz_store.cc:276] 0709 13:08:03.156970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.173973 (+ 17003us) service_pool.cc:224] Handling call
0709 13:08:04.341134 (+1167161us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540040a1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:04.341289 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106639104}
I20260709 13:08:04.345153 25395 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:04.355825 25934 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.356629 25934 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.357216 25934 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:04.364743 26066 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:08:04.368260 26035 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.951s	user 0.007s	sys 0.006s
I20260709 13:08:04.371862 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1561) took 1220 ms. Trace:
I20260709 13:08:04.372346 21217 rpcz_store.cc:276] 0709 13:08:03.150822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.156850 (+  6028us) service_pool.cc:224] Handling call
0709 13:08:03.157241 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:03.157806 (+   565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:03.157834 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:03.158365 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:08:03.242597 (+ 84232us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:03.242637 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:03.242701 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:04.367699 (+1124998us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b24001e5798. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f2b59c4 00007f7f3f2b3acb 00007f7f3f46edc9 00007f7f3f46ea55 00007f7f3f46e09d 00007f7f420a010e 00007f7f420b788c 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221
0709 13:08:04.371372 (+  3673us) raft_consensus.cc:1685] finished
0709 13:08:04.371397 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:04.371642 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35853,"raft.queue_time_us":5717,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":10422,"spinlock_wait_cycles":52763520,"thread_start_us":83763,"threads_started":1,"wal-append.queue_time_us":137088}
I20260709 13:08:04.375990 25935 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:04.376533 25935 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:04.377640 25935 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:04.384258 21035 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:04.385236 25935 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:04.306627 25878 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:04.294070 26060 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:08:04.269114 26074 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
W20260709 13:08:04.400063 26062 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:08:04.307516 26077 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:04.289716 26047 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.387140 25963 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:04.290468 26068 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.317739 26070 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.389869 25816 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:04.443708 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 161819us
I20260709 13:08:04.388741 25817 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.317224 26059 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:04.453114 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:04.468183 21072 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:04.468333 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:04.406131 21066 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:04.469043 21065 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:04.429582 25934 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.406694 25826 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:08:04.446933 26052 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.338s	user 0.002s	sys 0.002s
I20260709 13:08:04.444034 25789 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:08:04.499577 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 249068us
I20260709 13:08:04.478879 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:04.451393 26072 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:04.390125 26063 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.153s
I20260709 13:08:04.548450 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:04.553170 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.556908 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 625) took 1050 ms. Trace:
I20260709 13:08:04.474243 25817 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.476604 25826 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:04.572834 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 347987us
W20260709 13:08:04.576990 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 99869us
I20260709 13:08:04.571122 21213 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
W20260709 13:08:04.578454 21071 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.001s
I20260709 13:08:04.579216 21071 raft_consensus.cc:2410] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 15 because replica has last-logged OpId of term: 14 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 13:08:04.579509 21213 pending_rounds.cc:85] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 8
W20260709 13:08:04.580089 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 246114us
I20260709 13:08:04.559312 25898 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.558709 25789 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:04.509472 26060 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:04.598748 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:04.565039 21218 rpcz_store.cc:276] 0709 13:08:03.506299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.797688 (+291389us) service_pool.cc:224] Handling call
0709 13:08:03.824612 (+ 26924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:03.898225 (+ 73613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:03.898258 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:03.913768 (+ 15510us) log.cc:844] Serialized 26 byte log entry
0709 13:08:04.231852 (+318084us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:04.336847 (+104995us) log.cc:844] Serialized 14 byte log entry
0709 13:08:04.556035 (+219188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:04.556444 (+   409us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:04.556534 (+    90us) raft_consensus.cc:1685] finished
0709 13:08:04.556550 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:04.556758 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70850,"prepare.queue_time_us":18768,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":155211,"spinlock_wait_cycles":74048512,"thread_start_us":533574,"threads_started":3,"wal-append.queue_time_us":348430}
I20260709 13:08:04.562474 25927 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.562592 26072 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:04.597888 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:04.580482 21213 pending_rounds.cc:107] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:08:04.614104 21213 raft_consensus.cc:2889] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:04.565583 26075 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:04.576965 25826 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:04.463683 26070 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:04.621191 26070 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:04.622079 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:04.624907 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:04.643565 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 91266.5us
I20260709 13:08:04.609508 25789 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.633129 26077 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:04.650434 21111 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:04.636094 25889 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:04.666023 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:04.664402 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:04.669431 21797 raft_consensus.cc:2410] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:08:04.676565 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1022) took 1008 ms. Trace:
I20260709 13:08:04.677047 21798 rpcz_store.cc:276] 0709 13:08:03.667908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.668013 (+   105us) service_pool.cc:224] Handling call
0709 13:08:03.672216 (+  4203us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:03.673820 (+  1604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:03.673848 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:03.674316 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:08:03.856765 (+182449us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:03.856807 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:03.856872 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:04.202654 (+345782us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:04.663255 (+460601us) raft_consensus.cc:1685] finished
0709 13:08:04.663299 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:04.676485 (+ 13186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1159,"raft.queue_time_us":2314,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":103,"spinlock_wait_cycles":4916224,"thread_start_us":181711,"threads_started":1,"wal-append.queue_time_us":397991}
I20260709 13:08:04.680451 26068 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:04.692627 25817 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:04.693821 26076 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:08:04.695410 25853 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:04.696970 25853 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:04.697331 25853 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.697669 25853 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:04.699359 26070 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:04.699795 26070 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:04.700145 26070 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:04.636929 26060 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.708038 26047 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.708778 26047 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:04.709090 26060 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:04.666960 26085 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:08:04.653354 26077 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:04.684428 25441 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.004s
I20260709 13:08:04.709141 26047 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:04.684212 26032 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.865s	user 0.013s	sys 0.000s
I20260709 13:08:04.653779 26096 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:04.747491 21797 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:04.750972 25789 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:04.751668 25789 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.752103 25789 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.753945 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 806) took 1073 ms. Trace:
I20260709 13:08:04.754354 21067 rpcz_store.cc:276] 0709 13:08:03.680957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.681121 (+   164us) service_pool.cc:224] Handling call
0709 13:08:04.753738 (+1072617us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b5400215054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:04.753868 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1169560448}
I20260709 13:08:04.754969 26105 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.755543 26105 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.758169 25863 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:04.703094 25919 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.743400 26096 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:04.671201 25889 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.721431 26077 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:04.741531 26047 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:04.759262 25863 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:04.760843 26105 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:04.785287 25863 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.786247 25878 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:04.721802 26017 consensus_meta.cc:287] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.532s	user 0.004s	sys 0.007s
I20260709 13:08:04.773597 26103 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:04.772181 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2571) took 1827 ms. Trace:
I20260709 13:08:04.779367 26096 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:04.770550 26069 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.463s	user 0.000s	sys 0.002s
I20260709 13:08:04.802947 21143 rpcz_store.cc:276] 0709 13:08:02.945119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.227140 (+282021us) service_pool.cc:224] Handling call
0709 13:08:03.413149 (+186009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:03.414949 (+  1800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:04.771936 (+1356987us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540020add0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:04.772086 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426922880}
I20260709 13:08:04.792465 21072 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
W20260709 13:08:04.762534 26033 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.351s	user 0.004s	sys 0.010s
W20260709 13:08:04.766775 26078 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 13:08:04.767802 26082 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 13:08:04.776831 25756 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:08:04.766520 25994 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:08:04.766062 26044 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.286s	user 0.005s	sys 0.009s
W20260709 13:08:04.784449 26057 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.000s
I20260709 13:08:04.837442 25994 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:04.764611 26106 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:08:04.837808 26057 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.838477 25994 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.839892 25994 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:04.840608 26057 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:04.766142 25930 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.615s	user 0.008s	sys 0.000s
I20260709 13:08:04.766307 26032 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.841688 25930 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.843501 26032 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:04.844029 25930 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.844739 21145 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:04.805071 26017 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.852348 21145 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:04.852993 25826 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.771255 25833 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.854780 26070 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.767354 25919 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:04.773249 25889 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.760527 26097 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:04.859200 21070 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:08:04.857244 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1559) took 1707 ms. Trace:
I20260709 13:08:04.859297 21795 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 13:08:04.867141 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 269342us
I20260709 13:08:04.858469 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:04.874154 21801 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:04.871050 21214 rpcz_store.cc:276] 0709 13:08:03.150145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.154291 (+  4146us) service_pool.cc:224] Handling call
0709 13:08:03.154623 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:03.154901 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:03.154925 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:03.155484 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:08:03.195031 (+ 39547us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:03.195073 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:03.195143 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:04.762277 (+1567134us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b24001e5948. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f2b59c4 00007f7f3f2b3acb 00007f7f3f46edc9 00007f7f3f46ea55 00007f7f3f46e09d 00007f7f420a010e 00007f7f420b788c 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221
0709 13:08:04.856927 (+ 94650us) raft_consensus.cc:1685] finished
0709 13:08:04.856951 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:04.857174 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41868544,"thread_start_us":38839,"threads_started":1,"wal-append.queue_time_us":164601}
I20260709 13:08:04.889752 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1017) took 1541 ms. Trace:
I20260709 13:08:04.890358 21794 rpcz_store.cc:276] 0709 13:08:03.348719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.348848 (+   129us) service_pool.cc:224] Handling call
0709 13:08:04.889453 (+1540605us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b5400633dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:04.889674 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309122816}
I20260709 13:08:04.884591 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:04.855208 25826 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.855877 25863 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:04.898689 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 616) took 1411 ms. Trace:
I20260709 13:08:04.904379 21220 rpcz_store.cc:276] 0709 13:08:03.488120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.067406 (+579286us) service_pool.cc:224] Handling call
0709 13:08:04.067850 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:04.163065 (+ 95215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:04.163104 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:04.164852 (+  1748us) log.cc:844] Serialized 26 byte log entry
0709 13:08:04.300576 (+135724us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:04.397668 (+ 97092us) log.cc:844] Serialized 14 byte log entry
0709 13:08:04.445315 (+ 47647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:04.445392 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:04.898309 (+452917us) raft_consensus.cc:1685] finished
0709 13:08:04.898336 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:04.898503 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51454,"prepare.queue_time_us":98600,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":304584,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":70209,"thread_start_us":172577,"threads_started":2,"wal-append.queue_time_us":265132}
I20260709 13:08:04.869341 26017 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:04.870218 26097 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:04.857124 25833 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:04.771327 26072 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.908035 26097 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:04.909751 26097 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:04.857484 25853 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:04.856427 25789 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:04.856701 25878 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:04.914042 25853 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:04.919651 26070 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:04.922189 26096 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:04.927767 26081 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.451s	user 0.004s	sys 0.000s
I20260709 13:08:04.935514 26061 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:04.936118 26061 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:04.936535 26061 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:04.940995 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:04.944089 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.949888 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:04.958770 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:04.959467 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:04.960402 21066 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:04.962536 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:04.963764 21068 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:08:04.965780 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:04.966981 21064 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:04.969683 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:04.970767 21072 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:04.986451 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:04.987522 21070 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:08:04.990597 21072 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:08:04.987046 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.015s	sys 0.036s
W20260709 13:08:04.993458 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.016s	sys 0.036s
W20260709 13:08:04.993911 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 605298us
I20260709 13:08:04.989033 25878 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:05.036319 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:05.036981 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:05.041137 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:05.041358 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:05.041819 21064 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
W20260709 13:08:05.042280 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 148971us
W20260709 13:08:05.042788 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:05.035709 26055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.049s	sys 0.076s
I20260709 13:08:05.050549 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:05.056319 21072 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
W20260709 13:08:05.052366 26055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.049s	sys 0.076s
I20260709 13:08:04.989375 26077 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:05.060222 21067 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:05.067433 21798 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:05.070371 26077 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:05.034852 26047 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:05.075654 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:05.079720 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 501072us
I20260709 13:08:05.080504 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:05.107725 21138 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:05.115304 21139 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:08:05.115725 21139 pending_rounds.cc:85] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 4
I20260709 13:08:05.127926 21139 pending_rounds.cc:107] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:08:05.128302 21139 raft_consensus.cc:2889] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:05.102031 25898 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.122012 26072 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:08:05.134357 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 53936.3us
I20260709 13:08:05.135062 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:05.115738 26096 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:05.137580 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:05.149495 21143 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:05.152568 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:05.159775 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:05.160481 21139 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:05.160574 25706 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:05.160830 21138 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:08:05.161252 21138 pending_rounds.cc:85] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 12
I20260709 13:08:05.161158 25706 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:05.161633 21138 pending_rounds.cc:107] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:08:05.161979 21138 raft_consensus.cc:2889] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:05.162386 25706 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:05.163393 26117 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:05.164876 26117 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:05.165066 26086 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:05.165369 26117 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:05.165750 26086 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:05.167397 26070 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:05.167493 26099 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:05.168073 26099 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:05.167938 26070 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:05.169191 26099 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:05.170100 26070 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:05.170784 26099 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:05.172165 26101 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:05.176697 21139 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
W20260709 13:08:05.177597 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 527274us
I20260709 13:08:05.182161 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:05.183660 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:05.184417 21794 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:05.172606 26101 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.133698 25898 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:05.199707 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:05.166517 26117 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:05.202661 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:05.184602 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:05.150779 26072 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:05.207494 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.178783 25826 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.209642 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:05.152205 25817 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:08:05.210327 21139 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:05.177136 26074 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:05.213307 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:05.212193 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:05.211874 21066 pending_rounds.cc:85] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 6
I20260709 13:08:05.224578 21066 pending_rounds.cc:107] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:08:05.224835 21066 raft_consensus.cc:2889] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:05.226639 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:05.227363 21069 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:08:05.228174 21063 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:05.228235 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:05.229173 21069 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:05.229482 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:05.229910 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:05.230449 21063 raft_consensus.cc:2410] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:08:05.230749 21069 raft_consensus.cc:2410] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:08:05.231091 26086 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:05.231448 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:05.209849 26096 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:05.203311 25706 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:05.235589 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:05.236019 26047 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:05.236275 21138 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:08:05.210047 26072 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:05.237223 26120 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:08:05.237447 21146 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:05.235049 26096 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:05.215921 21187 leader_election.cc:400] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14.
I20260709 13:08:05.246100 21187 leader_election.cc:403] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:05.219763 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:05.195948 26101 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:05.245692 21140 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:05.213973 25853 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:08:05.250627 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:05.251250 21142 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:05.251374 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66864.8us
I20260709 13:08:05.252107 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:05.252269 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:05.257761 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:05.260005 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65140us
I20260709 13:08:05.262074 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:05.263012 21138 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:08:05.264532 21139 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:05.269649 21063 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:05.270506 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:05.274421 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.269701 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:05.279681 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:05.280547 21139 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:05.275607 26106 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14.
I20260709 13:08:05.283902 26106 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:05.284384 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:05.276289 26059 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:05.288411 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:05.289302 21071 raft_consensus.cc:2368] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16.
I20260709 13:08:05.291455 26083 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:05.284471 26106 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:05.248315 26105 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:05.295238 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:05.296384 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 48709.5us
I20260709 13:08:05.297323 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:05.303726 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:05.300067 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:05.315091 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 121597us
I20260709 13:08:05.316640 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:05.323463 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:05.223764 25826 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:08:05.367444 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50921.7us
I20260709 13:08:05.293208 26047 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.292088 26083 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:05.302156 26055 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.278554 25762 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:08:05.404683 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:05.404968 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:05.303143 26105 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.405552 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:05.369422 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46033.5us
W20260709 13:08:05.410617 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 147345us
I20260709 13:08:05.257277 26097 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:08:05.409569 21067 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:05.248378 26074 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:05.414592 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 110946us
I20260709 13:08:05.405547 26075 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:05.408733 26077 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:05.425153 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:05.426493 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:05.424669 21139 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:05.406330 26060 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.446655 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:05.429311 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:05.410745 25495 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:05.417908 26086 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:05.448940 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:05.440927 26130 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:08:05.450732 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:05.427388 26105 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:05.436411 26057 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:05.481304 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:05.459796 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:05.463799 26086 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.451936 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:05.437996 26028 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:05.460139 25878 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:05.471179 26111 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:05.448580 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:05.440640 26106 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:05.572342 26111 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:05.572749 26111 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.573081 26111 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:05.454712 25789 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:05.575052 25789 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.419576 26055 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:05.475046 21800 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:05.449992 25495 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.576270 26020 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.576464 25495 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:05.476655 21798 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:08:05.443982 26059 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:08:05.477684 26070 raft_consensus.cc:2764] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:08:05.446488 26075 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:05.479462 26080 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:05.414394 26047 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:05.480026 25826 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:05.483647 21799 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:05.485484 21793 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:05.512573 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:05.549803 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.559417 26086 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:05.568038 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 116185us
I20260709 13:08:05.571326 25878 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:05.575476 25789 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:05.473325 21792 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:05.575906 25889 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:05.477262 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:05.581642 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 92466.2us
W20260709 13:08:05.597748 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 114538us
I20260709 13:08:05.455679 26112 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:05.583069 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:05.446254 26026 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:08:05.450706 26032 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:08:05.413729 26061 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:05.436988 26096 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:05.601192 21142 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:05.454813 26060 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:05.455010 25817 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.141s
I20260709 13:08:05.576893 26020 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:05.624257 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 195044us
W20260709 13:08:05.651458 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52891.4us
I20260709 13:08:05.587795 25930 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:05.597980 26075 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:05.588121 26059 consensus_queue.cc:1243] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:08:05.653484 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:05.605507 25878 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:05.669716 21063 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:05.607373 26112 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.609634 25935 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:05.600013 25919 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
W20260709 13:08:05.691143 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.628476 25826 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:08:05.699750 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:05.700416 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 376007us
I20260709 13:08:05.630338 26057 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
W20260709 13:08:05.701193 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.703871 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:05.678637 25935 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:05.678876 26112 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:05.745827 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.738727 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:05.747519 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:05.683543 25817 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:05.780841 21216 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
W20260709 13:08:05.783207 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 202548us
I20260709 13:08:05.751987 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:05.753530 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:05.698544 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 244924us
I20260709 13:08:05.747502 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:05.665802 26080 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:08:05.696216 26077 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:05.793411 21217 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:08:05.673452 26109 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:08:05.750748 25935 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:05.802977 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:05.707497 26094 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:05.688829 26145 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:05.801662 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:05.829331 21213 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:05.810022 21217 pending_rounds.cc:85] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 7
I20260709 13:08:05.824120 21798 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:05.819461 21219 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 9.
I20260709 13:08:05.830181 21217 pending_rounds.cc:107] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:08:05.830560 21217 raft_consensus.cc:2889] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:05.788931 25817 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:05.796443 26017 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:05.835420 21220 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:05.820820 25833 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:08:05.854462 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 205567us
I20260709 13:08:05.876982 26047 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:05.870044 26099 consensus_meta.cc:287] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.694s	user 0.013s	sys 0.000s
W20260709 13:08:05.870054 26072 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.001s
W20260709 13:08:05.883858 21036 connection.cc:256] RPC call timeout handler was delayed by -2.5084242409999433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
I20260709 13:08:05.853427 26083 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:05.862383 26143 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:05.883204 25833 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:05.860529 25759 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:08:05.861270 26070 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:08:05.880036 26074 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:05.847980 26094 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:05.861099 26017 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:05.949483 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1848471409999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1902s for a response
I20260709 13:08:05.889144 26055 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:05.836719 26145 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:05.955188 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 207756us
W20260709 13:08:05.943169 26136 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.422s	user 0.001s	sys 0.000s
W20260709 13:08:05.956537 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 162120us
I20260709 13:08:05.962383 21218 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:05.967582 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:05.941145 26144 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.952945 26061 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:05.966137 26137 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:08:05.900616 26072 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:05.986392 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.902s (SENT) [suppressed 25 similar messages]
I20260709 13:08:05.901196 26077 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:06.027699 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:06.032876 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:06.033767 26083 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:06.034302 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:05.974087 26060 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:06.037660 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 203499us
I20260709 13:08:06.040313 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:05.975678 26110 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:08:06.042796 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 254009us
I20260709 13:08:05.983953 26020 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:06.051534 26167 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:06.009186 26108 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.949800 25817 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:05.972213 26143 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:08:06.048643 26070 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:06.050073 26061 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:06.073263 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:06.077664 21799 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:06.073104 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:06.055727 26155 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:05.979208 26144 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:06.091416 26134 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
W20260709 13:08:06.091698 26149 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:08:06.091784 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:06.093204 26141 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:08:06.093865 26152 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:08:06.094358 26162 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:06.095021 26162 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:06.095847 26148 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 13:08:06.097429 26140 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:08:06.097589 26162 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:06.100062 26094 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:06.091452 26135 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:08:06.106921 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:06.099125 26111 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.002s
I20260709 13:08:06.113792 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:06.127358 21798 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:06.128756 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.055856 25833 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:06.080183 26083 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:08:06.140021 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.067612 26108 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:06.143932 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:06.145936 26108 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:06.129135 25706 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.893s	user 0.013s	sys 0.000s
I20260709 13:08:06.133438 25853 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:06.120164 26155 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:06.127502 26070 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:06.135109 26162 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:06.150895 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:06.108996 26111 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:06.153432 25495 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.002s
I20260709 13:08:06.162019 21798 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:06.155926 25853 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:06.192767 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.193377 26144 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:06.192910 26070 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:06.197461 26028 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.000s
I20260709 13:08:06.197983 26028 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:08:06.203655 26105 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.644s	user 0.002s	sys 0.010s
I20260709 13:08:06.204239 26105 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:08:06.205750 26151 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.356s	user 0.001s	sys 0.001s
I20260709 13:08:06.208578 26074 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:06.209233 26074 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:06.210531 26101 consensus_meta.cc:287] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.932s	user 0.010s	sys 0.000s
I20260709 13:08:06.212256 26155 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:06.216992 25898 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:06.160287 26017 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:06.178786 25706 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:06.178960 26145 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:06.190945 26162 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:06.219893 25853 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:06.224478 26134 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:06.240233 26047 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:06.236622 26162 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:06.249017 26117 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:06.228307 26072 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:06.182415 25495 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:06.266566 26032 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:06.267168 26032 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:06.268342 26032 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:06.269564 26032 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:06.210589 26074 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:06.264315 25789 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.639s	user 0.011s	sys 0.002s
I20260709 13:08:06.211138 26101 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:06.228693 26111 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:06.260401 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1518) took 1151 ms. Trace:
I20260709 13:08:06.255468 21218 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 8.
I20260709 13:08:06.313613 21070 rpcz_store.cc:276] 0709 13:08:05.109068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.109212 (+   144us) service_pool.cc:224] Handling call
0709 13:08:05.122885 (+ 13673us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:05.173016 (+ 50131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:05.173060 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:05.197939 (+ 24879us) log.cc:844] Serialized 52 byte log entry
0709 13:08:05.414974 (+217035us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:05.454110 (+ 39136us) log.cc:844] Serialized 14 byte log entry
0709 13:08:05.789313 (+335203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.789415 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:06.260089 (+470674us) raft_consensus.cc:1685] finished
0709 13:08:06.260120 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.260317 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41033,"prepare.queue_time_us":25997,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":8517,"raft.queue_time_us":56087,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":2114,"thread_start_us":513536,"threads_started":2,"wal-append.queue_time_us":414017}
I20260709 13:08:06.245025 26047 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:06.338619 26105 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:06.339761 26074 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:06.281363 25789 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:06.351506 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 365111us
W20260709 13:08:06.434957 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 467478us
I20260709 13:08:06.450187 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 640) took 1225 ms. Trace:
I20260709 13:08:06.450712 21147 rpcz_store.cc:276] 0709 13:08:05.224690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.224825 (+   135us) service_pool.cc:224] Handling call
0709 13:08:05.250524 (+ 25699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:05.250732 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:05.317377 (+ 66645us) log.cc:844] Serialized 14 byte log entry
0709 13:08:05.653578 (+336201us) log.cc:844] Serialized 14 byte log entry
0709 13:08:06.202031 (+548453us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b18001c98a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:06.449783 (+247752us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:06.449850 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:06.449906 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.449987 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.450122 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":808640,"prepare.queue_time_us":4296,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":2182,"raft.queue_time_us":25149,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7702,"spinlock_wait_cycles":14997760,"thread_start_us":388895,"threads_started":2,"wal-append.queue_time_us":489976}
W20260709 13:08:06.453554 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 800071us
I20260709 13:08:06.453975 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:06.454960 21071 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:06.450263 26125 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.223s	user 0.000s	sys 0.009s
I20260709 13:08:06.515101 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 933) took 1568 ms. Trace:
I20260709 13:08:06.515797 25889 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:06.516322 25889 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.516688 25889 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:06.520808 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:06.528236 21071 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:08:06.515554 21801 rpcz_store.cc:276] 0709 13:08:04.947013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.947173 (+   160us) service_pool.cc:224] Handling call
0709 13:08:04.947609 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:04.948081 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:04.948121 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:04.971869 (+ 23748us) log.cc:844] Serialized 26 byte log entry
0709 13:08:05.138654 (+166785us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:05.138723 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.138818 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:06.449582 (+1310764us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:06.514784 (+ 65202us) raft_consensus.cc:1685] finished
0709 13:08:06.514808 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.515035 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9880,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6000,"spinlock_wait_cycles":33054080,"thread_start_us":81469,"threads_started":1,"wal-append.queue_time_us":191003}
I20260709 13:08:06.380415 26047 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:06.551054 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:06.541159 26112 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.001s
I20260709 13:08:06.557319 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:06.552676 26073 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:06.552994 25495 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:06.439244 25853 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:06.561996 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 448281us
I20260709 13:08:06.570509 21147 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:06.548733 26101 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:06.579035 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 124508us
I20260709 13:08:06.561116 25789 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:06.579912 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:06.585268 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:06.596375 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:06.561846 26112 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:06.599924 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 447940us
I20260709 13:08:06.602561 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:06.603012 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:06.603750 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:06.606002 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:06.579202 26060 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:08:06.614730 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:06.627701 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.633867 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:06.635419 26157 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:08:06.635845 26183 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:08:06.636431 26157 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:06.636847 26157 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:06.637025 21069 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:08:06.563932 26073 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:06.636243 26059 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:06.604548 26101 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:06.647711 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.665746 21221 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
W20260709 13:08:06.612030 26116 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.409s	user 0.000s	sys 0.023s
I20260709 13:08:06.636893 26128 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:06.642493 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:06.643065 26075 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.967s	user 0.014s	sys 0.000s
I20260709 13:08:06.621979 25395 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:06.643993 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.644658 21795 raft_consensus.cc:2368] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:08:06.644948 21066 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:08:06.681607 21066 pending_rounds.cc:85] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 7
I20260709 13:08:06.681893 21066 pending_rounds.cc:107] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:08:06.682137 21066 raft_consensus.cc:2889] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:06.645462 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:06.645357 26157 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:06.646822 26105 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.648773 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:06.640498 26104 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:06.604986 26026 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:06.613838 26054 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:08:06.625512 25898 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:08:06.606920 26080 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:06.652410 26167 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.653491 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.655714 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.656117 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.656653 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.663688 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:06.644165 21064 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:06.704141 21220 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:08:06.667830 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:06.705281 21069 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:06.668823 21070 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:06.611958 25934 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:06.693689 25935 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.861s	user 0.013s	sys 0.001s
I20260709 13:08:06.679698 21143 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:08:06.682842 21063 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:06.684899 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.649552 26020 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:06.690167 21065 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:06.625671 25441 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:08:06.693053 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:06.715901 21215 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.006s	sys 0.009s
I20260709 13:08:06.664814 26101 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:06.680521 25395 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:06.674820 21212 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:06.709177 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.703262 21219 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:06.689245 26162 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.714039 21221 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:08:06.705875 26077 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:06.716286 21215 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.006s	sys 0.010s
I20260709 13:08:06.719462 21139 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 8: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:06.691244 26104 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:06.719259 25441 consensus_queue.cc:1243] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:08:06.719527 26185 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:06.735531 26109 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.710847 25853 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:06.736011 26185 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:06.708491 26157 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:06.711416 26167 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:06.682276 26075 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:06.738467 26185 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:06.738876 26167 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:06.736148 26109 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:06.724351 25878 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.038s	user 0.012s	sys 0.001s
I20260709 13:08:06.740936 25878 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:06.707623 25934 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:06.742619 25934 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:06.743476 21072 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:06.739507 26099 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:06.744015 26099 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:06.744244 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:06.744976 26099 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:06.745431 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:06.746758 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:06.746465 26099 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:06.722528 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:06.727660 25395 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:06.752655 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:06.753430 21067 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:06.754453 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:06.756677 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:06.756102 26109 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:06.758306 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:06.759205 21063 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:06.760936 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1580) took 1067 ms. Trace:
I20260709 13:08:06.761317 21218 rpcz_store.cc:276] 0709 13:08:05.693359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.698892 (+  5533us) service_pool.cc:224] Handling call
0709 13:08:06.760735 (+1061843us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b540015c554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:06.760869 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409216256}
I20260709 13:08:06.762487 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.723409 26101 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:06.726204 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.722332 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:06.721484 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.733244 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:06.736328 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:06.790251 21221 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:08:06.763187 21218 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:08:06.731727 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.792196 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:06.729949 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:06.733028 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.795121 21212 raft_consensus.cc:2376] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14. Re-sending same reply.
I20260709 13:08:06.795656 21035 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:06.763099 21066 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:06.722836 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:06.797796 21216 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 7.
I20260709 13:08:06.798182 21145 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:06.755064 21067 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:06.798828 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.805263 21216 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
W20260709 13:08:06.812003 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.812250 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.799722 21072 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:06.727677 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:06.701398 26128 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:06.801081 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 362019us
I20260709 13:08:06.723039 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:06.733870 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:06.819746 21765 leader_election.cc:336] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.078s (SENT)
I20260709 13:08:06.814174 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1591) took 1025 ms. Trace:
I20260709 13:08:06.730737 21214 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:06.694207 26105 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:08:06.917302 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.840215 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:06.930538 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:06.914753 26096 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:06.913615 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:06.917510 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:07.246966 26060 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.013s	sys 0.024s
I20260709 13:08:06.837991 26111 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:08:07.628860 26060 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.019s	sys 0.035s
W20260709 13:08:07.629281 21142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.902s	user 0.013s	sys 0.021s
I20260709 13:08:07.022189 21067 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:06.930344 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:06.735394 25495 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:06.732887 26077 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:07.630970 25495 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:07.631891 26105 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:07.349408 25863 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.008s	sys 0.033s
I20260709 13:08:06.838497 21215 rpcz_store.cc:276] 0709 13:08:05.788744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.818004 (+ 29260us) service_pool.cc:224] Handling call
0709 13:08:05.832934 (+ 14930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:05.833195 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:05.973942 (+140747us) log.cc:844] Serialized 14 byte log entry
0709 13:08:06.734107 (+760165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:06.734204 (+    97us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:06.734288 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.734469 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.814115 (+ 79646us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95574,"prepare.queue_time_us":231190,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":91394,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":1416,"thread_start_us":776965,"threads_started":3,"wal-append.queue_time_us":216819}
I20260709 13:08:07.632666 21064 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:07.632755 25863 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.012s	sys 0.042s
W20260709 13:08:06.930800 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 111084us
I20260709 13:08:07.633600 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:07.633945 21765 leader_election.cc:336] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.098s (SENT)
I20260709 13:08:07.634289 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:07.634348 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:07.435214 25930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.000s	sys 0.027s
W20260709 13:08:07.390354 26032 consensus_meta.cc:287] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.120s	user 0.009s	sys 0.006s
I20260709 13:08:06.893990 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:06.918977 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:07.636474 21218 raft_consensus.cc:2376] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15. Re-sending same reply.
I20260709 13:08:06.812125 25878 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:06.840798 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 106995us
I20260709 13:08:07.637516 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:06.734994 26179 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:08:07.638422 21218 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:08:07.264567 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.003s	sys 0.018s
I20260709 13:08:06.936232 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 929) took 2126 ms. Trace:
W20260709 13:08:07.284422 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.007s	sys 0.015s
W20260709 13:08:07.639369 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.999s	user 0.004s	sys 0.028s
I20260709 13:08:06.917186 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:06.845330 26109 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.795s
I20260709 13:08:07.640334 21138 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:07.243927 26089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.003s	sys 0.018s
I20260709 13:08:07.641594 21064 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:06.738914 26075 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:07.643216 26075 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:07.625283 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:07.568425 25395 consensus_meta.cc:287] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.820s	user 0.012s	sys 0.000s
I20260709 13:08:07.644560 21219 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:06.893143 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:07.645278 25395 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:07.072806 26026 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:08:07.643414 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.013s	sys 0.024s
I20260709 13:08:07.646798 26026 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:07.647324 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 1.02099e+06us
I20260709 13:08:06.732087 26162 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:06.716456 25935 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:07.629633 21142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.013s	sys 0.021s
I20260709 13:08:07.646626 21216 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:08:07.654546 26077 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:07.031332 26125 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:08:07.639526 21797 rpcz_store.cc:276] 0709 13:08:04.787040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.787607 (+   567us) service_pool.cc:224] Handling call
0709 13:08:04.787963 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:04.788342 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:04.788378 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:04.810322 (+ 21944us) log.cc:844] Serialized 26 byte log entry
0709 13:08:05.077765 (+267443us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:05.077828 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.077926 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:06.548855 (+1470929us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:06.768529 (+219674us) raft_consensus.cc:1685] finished
0709 13:08:06.768561 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.913163 (+144602us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":15350,"spinlock_wait_cycles":5734784,"thread_start_us":236055,"threads_started":1,"wal-append.queue_time_us":297753}
I20260709 13:08:07.635063 21221 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:07.628746 21213 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:07.658334 26110 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:07.636822 21217 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:07.659060 26110 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:07.659706 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:07.641060 26109 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:07.660784 26109 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:07.661494 21063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.014s	sys 0.012s
W20260709 13:08:07.661935 21063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.014s	sys 0.012s
I20260709 13:08:07.661692 26109 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:07.435237 26192 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.440s	user 0.001s	sys 0.001s
I20260709 13:08:07.663008 26109 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:07.285776 26142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.001s	sys 0.036s
W20260709 13:08:07.664178 26142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.001s	sys 0.037s
W20260709 13:08:07.414956 25889 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.898s	user 0.013s	sys 0.002s
I20260709 13:08:07.664942 25889 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:07.635172 25930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.000s	sys 0.033s
I20260709 13:08:07.666129 21140 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:08:07.645946 21212 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:07.667483 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:07.689980 21220 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.873s	user 0.008s	sys 0.003s
I20260709 13:08:07.530485 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:07.074330 26112 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:07.668790 26162 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:07.669658 21035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.001s	sys 0.009s
W20260709 13:08:07.724649 21035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.001s	sys 0.009s
W20260709 13:08:07.725049 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 911600us
I20260709 13:08:07.646318 26032 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:07.670579 21214 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:07.645849 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:07.673491 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:07.640933 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 1.0262e+06us
W20260709 13:08:07.732924 21147 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.998s	user 0.015s	sys 0.009s
I20260709 13:08:07.001897 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1029) took 1562 ms. Trace:
W20260709 13:08:07.641819 26089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.011s	sys 0.023s
I20260709 13:08:07.634759 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:07.681881 26074 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.341s	user 0.010s	sys 0.002s
I20260709 13:08:07.682876 25441 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.746260 21219 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
W20260709 13:08:07.501461 26086 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.875s	user 0.016s	sys 0.001s
I20260709 13:08:06.731638 26061 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:06.764457 25963 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:07.692160 21141 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:08:07.651463 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.658433 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:06.930959 26128 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:07.678419 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:07.549266 26144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.023s	sys 0.029s
I20260709 13:08:07.653673 25817 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:07.650014 25833 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:07.707598 21220 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:07.714062 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:07.771039 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2878) took 1108 ms. Trace:
I20260709 13:08:07.772390 21220 rpcz_store.cc:276] 0709 13:08:06.662603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.712903 (+ 50300us) service_pool.cc:224] Handling call
0709 13:08:07.770959 (+1058056us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.714466 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2603) took 1143 ms. Trace:
W20260709 13:08:07.715346 21764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.030s	sys 0.025s
I20260709 13:08:07.643996 26075 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:07.722882 21138 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
W20260709 13:08:07.773995 21764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.030s	sys 0.025s
W20260709 13:08:07.726888 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 777042us
W20260709 13:08:07.733373 21147 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.015s	sys 0.009s
I20260709 13:08:07.734664 21213 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:07.776635 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2876) took 1114 ms. Trace:
I20260709 13:08:07.777057 21213 rpcz_store.cc:276] 0709 13:08:06.662096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.802971 (+140875us) service_pool.cc:224] Handling call
0709 13:08:07.776570 (+973599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.674651 26073 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:07.733630 21795 rpcz_store.cc:276] 0709 13:08:05.439787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.476995 (+ 37208us) service_pool.cc:224] Handling call
0709 13:08:06.997347 (+1520352us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540040a1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:07.001793 (+  4446us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2380220672}
I20260709 13:08:07.740433 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2868) took 1080 ms. Trace:
I20260709 13:08:07.778543 21214 rpcz_store.cc:276] 0709 13:08:06.660057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.839893 (+179836us) service_pool.cc:224] Handling call
0709 13:08:07.740354 (+900461us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:07.741276 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 78996.6us
W20260709 13:08:07.744793 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 71387us
I20260709 13:08:07.745734 21113 leader_election.cc:400] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16.
I20260709 13:08:07.751669 21221 raft_consensus.cc:2376] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14. Re-sending same reply.
I20260709 13:08:07.667997 25889 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:07.703414 25495 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:07.766659 26144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.026s	sys 0.029s
I20260709 13:08:07.716028 26111 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:07.786751 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:07.787490 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:07.787794 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:07.788182 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:07.803906 21221 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:07.782133 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:07.779196 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:07.847949 21215 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:07.780017 21113 leader_election.cc:403] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:08:07.793322 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:07.793589 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:06.735131 26104 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:07.755597 25441 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:07.757290 26061 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:07.843930 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:07.844261 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 170386us
I20260709 13:08:07.795101 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:07.845531 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:07.861718 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:07.864951 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 69926.1us
I20260709 13:08:07.766297 26199 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:08:07.848006 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65995.4us
I20260709 13:08:07.773904 21144 rpcz_store.cc:276] 0709 13:08:06.571419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.571637 (+   218us) service_pool.cc:224] Handling call
0709 13:08:06.579883 (+  8246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:06.580094 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:06.580334 (+   240us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.540886 (+960552us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b1800033400. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:08:07.714093 (+173207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:07.714153 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:07.714206 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.714270 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:07.714408 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":114,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":2476,"raft.queue_time_us":156,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":83,"spinlock_wait_cycles":3447552,"thread_start_us":172587,"threads_started":1,"wal-append.queue_time_us":1180464}
I20260709 13:08:07.766016 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:07.696062 26101 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.851s	user 0.010s	sys 0.002s
I20260709 13:08:07.648742 26026 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:07.757028 25963 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:07.856213 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 110630us
I20260709 13:08:07.740221 26032 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:07.661470 26110 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:07.861720 21801 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:07.763340 26086 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:07.865971 21220 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:07.870019 21214 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:07.925974 21221 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:08:07.965514 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:07.972747 21213 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:07.861392 26089 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16.
I20260709 13:08:07.936338 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:07.982081 21138 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:07.971164 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.985581 21218 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
W20260709 13:08:07.976858 21143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.037s	sys 0.050s
W20260709 13:08:07.986631 21143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.037s	sys 0.050s
I20260709 13:08:07.949606 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2884) took 1104 ms. Trace:
I20260709 13:08:07.987322 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:07.890657 21212 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:07.976799 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2879) took 1313 ms. Trace:
W20260709 13:08:07.971698 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 206936us
I20260709 13:08:07.988158 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2872) took 1327 ms. Trace:
I20260709 13:08:07.988231 21214 rpcz_store.cc:276] 0709 13:08:06.662880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.861422 (+1198542us) service_pool.cc:224] Handling call
0709 13:08:07.976721 (+115299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.988729 21221 rpcz_store.cc:276] 0709 13:08:06.661075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.986154 (+1325079us) service_pool.cc:224] Handling call
0709 13:08:07.988099 (+  1945us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.989840 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2873) took 1328 ms. Trace:
I20260709 13:08:07.853753 26096 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:07.987669 21219 rpcz_store.cc:276] 0709 13:08:06.845711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.755815 (+910104us) service_pool.cc:224] Handling call
0709 13:08:07.949522 (+193707us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.991421 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:07.989984 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:07.990912 21212 rpcz_store.cc:276] 0709 13:08:06.661356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.795974 (+134618us) service_pool.cc:224] Handling call
0709 13:08:07.989752 (+1193778us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:07.843624 25495 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:08.022176 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:07.936960 25889 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:08.021418 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:07.934834 25395 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:07.999533 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:08.014175 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:08.033610 21219 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 15 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:08:08.014448 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:07.950802 26199 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.866434 26162 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.130s
W20260709 13:08:08.040721 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 178675us
I20260709 13:08:08.070583 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:07.866724 26191 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.104549 21217 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:08:07.944983 26101 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:07.980481 26110 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.064581 21213 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 10.
I20260709 13:08:08.127710 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2877) took 1465 ms. Trace:
I20260709 13:08:08.128266 21213 rpcz_store.cc:276] 0709 13:08:06.662349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.863077 (+1200728us) service_pool.cc:224] Handling call
0709 13:08:08.127628 (+264551us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:08.063072 26199 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:08.032882 26097 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:08.073344 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:08.073508 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:08.073768 26157 consensus_meta.cc:287] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.337s	user 0.013s	sys 0.001s
I20260709 13:08:08.065714 25759 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
W20260709 13:08:08.123185 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 190720us
I20260709 13:08:08.158913 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 637) took 1040 ms. Trace:
I20260709 13:08:08.074246 26032 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:08.074513 26178 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:08.075479 26075 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:08.077347 21214 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:08:08.079824 26200 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.081403 26053 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:08.083410 26089 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:08.089988 26221 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
I20260709 13:08:07.844682 26099 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.196s
I20260709 13:08:08.093494 26086 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:08.093969 26112 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:08:08.094568 26212 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:08:08.095324 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 80921.9us
I20260709 13:08:08.207592 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:08.209195 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:08.210860 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:08.067513 25495 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.096441 25935 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:08.096890 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 98083.6us
I20260709 13:08:08.156894 26215 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:08.099552 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2880) took 1436 ms. Trace:
I20260709 13:08:08.221570 21219 rpcz_store.cc:276] 0709 13:08:06.663151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.013891 (+1350740us) service_pool.cc:224] Handling call
0709 13:08:08.099482 (+ 85591us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:08.108299 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:08.063444 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 127853us
I20260709 13:08:08.106797 21138 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
W20260709 13:08:08.223064 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 114886us
I20260709 13:08:08.159729 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:08.159454 21213 rpcz_store.cc:276] 0709 13:08:07.118918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.157974 (+1039056us) service_pool.cc:224] Handling call
0709 13:08:08.158373 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.158588 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:08.158611 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:08.158654 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:08.158694 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.158745 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.158857 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:08.096447 26047 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:08.116888 26026 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:08.065773 25889 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:08.227413 26228 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.227972 26228 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.228118 25889 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:08.230237 26228 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:08.231029 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 71388us
W20260709 13:08:08.231402 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:08.224427 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 202290us
W20260709 13:08:08.231998 21765 connection.cc:256] RPC call timeout handler was delayed by -1.3391320069999892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22450000000000003s for a response
I20260709 13:08:08.168144 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2886) took 1173 ms. Trace:
I20260709 13:08:08.234838 21214 rpcz_store.cc:276] 0709 13:08:06.994997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.076523 (+1081526us) service_pool.cc:224] Handling call
0709 13:08:08.076928 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.164834 (+ 87906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.168065 (+  3231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:08.170042 21139 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:08.169615 21065 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:08:08.120788 26070 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:08.192185 26200 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:08.195604 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:08.196200 21220 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:08.195607 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:08.200178 25441 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:08.164202 26178 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.168965 26032 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.189332 26157 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:08.200654 26089 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:08.204795 25934 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.462s	user 0.012s	sys 0.000s
W20260709 13:08:08.166791 26195 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.002s	sys 0.010s
I20260709 13:08:08.220139 21185 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:08.097631 26105 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:08.226485 25759 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:08.236370 21065 pending_rounds.cc:85] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 8
I20260709 13:08:08.300221 21065 pending_rounds.cc:107] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 12.9
W20260709 13:08:08.298870 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78816.7us
I20260709 13:08:08.300506 21065 raft_consensus.cc:2889] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:08.231878 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2614) took 1511 ms. Trace:
I20260709 13:08:08.258425 21068 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:08.259406 21797 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:08.262439 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:08.116474 26191 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.270768 21219 pending_rounds.cc:85] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 6
I20260709 13:08:08.311131 21219 pending_rounds.cc:107] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:08:08.311448 21219 raft_consensus.cc:2889] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:08.263835 21765 leader_election.cc:336] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.159s (SENT)
I20260709 13:08:08.312139 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:08.163607 26097 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.227991 26094 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:08.158483 26110 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.114292 26111 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:08.123333 25963 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:08.219601 26053 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:08.295714 21144 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:08.303059 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:08.341745 21068 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:08:08.361066 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58108us
I20260709 13:08:08.272897 25930 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:08.275000 26178 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:08.240056 26215 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:08.299718 26195 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.007s	sys 0.016s
I20260709 13:08:08.301482 21143 rpcz_store.cc:276] 0709 13:08:06.721005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.721241 (+   236us) service_pool.cc:224] Handling call
0709 13:08:06.721666 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:06.841118 (+119452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:06.841150 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:06.841615 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:08:07.987217 (+1145602us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:07.987270 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.987379 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:08.231640 (+244261us) raft_consensus.cc:1685] finished
0709 13:08:08.231658 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.231809 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38541,"raft.queue_time_us":42549,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":54620,"thread_start_us":1007076,"threads_started":1,"wal-append.queue_time_us":1252485}
I20260709 13:08:08.438362 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:08.439044 21143 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:08.309684 25863 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:08:08.334532 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 72179us
I20260709 13:08:08.445664 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:08.257606 26144 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:08:08.274941 26074 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:08:08.309732 26232 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:08.338795 26128 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:08.446959 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2871) took 1786 ms. Trace:
I20260709 13:08:08.326427 26075 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:08.292260 26157 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.290526 26032 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:08.338917 21218 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:08.359191 25878 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:08.471735 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 207910us
I20260709 13:08:08.473628 21221 rpcz_store.cc:276] 0709 13:08:06.660815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.014830 (+1354015us) service_pool.cc:224] Handling call
0709 13:08:08.015168 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:08.075155 (+ 59987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:08.075201 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:08.075872 (+   671us) log.cc:844] Serialized 26 byte log entry
0709 13:08:08.168248 (+ 92376us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:08.435828 (+267580us) log.cc:844] Serialized 14 byte log entry
0709 13:08:08.446606 (+ 10778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.446685 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:08.446743 (+    58us) raft_consensus.cc:1685] finished
0709 13:08:08.446756 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.446897 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56495,"raft.queue_time_us":59645,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":48,"spinlock_wait_cycles":17280,"thread_start_us":310845,"threads_started":2,"wal-append.queue_time_us":124233}
I20260709 13:08:08.359943 26167 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:08:08.454957 26105 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:08.275324 26096 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:08.448117 26094 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:08.598529 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2611) took 1892 ms. Trace:
I20260709 13:08:08.299532 25934 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:08.465449 26162 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:08.467171 25963 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:08.626657 21215 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.778s	user 0.008s	sys 0.004s
I20260709 13:08:08.599092 21142 rpcz_store.cc:276] 0709 13:08:06.709289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.709504 (+   215us) service_pool.cc:224] Handling call
0709 13:08:06.709800 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:06.726464 (+ 16664us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:06.726821 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.675859 (+949038us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:07.675937 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:07.676018 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.598165 (+922147us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b54002b52d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:08.598232 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.598446 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8643,"prepare.queue_time_us":238,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":16368,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":7620,"spinlock_wait_cycles":48501632,"thread_start_us":939869,"threads_started":1,"wal-append.queue_time_us":972933}
I20260709 13:08:08.639202 21215 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:08.640055 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 635) took 1521 ms. Trace:
I20260709 13:08:08.640276 21212 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:08.640506 21215 rpcz_store.cc:276] 0709 13:08:07.118377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.757824 (+639447us) service_pool.cc:224] Handling call
0709 13:08:08.639975 (+882151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:08.633296 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.496277 25863 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:08.641750 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 280052us
W20260709 13:08:08.593344 26235 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:08:08.646847 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 436010us
W20260709 13:08:08.650815 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.466804 26227 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.183s
I20260709 13:08:08.656277 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2616) took 1812 ms. Trace:
W20260709 13:08:08.642457 26206 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.427s	user 0.000s	sys 0.001s
I20260709 13:08:08.603602 26094 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:08.657352 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2342) took 2013 ms. Trace:
I20260709 13:08:08.647899 26128 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.681730 21065 rpcz_store.cc:276] 0709 13:08:06.644034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.843315 (+199281us) service_pool.cc:224] Handling call
0709 13:08:06.917253 (+ 73938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.301059 (+1383806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.657108 (+356049us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400214b54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:08.657292 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2872490240}
I20260709 13:08:08.659879 21145 rpcz_store.cc:276] 0709 13:08:06.843801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.843939 (+   138us) service_pool.cc:224] Handling call
0709 13:08:06.917608 (+ 73669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.794792 (+877184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:07.794835 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:07.794884 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:07.794944 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.656032 (+861088us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54002954d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:08.656082 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.656214 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016746240}
I20260709 13:08:08.476735 26232 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.647123 25863 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:08.690323 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:08.700856 21142 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:08.646392 26226 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:08.650446 26162 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:08.594369 25878 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:08.707078 26089 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:08.709405 26200 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:08:08.709897 26053 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.652810 26224 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:08.737453 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 87148.4us
I20260709 13:08:08.691113 26243 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:08.711860 25863 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:08.745702 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:08.747396 21145 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:08.745227 26162 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:08.717995 25495 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:08.718441 26214 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:08.743563 26191 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:08.794023 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:08.805208 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 66844.2us
W20260709 13:08:08.801687 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 356092us
W20260709 13:08:08.802103 25889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.008s	sys 0.009s
W20260709 13:08:08.802373 21035 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.005s	sys 0.008s
I20260709 13:08:08.699112 26128 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.806699 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:08.808996 21797 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:08.816469 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:08.818120 21035 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.005s	sys 0.009s
I20260709 13:08:08.747275 26053 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:08.760691 26209 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.474s	user 0.001s	sys 0.000s
I20260709 13:08:08.825320 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:08.836967 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.842032 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:08.826135 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 662204us
I20260709 13:08:08.788416 26214 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:08.843993 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:08.863493 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57533.2us
W20260709 13:08:08.818077 25889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.008s	sys 0.009s
I20260709 13:08:08.872325 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:08.788314 26246 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:08.731102 26226 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:08.882128 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47407.1us
I20260709 13:08:08.885332 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2874) took 2223 ms. Trace:
I20260709 13:08:08.885865 21218 rpcz_store.cc:276] 0709 13:08:06.661595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.014862 (+1353267us) service_pool.cc:224] Handling call
0709 13:08:08.015761 (+   899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.492709 (+476948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.885047 (+392338us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54004349d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:08.885267 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":744059904}
I20260709 13:08:08.825001 26224 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.897984 21071 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:08:08.825011 26110 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:08.894134 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:08.749087 26243 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:08.843457 26191 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:08.840097 26223 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:08.901013 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:08.907780 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:08.874744 26214 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:08.454813 26229 consensus_queue.cc:1175] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" responder_term: 8 status { last_received { term: 7 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:08.942576 21065 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:08.942224 21068 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:08.943123 21146 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:08:08.941056 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:08.881304 26246 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:08.943964 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 99349.6us
I20260709 13:08:08.803731 26094 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:08.963011 21143 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.001s
W20260709 13:08:08.948906 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.000s	sys 0.016s
I20260709 13:08:08.898569 26157 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:08.900964 26232 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:08.975021 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:08.973165 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2337) took 2334 ms. Trace:
I20260709 13:08:08.976987 21066 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:08.902277 26215 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:08.892802 26167 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:08:08.976631 26182 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:08.938040 26196 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:09.000020 21063 rpcz_store.cc:276] 0709 13:08:06.637976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.759877 (+121901us) service_pool.cc:224] Handling call
0709 13:08:06.760278 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:06.760757 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:06.760796 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:06.761319 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:08:07.662415 (+901096us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:07.662528 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.662600 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:08.949527 (+1286927us) raft_consensus.cc:1685] finished
0709 13:08:08.949557 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.972388 (+ 22831us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":201,"thread_start_us":900633,"threads_started":1,"wal-append.queue_time_us":1403560}
W20260709 13:08:08.972055 25817 consensus_meta.cc:287] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.200s	user 0.013s	sys 0.000s
W20260709 13:08:08.984045 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.007s	sys 0.019s
W20260709 13:08:09.055007 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 754688us
I20260709 13:08:08.983208 21143 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:08.949339 26223 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:08.974471 26252 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:08.985839 26094 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:08.988517 21146 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:08.947546 26246 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:09.010159 21068 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:08.971392 26032 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:08.938534 25934 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:09.042780 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.908535 26224 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:09.048177 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 175927us
W20260709 13:08:09.048647 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 73723.1us
I20260709 13:08:09.058426 21070 raft_consensus.cc:2410] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:08:09.063506 21071 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:09.069978 21065 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:09.086578 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.089715 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.092204 21068 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:09.056545 26105 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:09.093546 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:09.094151 21071 raft_consensus.cc:2368] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10.
I20260709 13:08:09.095702 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2623) took 1318 ms. Trace:
I20260709 13:08:09.098279 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.096539 21145 rpcz_store.cc:276] 0709 13:08:07.777758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.763973 (+986215us) service_pool.cc:224] Handling call
0709 13:08:08.815872 (+ 51899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.816083 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:08.950349 (+134266us) log.cc:844] Serialized 14 byte log entry
0709 13:08:09.095228 (+144879us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:09.095325 (+    97us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:09.095395 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.095488 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.095620 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9882,"prepare.queue_time_us":248936,"thread_start_us":222806,"threads_started":2}
I20260709 13:08:09.101382 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:09.102120 21065 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:09.066512 26252 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:09.098948 21063 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:09.065892 25878 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.109946 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.110821 21068 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:09.083694 25930 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:09.084105 25441 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:09.089690 26222 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.095779 25934 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.121811 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:09.070901 26089 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:08:09.126199 21068 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:09.119011 26099 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:09.127396 26099 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.120112 25863 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.129729 26099 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:09.129979 26250 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:08:09.112358 26241 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.130622 26241 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.132884 26241 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:09.119064 25889 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.134532 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:09.134908 25889 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.135576 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2624) took 1357 ms. Trace:
I20260709 13:08:09.135881 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.136018 21144 rpcz_store.cc:276] 0709 13:08:07.778041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.013894 (+235853us) service_pool.cc:224] Handling call
0709 13:08:09.135406 (+1121512us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540036c754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:09.135516 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320820608}
I20260709 13:08:09.136726 21063 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:09.137091 25889 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:09.137558 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:09.137733 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:09.138370 21144 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:09.142750 26026 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.917s	user 0.008s	sys 0.000s
I20260709 13:08:09.143445 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.144013 26097 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:09.144505 21063 raft_consensus.cc:2410] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:08:09.083015 26215 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:08:09.130254 25863 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:09.087590 26196 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:08:09.153482 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:09.154609 25963 consensus_meta.cc:287] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.506s	user 0.005s	sys 0.004s
I20260709 13:08:09.096146 26167 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:09.113233 26208 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.304s	user 0.001s	sys 0.001s
I20260709 13:08:09.164140 21065 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:09.108443 26253 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:09.166013 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.144654 21141 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:09.165980 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.106371 26226 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.161633 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:09.067121 26259 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:08:09.109227 26105 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:09.168479 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:09.119342 25817 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:09.108886 26252 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:09.185135 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 104827us
I20260709 13:08:09.186796 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.173657 21063 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:09.175710 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:09.177006 21797 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:09.168715 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:09.210906 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.178650 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:09.179934 21064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2336) took 2534 ms (client timeout 1620 ms). Trace:
I20260709 13:08:09.184350 26223 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:09.279872 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 93186.8us
I20260709 13:08:09.280746 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.283811 26223 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:09.284372 21800 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:08:09.184814 21187 connection.cc:256] RPC call timeout handler was delayed by -1.0532663200000343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
I20260709 13:08:09.286015 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.286329 26224 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:09.286686 21219 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:09.287770 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.287842 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:09.288622 21215 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:09.289232 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:09.289768 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.292286 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2605) took 2714 ms. Trace:
I20260709 13:08:09.187978 26245 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.189109 26061 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.190671 26242 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:09.283079 26199 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.116s	user 0.012s	sys 0.000s
I20260709 13:08:09.195827 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:09.197633 25762 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:09.120363 26128 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:09.184345 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:09.264144 26257 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:08:09.172278 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:09.168051 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.225818 25889 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:08:09.180037 25963 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:09.272413 21113 leader_election.cc:400] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10.
W20260709 13:08:09.274874 26047 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.049s	user 0.005s	sys 0.004s
I20260709 13:08:09.275523 26243 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:09.226246 26032 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:09.294986 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:09.295621 21064 rpcz_store.cc:269] 0709 13:08:06.637731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.722012 (+ 84281us) service_pool.cc:224] Handling call
0709 13:08:09.172056 (+2450044us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b54003bf454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:09.172161 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940046208}
I20260709 13:08:09.292771 21147 rpcz_store.cc:276] 0709 13:08:06.582466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.598663 (+ 16197us) service_pool.cc:224] Handling call
0709 13:08:06.607565 (+  8902us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:06.694337 (+ 86772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:06.694411 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:06.702972 (+  8561us) log.cc:844] Serialized 26 byte log entry
0709 13:08:07.776120 (+1073148us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:07.776163 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.088661 (+312498us) spinlock_profiling.cc:243] Waited 83.1 ms on lock 0x7b540036cc54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:08.088729 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.291941 (+1203212us) raft_consensus.cc:1685] finished
0709 13:08:09.291965 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.292206 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3133,"raft.queue_time_us":18195,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3262,"spinlock_wait_cycles":191140224,"thread_start_us":1040835,"threads_started":1,"wal-append.queue_time_us":2005569}
I20260709 13:08:09.282886 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.168440 26191 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.337729 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:09.298890 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:09.297703 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 229978us
I20260709 13:08:09.191147 26196 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.156996 25441 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.308525 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:09.296809 26228 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:09.163954 25934 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.340158 21147 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:09.177999 21144 raft_consensus.cc:2368] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:08:09.309842 21796 raft_consensus.cc:2368] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:08:09.157208 25930 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.172767 26167 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.313298 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:09.311589 25863 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:09.316524 21113 leader_election.cc:403] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:08:09.318642 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 146430us
I20260709 13:08:09.319591 21799 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:09.163517 26222 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.302682 26089 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.336684 21066 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:09.176153 26253 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.339550 21219 raft_consensus.cc:2376] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15. Re-sending same reply.
I20260709 13:08:09.308769 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:09.308787 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 124525us
I20260709 13:08:09.341578 21214 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:08:09.339829 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 161289us
I20260709 13:08:09.345908 21220 raft_consensus.cc:2376] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8. Re-sending same reply.
I20260709 13:08:09.315620 21065 raft_consensus.cc:2410] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:08:09.353334 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.373852 21064 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:09.332981 26096 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:09.349996 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:09.354789 26178 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.893s	user 0.012s	sys 0.000s
W20260709 13:08:09.358551 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 86326.5us
I20260709 13:08:09.372184 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:09.306121 26224 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 13:08:09.356881 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 74097.2us
I20260709 13:08:09.380692 21147 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:09.380844 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:09.381541 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:09.382436 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:09.384032 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:09.378204 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.342931 26223 raft_consensus.cc:2764] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:08:09.347815 25863 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.314966 26199 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.348281 26231 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:09.384316 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:09.402134 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.426944 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:09.355551 25963 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.430936 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:09.309362 26242 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.385788 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.425370 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:09.356542 26109 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.385526 21144 raft_consensus.cc:2410] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:08:09.345703 26191 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.433024 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.369145 25889 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 9. Current term is 10.
I20260709 13:08:09.310374 26245 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.342639 26196 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.379549 26096 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:09.430003 26026 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:08:09.439746 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 137463us
I20260709 13:08:09.447567 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:09.445966 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.396032 26217 raft_consensus.cc:2764] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:08:09.357868 26243 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:08:09.354720 26228 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:09.456374 26264 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:08:09.410332 26251 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:09.369175 26247 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:08:09.472149 26251 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:09.473419 26251 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:09.473614 26086 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:09.449934 26260 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:09.452740 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:09.455814 21067 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:09.479413 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2345) took 1307 ms. Trace:
I20260709 13:08:09.463512 26167 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:09.479884 21067 rpcz_store.cc:276] 0709 13:08:08.172070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.455160 (+1283090us) service_pool.cc:224] Handling call
0709 13:08:09.455497 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.479036 (+ 23539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.479342 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:09.460865 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:09.477669 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 124117us
I20260709 13:08:09.430023 26032 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:09.427986 26111 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.425793 26128 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:09.466150 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:09.517835 26128 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:09.518205 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52106us
I20260709 13:08:09.518577 26128 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:09.519302 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:09.410009 26047 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:09.422740 26265 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:09.468528 21064 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:09.520638 26047 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.520936 26265 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:09.521543 26200 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:09.522076 26265 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.433501 25963 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:09.528903 26260 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:09.529387 26260 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.529737 26260 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:09.433410 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47705.9us
I20260709 13:08:09.446084 26259 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:09.435562 26231 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:09.448115 26261 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:09.425227 25833 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:08:09.464618 26096 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:09.428982 25863 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:09.372831 26089 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.449707 26109 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.454105 26242 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.412909 26178 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.483349 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.432709 26246 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.488197 26204 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:09.488516 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.488646 25878 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:09.489223 25441 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:09.452700 26253 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:09.465076 26026 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:09.469041 26214 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.001s
I20260709 13:08:09.494745 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.369388 26105 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.496975 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2604) took 2923 ms. Trace:
I20260709 13:08:09.500277 26191 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:09.474004 26086 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:09.516984 26199 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:09.521059 26047 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:09.521453 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:09.469174 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.450803 26223 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:09.535972 26259 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:09.544027 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 63053.6us
I20260709 13:08:09.610023 21067 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:09.629194 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:09.630208 21067 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:09.608294 21214 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:09.609265 26032 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:09.620358 26155 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:09.542403 25934 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:09.496016 26157 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 13:08:09.616910 21218 raft_consensus.cc:2368] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14.
I20260709 13:08:09.642217 26245 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.643784 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.643849 26252 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.644640 21071 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:09.645606 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:09.646103 21071 raft_consensus.cc:2376] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13. Re-sending same reply.
W20260709 13:08:09.616873 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 122209us
I20260709 13:08:09.619267 21069 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:09.619115 21140 rpcz_store.cc:276] 0709 13:08:06.573964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.577303 (+  3339us) service_pool.cc:224] Handling call
0709 13:08:06.579128 (+  1825us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:07.666559 (+1087431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.496618 (+1830059us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400259ed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:09.496896 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2499291904}
I20260709 13:08:09.646888 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.647220 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.647861 25833 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:09.648658 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.649281 21069 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:09.649627 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:09.650094 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.539644 26111 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.651273 25833 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:09.651945 25833 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:09.652475 26228 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.652778 26111 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:09.653638 21216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2866) took 2994 ms (client timeout 2209 ms). Trace:
W20260709 13:08:09.654022 21216 rpcz_store.cc:269] 0709 13:08:06.659517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.805924 (+146407us) service_pool.cc:224] Handling call
0709 13:08:09.653417 (+2847493us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b54004420d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:09.653541 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42018048}
I20260709 13:08:09.631412 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.655846 21067 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:09.658380 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.660091 25963 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.618171 26026 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:09.619940 26200 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:09.638461 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.670635 21064 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:09.670980 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.653167 25833 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.650983 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:09.610728 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 91544.3us
I20260709 13:08:09.666313 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.667510 26243 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.669481 21147 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:09.672127 21067 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:09.620651 26214 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.677651 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.673619 21071 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:09.679067 26214 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.677958 21069 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:09.679183 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.681164 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.681478 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.682993 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.684625 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.685446 21063 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:09.687340 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.687489 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:09.688490 21144 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:09.691205 21214 raft_consensus.cc:2368] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:09.691934 21071 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:09.634294 26155 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:09.696002 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.696677 21221 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:09.668681 26245 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.661754 26252 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.662451 25878 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:09.693590 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:09.700127 21036 connection.cc:256] RPC call timeout handler was delayed by -2.0888600060000044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2195s for a response
I20260709 13:08:09.700419 21221 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:08:09.720780 21036 connection.cc:256] RPC call timeout handler was delayed by -2.0762600059999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 13:08:09.721136 21036 connection.cc:256] RPC call timeout handler was delayed by -2.0492600060000541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
I20260709 13:08:09.701534 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.701843 26246 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.704597 25930 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.670171 26086 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.708729 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.709086 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:09.711869 26266 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.247s	user 0.003s	sys 0.000s
I20260709 13:08:09.714028 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.729265 21218 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:09.714159 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.714807 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.721791 21216 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:09.683784 26261 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:09.730145 21067 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:09.730261 21215 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:09.719843 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.731735 21065 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:09.733770 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.723107 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.735766 21214 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:09.722419 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.689910 26111 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:08:09.737967 25762 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:09.740547 25878 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.744801 25863 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:08:09.746165 21217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 638) took 2627 ms (client timeout 1881 ms). Trace:
W20260709 13:08:09.746579 21217 rpcz_store.cc:269] 0709 13:08:07.119178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.659453 (+540275us) service_pool.cc:224] Handling call
0709 13:08:09.745975 (+2086522us) spinlock_profiling.cc:243] Waited 88.9 ms on lock 0x7b540034c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:09.746085 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204491648}
W20260709 13:08:09.747663 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.745204 26222 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.749171 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.691707 26247 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:09.764407 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:09.777484 21147 raft_consensus.cc:2368] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:08:09.778250 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.771605 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:09.667887 26259 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.780448 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.780717 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:09.713430 26155 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:09.788702 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:09.783393 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.783522 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:09.791426 21067 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:09.782526 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.690310 26265 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:08:09.745944 26260 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:09.786043 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 135121us
I20260709 13:08:09.828456 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:09.790366 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:09.829020 21141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 651) took 2152 ms (client timeout 1756 ms). Trace:
I20260709 13:08:09.829398 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2882) took 3165 ms. Trace:
I20260709 13:08:09.792906 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.780089 26083 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.687080 26243 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.794234 21065 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:09.781905 26086 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:09.795182 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:09.793341 26105 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:09.790190 26032 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.804291 21067 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:09.807505 26089 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:09.811014 26061 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:09.811993 26265 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:09.813344 26096 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.788831 26200 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.819803 25930 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:09.825374 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:09.826735 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:09.781303 26167 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:08:09.830482 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.830065 21212 rpcz_store.cc:276] 0709 13:08:06.663651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.015846 (+1352195us) service_pool.cc:224] Handling call
0709 13:08:08.639995 (+624149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.641466 (+  1471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.829148 (+1187682us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54000bc7d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:09.829327 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1434624256}
I20260709 13:08:09.830482 21147 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
W20260709 13:08:09.837294 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:09.859099 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62733.1us
W20260709 13:08:09.829488 21141 rpcz_store.cc:269] 0709 13:08:07.676634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.770089 (+ 93455us) service_pool.cc:224] Handling call
0709 13:08:09.828792 (+2058703us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540036db54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:09.828944 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2945630464}
I20260709 13:08:09.729274 26245 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.844540 26236 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:09.829815 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:09.844738 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49665.3us
I20260709 13:08:09.846259 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 947) took 1577 ms. Trace:
I20260709 13:08:09.750455 26242 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:08:09.847528 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2627) took 1189 ms. Trace:
I20260709 13:08:09.849588 21063 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:08:09.853281 21140 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:09.824136 26247 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.849185 26105 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:09.854432 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:09.823304 26261 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.824841 26178 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:09.890554 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.894910 21072 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:09.870211 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.863605 21794 rpcz_store.cc:276] 0709 13:08:08.268979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.269404 (+   425us) service_pool.cc:224] Handling call
0709 13:08:08.269790 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.468443 (+198653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:08.468487 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:08.468538 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:08.468586 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.845923 (+1377337us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b5400380254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:09.845995 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.846180 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456249088}
I20260709 13:08:09.864441 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:09.866333 21143 rpcz_store.cc:276] 0709 13:08:08.657781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.056284 (+398503us) service_pool.cc:224] Handling call
0709 13:08:09.056755 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.059620 (+  2865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.059656 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.060114 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:08:09.303541 (+243427us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:09.303581 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.303646 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.711274 (+407628us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:09.847225 (+135951us) raft_consensus.cc:1685] finished
0709 13:08:09.847246 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.847466 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8283,"raft.queue_time_us":3313,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":14272,"spinlock_wait_cycles":3425664,"thread_start_us":196725,"threads_started":1,"wal-append.queue_time_us":293883}
I20260709 13:08:09.896545 25934 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:09.896749 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.896868 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:09.853780 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:09.898006 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:09.899019 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.899596 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.896986 25934 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:09.895557 21072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2344) took 1726 ms (client timeout 1811 ms). Trace:
I20260709 13:08:09.900251 25934 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:09.900465 21072 rpcz_store.cc:269] 0709 13:08:08.170620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.844297 (+1673677us) service_pool.cc:224] Handling call
0709 13:08:09.895499 (+ 51202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:09.866976 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.903509 26199 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:09.907618 21212 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:09.907608 26199 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.908428 26199 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:09.909087 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.910504 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:09.911087 21212 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:09.912580 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.917337 26199 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.850765 25852 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:09.920604 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.853128 26265 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:09.866333 26236 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.917402 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:09.864159 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:09.900246 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.844013 26083 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.928908 21147 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:09.934466 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.935211 21147 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:09.936604 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.937295 21147 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:09.925858 21212 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:09.857609 25930 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.939433 25930 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:09.919634 26162 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:09.929775 26229 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:09.855206 26283 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:09.931833 26278 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.932101 26236 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:09.932878 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 68778.3us
I20260709 13:08:09.861007 26094 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:09.941744 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.938675 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.942425 21143 raft_consensus.cc:2410] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:08:09.908427 25934 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.943629 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.890738 26070 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:09.920305 26105 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:09.923795 25852 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:09.938867 26128 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:09.959694 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:09.988278 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 266616us
I20260709 13:08:09.943657 26178 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.869313 26228 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:08:09.952630 26229 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:08:09.991858 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 88707.2us
I20260709 13:08:09.943835 26162 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:09.994062 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.008337 21212 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:09.919524 26247 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:09.944453 26251 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.024425 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:10.025749 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.026005 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:09.993808 26253 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:09.992594 26070 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:10.027839 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8259401630000411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
I20260709 13:08:10.032621 21071 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:10.032150 21036 connection.cc:256] RPC call timeout handler was delayed by -1.7315600059999952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25920000000000004s for a response
I20260709 13:08:10.025763 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:09.993176 26283 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:10.030776 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 177119us
I20260709 13:08:09.998742 25852 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:10.035102 21765 leader_election.cc:400] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14.
I20260709 13:08:10.035259 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.035638 21765 leader_election.cc:403] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:08:10.004434 26191 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:10.036099 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.037084 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.038460 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.039191 21217 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:10.039762 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.040544 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.009009 26229 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:10.046455 21214 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:09.999913 26278 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.044219 21141 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:10.050539 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.055419 21801 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:10.033507 26070 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:10.064533 21212 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:08:10.064608 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.068540 21797 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:10.006565 26178 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.070501 21217 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:10.049804 25833 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:10.071902 21036 connection.cc:256] RPC call timeout handler was delayed by -1.6163600060000363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.272s for a response
W20260709 13:08:10.072371 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5750026080000907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260709 13:08:10.072747 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4937823430000208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 13:08:10.073112 21036 connection.cc:256] RPC call timeout handler was delayed by -1.448960005999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2906s for a response
W20260709 13:08:10.073482 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3832600060000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2979s for a response
W20260709 13:08:10.073843 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3387647550000565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2635s for a response
W20260709 13:08:10.074208 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3092823430000635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:08:10.074579 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2150026080000771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23260000000000003s for a response
W20260709 13:08:10.074945 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0520600060000334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3347s for a response
I20260709 13:08:10.039413 26191 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:10.076433 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.077065 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.041149 26295 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.078584 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.079605 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.079741 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.084174 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.082376 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.084159 21796 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:10.089951 21217 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:10.090946 21035 leader_election.cc:400] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
W20260709 13:08:10.091269 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 321274us
I20260709 13:08:10.091316 21035 leader_election.cc:403] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:08:10.089413 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.096552 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:10.098567 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.099135 21221 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:10.101667 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:10.057945 26167 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:10.103163 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.103965 25495 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:10.103829 26167 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.104401 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:10.105208 25495 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:10.105695 25495 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:10.106530 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.101162 21797 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEARNER]: Advancing to term 12
I20260709 13:08:10.080035 26191 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:10.122706 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.097890 26243 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.126020 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.037822 26265 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.110522 25889 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:08:10.095719 26261 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:10.140986 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:10.202590 26288 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:08:10.095871 26162 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.129403 26083 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:10.230036 26260 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.186480 26096 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:10.267737 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.091853 26295 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:10.296339 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:10.376940 26285 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.282s	user 0.008s	sys 0.001s
W20260709 13:08:10.379927 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.380735 26245 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:10.381351 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2890) took 1081 ms. Trace:
I20260709 13:08:10.433467 21213 rpcz_store.cc:276] 0709 13:08:09.300165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.301045 (+   880us) service_pool.cc:224] Handling call
0709 13:08:09.301319 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.471250 (+169931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.471305 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.471759 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 13:08:09.607428 (+135669us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:09.615509 (+  8081us) log.cc:844] Serialized 14 byte log entry
0709 13:08:09.615734 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.812222 (+196488us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b1800474400. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:10.380988 (+568766us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400102f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:10.381051 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.381097 (+    46us) raft_consensus.cc:1685] finished
0709 13:08:10.381118 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.381285 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1910,"prepare.queue_time_us":11280,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":3367,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":8025,"spinlock_wait_cycles":393650688,"thread_start_us":135261,"threads_started":1,"wal-append.queue_time_us":153694}
W20260709 13:08:10.382447 26155 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.569s	user 0.022s	sys 0.004s
I20260709 13:08:10.383993 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1549) took 1192 ms. Trace:
I20260709 13:08:10.444186 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.444336 21071 rpcz_store.cc:276] 0709 13:08:09.191970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.696013 (+504043us) service_pool.cc:224] Handling call
0709 13:08:09.696376 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.035839 (+339463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.383732 (+347893us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54002acbd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:10.383933 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":769906944}
I20260709 13:08:10.392871 26178 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:10.385432 26105 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.385888 26094 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.387437 25934 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.109500 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:10.388188 26298 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:10.390607 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.035166 26283 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.391350 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.388195 26241 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:10.476253 26241 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:10.476522 26105 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:10.392357 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:10.108431 26217 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:08:10.093487 26204 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.159s
I20260709 13:08:10.478147 26204 consensus_queue.cc:1243] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:10.106541 25495 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.479525 26101 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.376163 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:10.481637 25863 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:10.482380 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 372433us
I20260709 13:08:10.483258 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:10.484130 26246 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:10.484230 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:10.484738 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:10.485271 21142 raft_consensus.cc:2393] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:08:10.485688 21142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3032) took 1370 ms (client timeout 1790 ms). Trace:
I20260709 13:08:10.396293 21221 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
W20260709 13:08:10.486027 21142 rpcz_store.cc:269] 0709 13:08:09.115067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.148557 (+ 33490us) service_pool.cc:224] Handling call
0709 13:08:10.484049 (+1335492us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:08:10.485628 (+  1579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39141248}
I20260709 13:08:10.486869 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.472222 21214 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
W20260709 13:08:10.403080 21144 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.003s
I20260709 13:08:10.403790 26256 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:08:10.382553 26254 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.412s	user 0.008s	sys 0.007s
W20260709 13:08:10.416992 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.381248 25852 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.418892 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:10.446043 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.384652 26242 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.448123 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:10.413443 26047 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.785s	user 0.012s	sys 0.008s
I20260709 13:08:10.393414 26282 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:10.450320 21147 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:10.508080 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3040) took 1390 ms. Trace:
I20260709 13:08:10.508556 21147 rpcz_store.cc:276] 0709 13:08:09.117175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.405905 (+1288730us) service_pool.cc:224] Handling call
0709 13:08:10.508012 (+102107us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:10.461303 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 351826us
W20260709 13:08:10.461596 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71113.1us
I20260709 13:08:10.512944 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:10.467564 26281 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.482s	user 0.003s	sys 0.006s
W20260709 13:08:10.467968 26293 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.342s	user 0.000s	sys 0.009s
I20260709 13:08:10.392354 26182 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:08:10.484820 26246 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:10.397425 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:10.538817 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:10.539188 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:10.539552 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.396416 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.491643 21221 raft_consensus.cc:2410] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:08:10.487258 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:10.541108 21142 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 10: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:10.541388 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.490782 25934 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:10.542128 21221 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 8.
I20260709 13:08:10.543411 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.492273 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.492703 21797 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:10.492789 21144 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:10.546039 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3036) took 1429 ms. Trace:
I20260709 13:08:10.546393 21144 rpcz_store.cc:276] 0709 13:08:09.116130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.787137 (+671007us) service_pool.cc:224] Handling call
0709 13:08:10.545957 (+758820us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.547299 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.511858 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:10.548108 21144 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:10.460188 26128 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.505895 26111 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:10.451432 26155 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.473351 26211 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:10.455014 26162 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:10.561411 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3034) took 1445 ms. Trace:
I20260709 13:08:10.537325 26263 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:10.562909 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 70731.1us
I20260709 13:08:10.576671 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.564143 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3033) took 1448 ms. Trace:
I20260709 13:08:10.540970 26246 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:10.577392 21144 rpcz_store.cc:276] 0709 13:08:09.115344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.547034 (+1431690us) service_pool.cc:224] Handling call
0709 13:08:10.564082 (+ 17048us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.504230 26217 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:10.533775 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2352) took 1257 ms. Trace:
I20260709 13:08:10.564781 25898 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:10.566807 26282 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:10.567240 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.561510 26247 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.572737 21142 rpcz_store.cc:276] 0709 13:08:09.115635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.487009 (+1371374us) service_pool.cc:224] Handling call
0709 13:08:10.561338 (+ 74329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.530200 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:08:10.604882 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 74777.3us
I20260709 13:08:10.605629 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2617) took 3689 ms. Trace:
I20260709 13:08:10.528836 26256 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:10.578400 21063 rpcz_store.cc:276] 0709 13:08:09.276226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.919603 (+643377us) service_pool.cc:224] Handling call
0709 13:08:09.930439 (+ 10836us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.930850 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:09.931178 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.016945 (+ 85767us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:10.028345 (+ 11400us) log.cc:844] Serialized 26 byte log entry
0709 13:08:10.028517 (+   172us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:10.028553 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.028625 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.387749 (+359124us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b4800080358. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:08:10.508114 (+120365us) raft_consensus.cc:1685] finished
0709 13:08:10.533547 (+ 25433us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.533700 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6562,"prepare.queue_time_us":9247,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":15901,"raft.run_cpu_time_us":2023,"raft.run_wall_time_us":362665,"spinlock_wait_cycles":5665152,"thread_start_us":78702,"threads_started":1,"wal-append.queue_time_us":162850}
I20260709 13:08:10.606029 21139 rpcz_store.cc:276] 0709 13:08:06.916012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.916267 (+   255us) service_pool.cc:224] Handling call
0709 13:08:06.916680 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.236341 (+1319661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.605389 (+2369048us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540020add4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:10.605574 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2881992448}
I20260709 13:08:10.607221 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:10.607151 26256 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.607856 25852 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.507457 26047 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.608534 25852 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:10.609058 25852 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:10.609431 25852 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:10.610389 26047 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.607411 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:10.565977 26111 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.610733 26241 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.611243 21139 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:10.463052 26094 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.589141 21147 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:08:10.589250 21794 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:10.608691 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:10.613108 21063 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:10.608451 26256 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:10.573436 25963 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.611316 26111 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:10.612246 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2628) took 1235 ms. Trace:
I20260709 13:08:10.623039 21147 rpcz_store.cc:276] 0709 13:08:09.377108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.535565 (+1158457us) service_pool.cc:224] Handling call
0709 13:08:10.612180 (+ 76615us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:10.591034 26061 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.735s	user 0.008s	sys 0.007s
I20260709 13:08:10.611898 21064 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:10.527429 26242 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:10.582293 26236 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.001s
I20260709 13:08:10.588312 25833 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14.
I20260709 13:08:10.622112 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.638075 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3043) took 1520 ms. Trace:
I20260709 13:08:10.638592 21142 rpcz_store.cc:276] 0709 13:08:09.118009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.617904 (+1499895us) service_pool.cc:224] Handling call
0709 13:08:10.637987 (+ 20083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.640326 21219 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:10.587658 25898 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.657027 25898 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:10.658107 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47994.1us
W20260709 13:08:10.599506 26284 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.384s	user 0.000s	sys 0.010s
W20260709 13:08:10.661352 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3050) took 1396 ms (client timeout 1794 ms). Trace:
W20260709 13:08:10.661788 21140 rpcz_store.cc:269] 0709 13:08:09.264468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.646959 (+382491us) service_pool.cc:224] Handling call
0709 13:08:10.661134 (+1014175us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54002e4354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:10.661287 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469404544}
I20260709 13:08:10.663520 26251 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.663671 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2351) took 1387 ms. Trace:
W20260709 13:08:10.636969 26167 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.025s	sys 0.043s
I20260709 13:08:10.664266 21067 rpcz_store.cc:276] 0709 13:08:09.275944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.850732 (+574788us) service_pool.cc:224] Handling call
0709 13:08:09.851603 (+   871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.857035 (+  5432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.857081 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.868524 (+ 11443us) log.cc:844] Serialized 26 byte log entry
0709 13:08:10.100318 (+231794us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:10.219701 (+119383us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.397242 (+177541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.408536 (+ 11294us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.609020 (+200484us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b18001a7be0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:10.663355 (+ 54335us) raft_consensus.cc:1685] finished
0709 13:08:10.663464 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.663615 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":119377,"prepare.run_cpu_time_us":8670,"prepare.run_wall_time_us":176836,"raft.queue_time_us":916,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":12166,"spinlock_wait_cycles":2397056,"thread_start_us":147392,"threads_started":1,"wal-append.queue_time_us":617985}
I20260709 13:08:10.672186 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2358) took 1197 ms. Trace:
W20260709 13:08:10.634133 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57536.2us
I20260709 13:08:10.630904 21139 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:10.620553 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.674081 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.675115 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.676151 21139 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:08:10.663153 21140 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
W20260709 13:08:10.643157 21799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 580) took 1482 ms (client timeout 1874 ms). Trace:
I20260709 13:08:10.646559 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.677838 21139 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:10.652567 26199 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:10.653544 26070 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.002s
I20260709 13:08:10.654542 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2350) took 1379 ms. Trace:
I20260709 13:08:10.624930 26261 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:10.641649 26074 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.628679 21215 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:10.621060 26086 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.768s	user 0.007s	sys 0.007s
I20260709 13:08:10.642130 21143 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 14. Ops: [12.9-12.9]
I20260709 13:08:10.707154 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3041) took 1589 ms. Trace:
I20260709 13:08:10.708154 21143 rpcz_store.cc:276] 0709 13:08:09.117463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.641564 (+1524101us) service_pool.cc:224] Handling call
0709 13:08:10.641883 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:10.706746 (+ 64863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.707071 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.651180 26094 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:10.675448 21067 rpcz_store.cc:276] 0709 13:08:09.474285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.667730 (+1193445us) service_pool.cc:224] Handling call
0709 13:08:10.671480 (+  3750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.671740 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:10.671775 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:10.671835 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:10.671886 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.671966 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.672108 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.725065 21067 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:10.725869 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 839) took 1075 ms. Trace:
I20260709 13:08:10.726289 21067 rpcz_store.cc:276] 0709 13:08:09.650624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.724384 (+1073760us) service_pool.cc:224] Handling call
0709 13:08:10.724833 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.725566 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.725801 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.679129 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3045) took 1560 ms. Trace:
I20260709 13:08:10.727887 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2895) took 1034 ms. Trace:
I20260709 13:08:10.727471 21139 rpcz_store.cc:276] 0709 13:08:09.118554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.677237 (+1558683us) service_pool.cc:224] Handling call
0709 13:08:10.677605 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.678407 (+   802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.679076 (+   669us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.728353 21219 rpcz_store.cc:276] 0709 13:08:09.693900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.694073 (+   173us) service_pool.cc:224] Handling call
0709 13:08:10.727727 (+1033654us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b5400442ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:10.727832 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087160192}
I20260709 13:08:10.729825 21139 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [10.9-11.10]
I20260709 13:08:10.730358 26086 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.730684 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3048) took 1467 ms. Trace:
I20260709 13:08:10.679319 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 837) took 1630 ms. Trace:
I20260709 13:08:10.731029 26086 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.731436 21066 rpcz_store.cc:276] 0709 13:08:09.048888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.049281 (+   393us) service_pool.cc:224] Handling call
0709 13:08:09.054476 (+  5195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.356027 (+301551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.679043 (+1323016us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b540041c654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:10.679223 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647968128}
I20260709 13:08:10.733336 21066 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:10.734079 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 842) took 1070 ms. Trace:
I20260709 13:08:10.733969 26086 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.735783 26228 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:10.736361 26228 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.738708 26228 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:10.739989 26228 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:10.740492 26242 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:10.740610 26228 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.741258 26277 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:10.741974 26277 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:10.743815 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:10.745369 26228 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:10.746143 21216 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:10.746747 21792 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:10.747715 26295 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:10.685139 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.749224 26278 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:10.749464 26094 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.750038 26094 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.750588 25817 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:10.751127 25817 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:10.752498 26094 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:10.749255 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:10.686756 21036 leader_election.cc:400] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:10.763799 21036 leader_election.cc:403] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:10.763960 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:10.765733 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:10.681874 21070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2334) took 4044 ms (client timeout 2159 ms). Trace:
I20260709 13:08:10.655951 21147 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: [9.8-12.9]
W20260709 13:08:10.771514 21070 rpcz_store.cc:269] 0709 13:08:06.637235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.756332 (+119097us) service_pool.cc:224] Handling call
0709 13:08:10.681566 (+3925234us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b54003bf954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:10.681775 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5291307392}
I20260709 13:08:10.772130 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3039) took 1655 ms. Trace:
W20260709 13:08:10.677095 21799 rpcz_store.cc:269] 0709 13:08:09.160720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.160890 (+   170us) service_pool.cc:224] Handling call
0709 13:08:10.642950 (+1482060us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54005e01d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:10.643098 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267469056}
I20260709 13:08:10.772802 21147 rpcz_store.cc:276] 0709 13:08:09.116918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.640007 (+1523089us) service_pool.cc:224] Handling call
0709 13:08:10.655699 (+ 15692us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:10.771740 (+116041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.772058 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.688496 21065 rpcz_store.cc:276] 0709 13:08:09.275610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.832514 (+556904us) service_pool.cc:224] Handling call
0709 13:08:09.832852 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.851625 (+ 18773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.851657 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.852157 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:08:09.992880 (+140723us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:10.008298 (+ 15418us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.008620 (+   322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.008721 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.654343 (+645622us) raft_consensus.cc:1685] finished
0709 13:08:10.654363 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.654486 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15485,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":2268,"raft.queue_time_us":8443,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":17576,"thread_start_us":64658,"threads_started":1,"wal-append.queue_time_us":134942}
I20260709 13:08:10.685388 21146 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:10.663447 21142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:10.716264 26155 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.718509 26111 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.719389 26074 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.700958 26070 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.721508 26109 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:10.779276 21216 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:08:10.747797 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.649274 26236 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:10.780498 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 93945.1us
I20260709 13:08:10.752278 25817 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:10.754009 26229 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.690s	user 0.007s	sys 0.004s
W20260709 13:08:10.708251 26026 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.027s	user 0.016s	sys 0.003s
I20260709 13:08:10.699046 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.781843 21145 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:10.774524 21142 raft_consensus.cc:606] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:10.734555 21066 rpcz_store.cc:276] 0709 13:08:09.663987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.732681 (+1068694us) service_pool.cc:224] Handling call
0709 13:08:10.733111 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.733805 (+   694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.734015 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.782616 21142 raft_consensus.cc:614] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:10.775784 21215 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:10.731217 21139 rpcz_store.cc:276] 0709 13:08:09.263779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.728969 (+1465190us) service_pool.cc:224] Handling call
0709 13:08:10.729440 (+   471us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:10.730362 (+   922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.730618 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:10.776904 21146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3026) took 1663 ms (client timeout 1694 ms). Trace:
I20260709 13:08:10.695986 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3044) took 1577 ms. Trace:
I20260709 13:08:10.756728 26265 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:10.784905 21140 rpcz_store.cc:276] 0709 13:08:09.118293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.662645 (+1544352us) service_pool.cc:224] Handling call
0709 13:08:10.662982 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.695712 (+ 32730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.695927 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.777945 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.786988 21143 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:10.787422 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.788225 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:10.788466 21143 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:08:10.784370 21146 rpcz_store.cc:269] 0709 13:08:09.113210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.124112 (+ 10902us) service_pool.cc:224] Handling call
0709 13:08:10.646382 (+1522270us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:08:10.776819 (+130437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15000832}
I20260709 13:08:10.801117 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.801134 21146 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:10.802785 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32968 (request call id 23) took 1310 ms. Trace:
I20260709 13:08:10.803318 21142 rpcz_store.cc:276] 0709 13:08:09.474732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.639444 (+1164712us) service_pool.cc:224] Handling call
0709 13:08:10.783225 (+143781us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.747694 25852 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.804935 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:10.805600 21070 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:10.806762 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 659) took 1430 ms. Trace:
I20260709 13:08:10.807153 21146 rpcz_store.cc:276] 0709 13:08:09.376071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.790429 (+1414358us) service_pool.cc:224] Handling call
0709 13:08:10.800883 (+ 10454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.806498 (+  5615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.806698 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:10.777506 25930 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.825s	user 0.003s	sys 0.017s
I20260709 13:08:10.813352 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.817713 21146 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:10.652623 26191 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.819399 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 662) took 1442 ms. Trace:
I20260709 13:08:10.819785 26196 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:10.819901 21146 rpcz_store.cc:276] 0709 13:08:09.376859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.807967 (+1431108us) service_pool.cc:224] Handling call
0709 13:08:10.817137 (+  9170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.819126 (+  1989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.819345 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.801762 26026 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.821202 21217 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:10.779537 26054 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:10.822247 26026 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.825492 21799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:08:10.826226 21799 raft_consensus.cc:606] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:10.818203 21147 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:10.820047 21213 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:10.804103 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 658) took 1428 ms. Trace:
I20260709 13:08:10.828552 21143 rpcz_store.cc:276] 0709 13:08:09.375384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.787746 (+1412362us) service_pool.cc:224] Handling call
0709 13:08:10.788054 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.789020 (+   966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.804037 (+ 15017us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.829530 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:10.816471 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:10.830258 21066 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:10.814246 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:10.808001 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:10.804948 21142 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:10.801872 21139 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:10.831154 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 660) took 1454 ms. Trace:
I20260709 13:08:10.828554 21212 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:10.831243 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.828958 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.776360 26167 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:10.831688 21142 rpcz_store.cc:276] 0709 13:08:09.376350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.804360 (+1428010us) service_pool.cc:224] Handling call
0709 13:08:10.804651 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.830923 (+ 26272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.831106 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.838604 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:10.839138 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:10.840468 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:10.779786 26295 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:10.833923 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:10.842074 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.869393 21792 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:10.843952 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:10.870476 21146 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:10.844076 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:10.872330 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:10.874774 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:10.875751 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.877174 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:10.874967 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 43807.4us
I20260709 13:08:10.806085 25852 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:10.828140 26238 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:10.888049 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.818109 26094 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:10.802177 26261 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:08:10.874619 21800 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
W20260709 13:08:10.880055 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46186.9us
I20260709 13:08:10.868746 26295 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.802472 26229 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.878096 25817 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:10.889340 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:10.841528 26059 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:10.841394 26073 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:10.910290 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1607) took 1043 ms. Trace:
I20260709 13:08:10.886416 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.893201 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.887737 26199 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:10.878888 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.895807 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:10.897511 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.914330 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.808588 26217 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:08:10.844890 26086 raft_consensus.cc:2764] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:08:10.866245 26228 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:10.743165 26277 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.884168 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:10.881685 26104 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:10.904758 21066 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:10.907136 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:10.886088 26109 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:10.881950 26099 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:10.829183 25930 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:10.891628 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2356) took 1559 ms. Trace:
I20260709 13:08:10.894187 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.911875 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.890993 26291 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:10.886981 26251 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:10.950229 21219 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:10.887943 26238 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:10.917353 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:10.919238 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:10.921339 25495 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:10.924798 26073 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:10.968060 21145 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:10.961288 26238 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:10.969658 26073 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.970533 25930 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:08:10.963351 26210 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:10.972994 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.976482 26109 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.977020 26109 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:10.980264 21141 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:10.980931 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2631) took 1577 ms. Trace:
I20260709 13:08:10.981284 21141 rpcz_store.cc:276] 0709 13:08:09.403964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.979756 (+1575792us) service_pool.cc:224] Handling call
0709 13:08:10.980078 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.980687 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.980878 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:10.985690 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.988631 26074 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:10.992148 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:10.993434 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:10.994545 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:10.995814 26111 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:10.997150 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1613) took 1099 ms. Trace:
I20260709 13:08:10.997535 21212 rpcz_store.cc:276] 0709 13:08:09.897416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.645451 (+748035us) service_pool.cc:224] Handling call
0709 13:08:10.645764 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.832821 (+187057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.996916 (+164095us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54004443d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:10.997079 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419922688}
I20260709 13:08:10.998759 26229 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:11.004671 26215 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:08:11.007439 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.924217 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:11.008404 21218 raft_consensus.cc:2376] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16. Re-sending same reply.
I20260709 13:08:10.903512 25762 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:10.901727 26295 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:11.012432 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.937016 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:11.013121 21218 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:11.013581 21066 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:10.939347 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:10.911890 26059 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:10.946596 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:10.944872 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60799.3us
I20260709 13:08:10.892995 25898 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:10.949103 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.921089 26083 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:11.027631 26191 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:11.030751 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1559) took 1752 ms. Trace:
I20260709 13:08:10.892558 26094 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.031713 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:11.031246 21072 rpcz_store.cc:276] 0709 13:08:09.278418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.092458 (+814040us) service_pool.cc:224] Handling call
0709 13:08:10.111812 (+ 19354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:10.388281 (+276469us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:10.411099 (+ 22818us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.843982 (+432883us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:10.873421 (+ 29439us) log.cc:844] Serialized 52 byte log entry
0709 13:08:10.873582 (+   161us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:10.873612 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.873673 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.030489 (+156816us) raft_consensus.cc:1685] finished
0709 13:08:11.030527 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.030689 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":79252,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":38598,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":13191,"thread_start_us":368031,"threads_started":2,"wal-append.queue_time_us":479248}
I20260709 13:08:10.945401 21071 rpcz_store.cc:276] 0709 13:08:09.332603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.445547 (+1112944us) service_pool.cc:224] Handling call
0709 13:08:10.445903 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:10.446307 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:10.446338 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:10.446861 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:08:10.671945 (+225084us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:10.672002 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.672093 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.891370 (+219277us) raft_consensus.cc:1685] finished
0709 13:08:10.891398 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.891557 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37,"raft.queue_time_us":869,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":6395,"thread_start_us":183727,"threads_started":1,"wal-append.queue_time_us":373880}
I20260709 13:08:11.033715 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:11.033946 25898 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:11.034444 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:11.035029 21071 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:11.034965 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:11.017535 26109 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:11.035967 21072 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:08:10.958612 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.036425 21218 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:11.036834 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:10.960219 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 44347.9us
I20260709 13:08:10.914222 26199 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.922744 21797 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:11.039932 21069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.031s	sys 0.047s
I20260709 13:08:10.962785 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:10.948310 26291 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.950443 26086 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:10.998771 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:11.017930 21070 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:10.995275 21065 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:11.005522 26218 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:08:11.021130 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.019371 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:11.025197 26251 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:08:11.025684 26303 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:08:10.949776 26246 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.125149 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:11.126694 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:11.017357 26229 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:10.956275 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:10.958397 21215 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:10.922525 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2357) took 1451 ms. Trace:
I20260709 13:08:11.038247 21072 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:10.921695 26228 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:11.040942 21069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.032s	sys 0.048s
I20260709 13:08:11.005766 26215 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:08:11.039865 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:11.149701 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.020483 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:11.044827 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 82105.9us
I20260709 13:08:11.084829 21212 raft_consensus.cc:2376] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10. Re-sending same reply.
I20260709 13:08:11.031522 26094 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:11.087688 21139 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:08:11.123832 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 104551us
I20260709 13:08:11.153299 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2633) took 1445 ms. Trace:
I20260709 13:08:11.132354 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:11.153726 21139 rpcz_store.cc:276] 0709 13:08:09.708024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.087056 (+1379032us) service_pool.cc:224] Handling call
0709 13:08:11.087444 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.153001 (+ 65557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.153245 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.155766 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2635) took 1443 ms. Trace:
I20260709 13:08:11.022284 26059 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:11.156136 21139 rpcz_store.cc:276] 0709 13:08:09.712626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.154768 (+1442142us) service_pool.cc:224] Handling call
0709 13:08:11.155137 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.155374 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.155411 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.155469 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:11.155514 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.155586 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.155706 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.159462 21139 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:11.135026 21796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce55364f10c348819e7c63c08633b184"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:08:11.164574 21796 raft_consensus.cc:606] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:11.165135 21796 raft_consensus.cc:614] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:11.042338 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:11.087397 26217 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:11.043746 26199 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:11.176769 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:11.177433 21072 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:11.179468 26242 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:11.180052 26242 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:11.180928 26242 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:11.182513 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 162134us
I20260709 13:08:11.023223 25762 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.144375 26083 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:11.147893 21071 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:11.137171 21063 rpcz_store.cc:276] 0709 13:08:09.470640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.613813 (+1143173us) service_pool.cc:224] Handling call
0709 13:08:10.614224 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.614419 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:10.631916 (+ 17497us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.922037 (+290121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:10.922090 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:10.922249 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.922321 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.922463 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6404,"prepare.queue_time_us":19512,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":1850,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":9132,"thread_start_us":288881,"threads_started":1}
W20260709 13:08:11.150154 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.150746 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:11.085042 26263 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.507s	user 0.002s	sys 0.008s
I20260709 13:08:11.152626 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:11.192662 21066 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:11.152639 26228 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:11.171697 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2638) took 1442 ms. Trace:
I20260709 13:08:11.196893 21139 rpcz_store.cc:276] 0709 13:08:09.729423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.158811 (+1429388us) service_pool.cc:224] Handling call
0709 13:08:11.159202 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.171363 (+ 12161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.171625 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:11.174778 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 132550us
I20260709 13:08:11.171319 26217 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.206430 26217 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:11.173305 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1560) took 1894 ms. Trace:
I20260709 13:08:11.214843 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 840) took 1564 ms. Trace:
W20260709 13:08:11.216094 26310 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:08:11.214677 21069 rpcz_store.cc:276] 0709 13:08:09.278685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.378091 (+1099406us) service_pool.cc:224] Handling call
0709 13:08:10.378415 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.378611 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:10.410391 (+ 31780us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.172630 (+762239us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.172867 (+   237us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.172930 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:11.172996 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.173087 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.173229 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7393,"prepare.queue_time_us":79217,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":90,"raft.queue_time_us":7574,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":12994,"spinlock_wait_cycles":24089856,"thread_start_us":768852,"threads_started":2,"wal-append.queue_time_us":801575}
I20260709 13:08:11.215549 21067 rpcz_store.cc:276] 0709 13:08:09.650893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.727201 (+1076308us) service_pool.cc:224] Handling call
0709 13:08:10.781448 (+ 54247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:10.801293 (+ 19845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:10.801326 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:10.816720 (+ 15394us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.019461 (+202741us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:11.019502 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.019569 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.214615 (+195046us) raft_consensus.cc:1685] finished
0709 13:08:11.214637 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.214782 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26019,"raft.queue_time_us":22132,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":14315,"thread_start_us":125828,"threads_started":1,"wal-append.queue_time_us":311820}
I20260709 13:08:11.217402 21069 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:11.217710 26307 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.261s	user 0.000s	sys 0.000s
I20260709 13:08:11.218011 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 846) took 1533 ms. Trace:
I20260709 13:08:11.218377 21069 rpcz_store.cc:276] 0709 13:08:09.684661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.216904 (+1532243us) service_pool.cc:224] Handling call
0709 13:08:11.217215 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.217801 (+   586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.217962 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.220351 21069 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
W20260709 13:08:11.220768 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.221024 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1563) took 1512 ms. Trace:
I20260709 13:08:11.221551 21069 rpcz_store.cc:276] 0709 13:08:09.708991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.219911 (+1510920us) service_pool.cc:224] Handling call
0709 13:08:11.220187 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.220826 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.220976 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.222728 21069 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:11.223547 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1565) took 1513 ms. Trace:
I20260709 13:08:11.225574 21069 rpcz_store.cc:276] 0709 13:08:09.709688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.222328 (+1512640us) service_pool.cc:224] Handling call
0709 13:08:11.222581 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.223350 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.223502 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.187465 26282 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:11.148411 26167 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:11.135943 26229 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:11.195019 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 843) took 1530 ms. Trace:
I20260709 13:08:11.255581 21071 rpcz_store.cc:276] 0709 13:08:09.664361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.194087 (+1529726us) service_pool.cc:224] Handling call
0709 13:08:11.194410 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.194604 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.194652 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.194707 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:11.194750 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.194820 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.194951 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.085307 26191 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 13:08:11.183025 26047 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.088027 26291 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:11.187505 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.264312 25762 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:11.189667 25441 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:11.230779 21066 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:08:11.198884 26263 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:11.252805 21069 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:08:11.258821 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.272817 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 845) took 1607 ms. Trace:
I20260709 13:08:11.272837 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 847) took 1457 ms. Trace:
I20260709 13:08:11.273344 21066 rpcz_store.cc:276] 0709 13:08:09.664988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.200230 (+1535242us) service_pool.cc:224] Handling call
0709 13:08:11.230467 (+ 30237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.272447 (+ 41980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.272738 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.274039 21069 rpcz_store.cc:276] 0709 13:08:09.815704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.252083 (+1436379us) service_pool.cc:224] Handling call
0709 13:08:11.252496 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.272544 (+ 20048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.272757 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.275158 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:11.223618 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:11.250998 26261 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:11.281473 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:11.254030 26074 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 13:08:11.286226 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 62722.5us
I20260709 13:08:11.283072 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:11.089148 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:11.287708 21069 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:11.288784 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:11.285524 21795 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:11.296561 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:11.296689 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:11.308225 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:11.313274 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2901) took 1589 ms. Trace:
I20260709 13:08:11.339932 21214 rpcz_store.cc:276] 0709 13:08:09.723829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.506778 (+782949us) service_pool.cc:224] Handling call
0709 13:08:10.541100 (+ 34322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:10.541509 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:10.541549 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:10.565418 (+ 23869us) log.cc:844] Serialized 26 byte log entry
0709 13:08:10.913896 (+348478us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:11.306420 (+392524us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.312940 (+  6520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.313013 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.313053 (+    40us) raft_consensus.cc:1685] finished
0709 13:08:11.313067 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.313203 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9818,"raft.queue_time_us":4176,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4483,"thread_start_us":671699,"threads_started":2,"wal-append.queue_time_us":512910}
I20260709 13:08:11.313716 21072 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:11.257793 26238 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.293228 25762 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:11.293407 25441 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:11.352522 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:11.297227 26278 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:11.271989 26047 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.306735 26289 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:11.346055 26301 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:08:11.340389 25863 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:11.366111 21065 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:11.299933 26260 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:08:11.341465 25889 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:11.339468 26144 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:11.377840 26167 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.378366 26167 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:11.379328 26251 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:11.379729 26251 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:11.380558 26251 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.381778 26251 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:11.385035 26054 consensus_meta.cc:287] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.000s
I20260709 13:08:11.385478 26054 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:11.349288 26217 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.387780 21139 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:11.387830 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2629) took 2009 ms. Trace:
I20260709 13:08:11.343560 26229 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.388715 21138 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:08:11.388286 26217 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:11.389225 26229 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:11.375524 25889 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.392067 26167 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.398119 26210 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:11.369990 26278 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:11.363263 26074 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:11.399732 26246 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:11.400004 26047 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:11.369038 26099 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:11.404349 21140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:11.407392 26226 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.412915 21142 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:11.439178 21140 raft_consensus.cc:606] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:11.439600 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 665) took 1352 ms. Trace:
I20260709 13:08:11.440078 21142 rpcz_store.cc:276] 0709 13:08:10.087390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.412056 (+1324666us) service_pool.cc:224] Handling call
0709 13:08:11.412468 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.439257 (+ 26789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.439537 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.441942 21142 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:11.442579 21071 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:11.414757 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:11.438544 26054 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.445881 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:11.419096 21064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1566) took 1709 ms (client timeout 1845 ms). Trace:
W20260709 13:08:11.447621 21064 rpcz_store.cc:269] 0709 13:08:09.710030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.719865 (+  9835us) service_pool.cc:224] Handling call
0709 13:08:11.418810 (+1698945us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b54000dc354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:11.419011 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1902316416}
I20260709 13:08:11.381489 26295 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.388289 21147 rpcz_store.cc:276] 0709 13:08:09.378143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.878036 (+1499893us) service_pool.cc:224] Handling call
0709 13:08:10.898136 (+ 20100us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:11.016658 (+118522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.016692 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:11.028405 (+ 11713us) log.cc:844] Serialized 52 byte log entry
0709 13:08:11.187536 (+159131us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:11.187580 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.380912 (+193332us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54003be554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:11.380961 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.387497 (+  6536us) raft_consensus.cc:1685] finished
0709 13:08:11.387520 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.387770 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2189,"raft.queue_time_us":2659,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":8748,"spinlock_wait_cycles":253167360,"thread_start_us":158576,"threads_started":1,"wal-append.queue_time_us":320451}
I20260709 13:08:11.427022 21068 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:08:11.427985 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:11.450423 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.450822 21140 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:11.438746 26086 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:11.446856 25817 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
I20260709 13:08:11.466636 21064 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:08:11.449944 26265 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.001s
W20260709 13:08:11.434053 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.366531 26289 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:11.452950 26314 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:08:11.486671 21142 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
W20260709 13:08:11.489441 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.448998 26191 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:11.492372 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 583) took 1447 ms. Trace:
I20260709 13:08:11.496914 21140 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:11.452121 26162 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:11.494613 21794 rpcz_store.cc:276] 0709 13:08:10.045109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.045294 (+   185us) service_pool.cc:224] Handling call
0709 13:08:10.045589 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.612651 (+567062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.492002 (+879351us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54004934d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:11.492286 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248551808}
I20260709 13:08:11.447065 26054 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.428633 21072 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:11.506101 21064 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:08:11.443852 26226 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:11.509418 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 201244us
I20260709 13:08:11.516546 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:11.520226 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2640) took 1523 ms. Trace:
I20260709 13:08:11.520934 21143 rpcz_store.cc:276] 0709 13:08:09.997270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.406579 (+1409309us) service_pool.cc:224] Handling call
0709 13:08:11.406891 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.407095 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:11.407431 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.519725 (+112294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.519799 (+    74us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:11.519866 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.519958 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.520126 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3808,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":132,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"thread_start_us":111523,"threads_started":1}
I20260709 13:08:11.477520 26074 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:08:11.539422 26319 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:08:11.539920 26074 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:08:11.540839 26320 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:08:11.459854 25863 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:08:11.524570 26054 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:11.502504 26162 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.543927 26162 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:11.525287 26236 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:11.545737 26236 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:11.546214 26236 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:11.547320 26236 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.548882 26236 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:11.550401 21113 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.845s (SENT) [suppressed 22 similar messages]
I20260709 13:08:11.464285 26086 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.446313 25441 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.557009 26295 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:11.557708 26295 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:11.558105 26295 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:11.532810 26238 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:11.508268 26059 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.534466 25762 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:11.559609 25762 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:11.560374 25762 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:11.561051 26167 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:11.561870 26167 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.561640 25762 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.562420 26167 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.566182 26242 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:11.566706 26242 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:11.546499 26241 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:11.567646 26242 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.559674 25898 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.569638 25898 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:10.911116 21221 rpcz_store.cc:276] 0709 13:08:09.866515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.544520 (+678005us) service_pool.cc:224] Handling call
0709 13:08:10.544854 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.545111 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:10.566886 (+ 21775us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.909492 (+342606us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.909848 (+   356us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:10.909908 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:10.909978 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.910062 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.910217 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7765,"prepare.queue_time_us":21156,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":7909,"thread_start_us":334014,"threads_started":1}
I20260709 13:08:11.540844 26074 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:11.572369 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:11.574205 21146 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:11.577515 26155 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:11.578054 26155 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:11.497531 26200 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.581466 26200 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:11.496814 26099 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.583647 26099 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.586297 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:11.587046 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:11.456030 26196 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:08:11.531543 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:11.593806 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.624356 21146 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:08:11.586989 21795 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:11.530359 21794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:08:11.499524 26191 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.662106 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:11.653661 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.684742 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:11.526862 26228 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.691082 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:11.691972 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:11.651381 26191 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:11.477650 26278 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.559003 26295 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.726326 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:11.697921 26265 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:11.701560 26089 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.579171 26155 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.733345 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:11.743642 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2622) took 3966 ms. Trace:
I20260709 13:08:11.716550 26242 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:11.712968 25762 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:11.744285 21138 rpcz_store.cc:276] 0709 13:08:07.777424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.227423 (+449999us) service_pool.cc:224] Handling call
0709 13:08:08.227717 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.389275 (+3161558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.743314 (+354039us) spinlock_profiling.cc:243] Waited 3.16 s on lock 0x7b54002e3e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:11.743565 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7268829568}
I20260709 13:08:11.752924 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:11.753537 21138 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:11.755939 25898 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:11.756074 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.750999 26265 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:11.756356 26167 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:11.757956 26112 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:11.758457 26112 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:11.758634 26324 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:08:11.758843 26112 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:11.731885 26059 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:11.760776 26086 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:11.761687 26328 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:11.761884 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 664) took 2264 ms. Trace:
W20260709 13:08:11.768419 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.768985 21068 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:11.762362 21141 rpcz_store.cc:276] 0709 13:08:09.497935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.982002 (+1484067us) service_pool.cc:224] Handling call
0709 13:08:10.982343 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:10.982673 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:10.982700 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:10.983176 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.453102 (+469926us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:11.453156 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.453252 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.761640 (+308388us) raft_consensus.cc:1685] finished
0709 13:08:11.761667 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.761820 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8207,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":415281,"threads_started":1,"wal-append.queue_time_us":699256}
W20260709 13:08:11.776762 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 226375us
I20260709 13:08:11.762419 26328 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:11.750228 25889 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:11.792861 21068 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:11.748152 26089 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:11.805397 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 288958us
I20260709 13:08:11.805996 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:11.765583 26099 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:11.812263 21138 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:11.819427 26295 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:11.784751 26167 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.826370 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:11.850800 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:11.807824 26199 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.631s	user 0.014s	sys 0.000s
I20260709 13:08:11.887701 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:11.889650 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:11.917971 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.918322 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:11.919080 21221 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:11.918015 26330 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:11.912364 26263 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.024s	sys 0.105s
I20260709 13:08:11.839000 21801 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:11.928555 21220 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
W20260709 13:08:11.930934 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 197685us
I20260709 13:08:11.932670 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:11.915910 26196 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:11.938541 21214 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:11.933034 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 2646) took 1180 ms. Trace:
I20260709 13:08:11.881265 26289 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:11.954478 21145 rpcz_store.cc:276] 0709 13:08:10.752141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.787732 (+ 35591us) service_pool.cc:224] Handling call
0709 13:08:11.932836 (+1145104us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540036c754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:11.932978 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411060480}
I20260709 13:08:11.793889 26328 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:11.935657 26263 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.025s	sys 0.105s
I20260709 13:08:11.960330 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:11.933406 26265 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:11.965736 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.936177 26334 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:11.884163 26167 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.918783 26199 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:11.986238 26342 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.921571 26200 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.032447 21142 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:11.938915 26330 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:11.929682 26291 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.667s	user 0.004s	sys 0.017s
I20260709 13:08:11.975665 26335 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:12.042078 26229 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.652s	user 0.002s	sys 0.046s
I20260709 13:08:11.940021 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:11.970988 26326 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
W20260709 13:08:12.048686 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 108771us
I20260709 13:08:12.050254 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.051863 26283 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:12.052393 26283 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:12.052811 26283 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:12.040818 26094 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.861s	user 0.016s	sys 0.011s
W20260709 13:08:12.044981 26332 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:08:12.059751 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:12.038280 26228 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.072767 26329 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:12.075827 26089 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.077235 26341 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:12.135303 26329 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.135711 26341 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:12.136756 26341 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.137622 26329 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.078651 26277 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:12.086683 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2384) took 1080 ms. Trace:
I20260709 13:08:12.086934 26229 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:12.088064 21067 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:12.086763 26340 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:08:12.165231 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:12.165872 21215 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:12.163007 21065 rpcz_store.cc:276] 0709 13:08:11.006023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.131668 (+125645us) service_pool.cc:224] Handling call
0709 13:08:12.086414 (+954746us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b540026eb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:12.086611 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485918464}
I20260709 13:08:12.093871 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1606) took 2227 ms. Trace:
I20260709 13:08:12.094616 26306 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:12.168926 21212 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:12.096272 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:12.170167 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 73975.3us
I20260709 13:08:12.172214 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:12.097277 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:12.173363 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 76167.1us
I20260709 13:08:12.175617 21220 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:12.097378 26243 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:12.099860 26070 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.103621 26226 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:12.104694 21801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1048) took 1686 ms (client timeout 2079 ms). Trace:
I20260709 13:08:11.993697 26334 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.045439 26335 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:12.110854 26339 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 13:08:12.111263 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 386301us
I20260709 13:08:12.111467 25898 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:12.115568 26330 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.115361 26094 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:12.115952 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.120245 26241 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.552s	user 0.012s	sys 0.014s
I20260709 13:08:12.126464 25395 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:12.191591 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.128345 26199 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.128628 26074 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:12.129570 26328 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.130975 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.162221 26283 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:12.131882 26162 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.587s	user 0.014s	sys 0.024s
I20260709 13:08:12.133128 26167 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.179241 26243 consensus_queue.cc:1243] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:08:12.193599 26251 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.811s	user 0.006s	sys 0.012s
I20260709 13:08:12.042703 26291 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.134503 26083 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:12.060328 26196 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:12.041177 26112 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.216787 26112 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.073113 26265 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:12.152194 21214 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:12.218788 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:12.141917 26282 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:12.159457 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.219640 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.220085 26282 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:12.219928 26032 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:12.220395 21145 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
W20260709 13:08:12.220711 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.166960 21213 rpcz_store.cc:276] 0709 13:08:09.866209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.502498 (+636289us) service_pool.cc:224] Handling call
0709 13:08:10.502825 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.827737 (+324912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.093580 (+1265843us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b540013c754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:12.093797 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729132800}
I20260709 13:08:12.167395 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.221565 26032 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:12.221968 26032 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:12.222143 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:12.222829 21065 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:12.223574 25863 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.224118 25863 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:12.164814 26236 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.615s	user 0.017s	sys 0.026s
I20260709 13:08:12.226397 26196 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.227807 21792 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
W20260709 13:08:12.176898 21801 rpcz_store.cc:269] 0709 13:08:10.418425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.418646 (+   221us) service_pool.cc:224] Handling call
0709 13:08:12.104487 (+1685841us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b54004b7dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:12.104621 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2253958016}
I20260709 13:08:12.184451 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2639) took 2324 ms. Trace:
I20260709 13:08:12.229876 21139 rpcz_store.cc:276] 0709 13:08:09.860419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.198146 (+1337727us) service_pool.cc:224] Handling call
0709 13:08:11.198524 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.388310 (+189786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.184213 (+795903us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54004236d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:12.184392 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434797952}
I20260709 13:08:12.184803 21212 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:12.232198 26334 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:12.234809 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.186877 21215 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:12.186918 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:12.236002 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.188956 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:12.190253 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1561) took 2482 ms. Trace:
I20260709 13:08:12.193068 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:12.237715 21069 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:12.237495 21067 rpcz_store.cc:276] 0709 13:08:09.708198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.217475 (+1509277us) service_pool.cc:224] Handling call
0709 13:08:12.087873 (+870398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.163929 (+ 76056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.190051 (+ 26122us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b540026eb50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:12.190194 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2000965888}
I20260709 13:08:12.168372 26306 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:12.194212 26074 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:12.176518 26089 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:12.176965 25930 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.198135 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:12.028364 26342 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.201215 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:12.246508 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2378) took 1287 ms. Trace:
I20260709 13:08:12.246727 21216 raft_consensus.cc:2410] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:08:12.133733 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.164422 26191 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.177939 26228 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
W20260709 13:08:12.206046 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 77184.2us
I20260709 13:08:12.207203 21147 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:12.189600 25762 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:12.209060 21143 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:12.210264 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.210852 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:12.211634 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:12.210753 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:12.213554 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 1.0628e+06us
I20260709 13:08:12.214063 25495 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:08:12.214519 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 991846us
I20260709 13:08:12.229115 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.194912 21144 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:12.194417 26070 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:08:12.265446 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 54756.7us
W20260709 13:08:12.265829 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.266602 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.266548 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:12.264657 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.273809 21067 raft_consensus.cc:2410] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:08:12.275352 21066 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:12.212967 26241 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:12.244750 21185 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:12.196503 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 110274us
I20260709 13:08:12.248138 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:12.258028 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.212780 26083 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:12.259919 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.289656 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:12.264305 21065 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:12.212838 26346 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.280791 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:12.296057 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:12.297052 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.297708 21139 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:12.295343 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:12.258597 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:12.196638 25395 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.231771 26283 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.289716 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:12.291648 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.304693 21217 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:12.306059 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.307828 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.309293 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.292529 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.310864 21147 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:12.292742 21212 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:12.293222 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.293440 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.313870 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.196470 26330 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:08:12.294212 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:12.315764 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.316509 21215 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 8.
I20260709 13:08:12.316987 21065 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:12.205138 26242 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:08:12.299428 21213 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:12.282927 21796 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:12.300352 21221 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:12.205803 26162 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.255184 26089 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:12.343677 21144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3058) took 2598 ms (client timeout 3296 ms). Trace:
I20260709 13:08:12.321738 21143 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:12.264573 26047 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:12.247936 26182 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:12.316006 26229 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:08:12.315768 26349 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:08:12.264309 26344 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:08:12.327337 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:12.327914 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48803.6us
I20260709 13:08:12.329802 26342 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.283663 26236 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:08:12.331419 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:12.332263 26347 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:08:12.333456 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.222788 26032 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.275142 25817 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.342200 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:12.301142 26265 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
W20260709 13:08:12.344096 21144 rpcz_store.cc:269] 0709 13:08:09.745235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.578127 (+832892us) service_pool.cc:224] Handling call
0709 13:08:12.343444 (+1765317us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540032c8d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:12.343607 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3496302336}
W20260709 13:08:12.344856 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 57778.3us
I20260709 13:08:12.345831 21141 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:12.346594 21140 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:12.275445 26261 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:12.440078 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:12.357021 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.445878 21065 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:12.446058 21138 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:12.357547 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.446720 21145 raft_consensus.cc:2368] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
W20260709 13:08:12.359375 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.281816 26226 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:12.317967 26346 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.284711 26289 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.284811 26200 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:08:12.369627 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 125004us
I20260709 13:08:12.371364 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.373133 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.295051 26217 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.302186 25441 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:12.375721 26047 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.246914 21071 rpcz_store.cc:276] 0709 13:08:10.959076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.542593 (+583517us) service_pool.cc:224] Handling call
0709 13:08:11.542933 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.543151 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:11.732839 (+189688us) log.cc:844] Serialized 14 byte log entry
0709 13:08:12.110272 (+377433us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:12.246161 (+135889us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:12.246218 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:12.246271 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.246346 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.246455 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":305582,"prepare.queue_time_us":263815,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":34822,"raft.queue_time_us":315022,"spinlock_wait_cycles":4987264,"thread_start_us":364927,"threads_started":2,"wal-append.queue_time_us":237102}
I20260709 13:08:12.317631 26232 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:12.389075 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.340401 26155 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:12.264317 25930 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.319478 25762 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:12.496924 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.406381 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.340858 26241 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:12.361689 26089 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:12.409191 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75807.9us
I20260709 13:08:12.409730 26263 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:12.409771 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:12.376004 26295 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:12.452044 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.453269 21143 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:12.439873 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.436645 25898 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:12.394552 26028 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:08:12.302737 26224 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.413250 26242 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:12.469902 21213 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:12.469184 21217 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:12.495618 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.396979 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.608645 21139 raft_consensus.cc:2368] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:08:12.608242 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:12.609668 21144 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:12.506978 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.494324 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.503516 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:12.612140 21140 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:12.612414 21065 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:12.456143 21068 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:12.456565 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2375) took 1588 ms. Trace:
I20260709 13:08:12.616014 21072 rpcz_store.cc:276] 0709 13:08:10.868315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.512496 (+644181us) service_pool.cc:224] Handling call
0709 13:08:11.515356 (+  2860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.242441 (+727085us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:12.242775 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 13:08:12.455320 (+212545us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:12.455379 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:12.455427 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.455794 (+   367us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.456510 (+   716us) inbound_call.cc:177] Queueing success response
0709 13:08:12.564266 (+107756us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b1800344e60. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
Metrics: {"mutex_wait_us":750068,"prepare.queue_time_us":113,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":730373,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":21792,"spinlock_wait_cycles":20367104,"thread_start_us":207413,"threads_started":1,"wal-append.queue_time_us":256384}
I20260709 13:08:12.603876 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2370) took 1833 ms. Trace:
W20260709 13:08:12.614759 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 111421us
I20260709 13:08:12.619606 21070 rpcz_store.cc:276] 0709 13:08:10.770608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.355891 (+585283us) service_pool.cc:224] Handling call
0709 13:08:11.371151 (+ 15260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:12.229470 (+858319us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:12.229501 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:12.229938 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:08:12.449300 (+219362us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:12.449364 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.449482 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:12.602915 (+153433us) raft_consensus.cc:1685] finished
0709 13:08:12.602995 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.603530 (+   535us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":857685,"raft.queue_time_us":858150,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":218916,"threads_started":1,"wal-append.queue_time_us":273094}
I20260709 13:08:12.621153 21111 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.510008 21142 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:12.623344 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.626351 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.629200 21072 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:12.633461 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.604027 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 844) took 2939 ms. Trace:
I20260709 13:08:12.614591 21140 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:08:12.641283 21063 rpcz_store.cc:276] 0709 13:08:09.664666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.193476 (+1528810us) service_pool.cc:224] Handling call
0709 13:08:11.193834 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:12.213519 (+1019685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:12.213562 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:12.214064 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:08:12.398962 (+184898us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:12.399002 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.399152 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:12.603501 (+204349us) raft_consensus.cc:1685] finished
0709 13:08:12.603542 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.603825 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1019338,"raft.queue_time_us":1025550,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":184533,"threads_started":1,"wal-append.queue_time_us":295218}
I20260709 13:08:12.646457 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1577) took 1382 ms. Trace:
I20260709 13:08:12.611039 21139 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:08:12.647056 21068 rpcz_store.cc:276] 0709 13:08:11.264121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.800991 (+536870us) service_pool.cc:224] Handling call
0709 13:08:12.455897 (+654906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.615193 (+159296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.645993 (+ 30800us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b54003bf450. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:12.646200 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505232768}
I20260709 13:08:12.648777 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.649739 21068 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
W20260709 13:08:12.600780 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 246256us
I20260709 13:08:12.643800 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.614774 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.629511 21070 raft_consensus.cc:2410] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:08:12.638353 21141 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:08:12.335093 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:12.664747 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 329603us
I20260709 13:08:12.675879 21063 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:08:12.677201 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:12.683591 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.691641 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.692838 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:12.654973 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.695495 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.695998 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5504378690000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.5301s for a response
I20260709 13:08:12.492372 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.696141 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:12.699421 21071 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:12.699759 21070 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:12.700646 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.701992 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.703893 21071 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:08:12.507627 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55680.1us
I20260709 13:08:12.500485 21216 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:12.495869 21218 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 10.
I20260709 13:08:12.704193 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:12.706360 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:12.708263 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:12.708804 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.637648 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.709908 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:12.679112 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:12.712026 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:12.714372 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:12.717913 21072 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:12.719352 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:12.719489 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:12.503257 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 93559.8us
I20260709 13:08:12.524839 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:12.770848 26241 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:12.771164 21792 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:12.771457 26241 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:12.773231 26241 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.775179 26333 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:12.776437 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.725649 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:12.777726 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:12.778136 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52769.3us
W20260709 13:08:12.778473 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 141089us
I20260709 13:08:12.779897 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:12.726442 21793 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:12.781038 26228 raft_consensus.cc:2764] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:08:12.445255 25817 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.782073 21068 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:08:12.783304 21794 raft_consensus.cc:606] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.783605 21068 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:12.783789 21794 raft_consensus.cc:614] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:12.784484 25817 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.497328 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:12.786185 21796 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:12.534842 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:12.787492 21792 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:12.787621 21213 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:12.788010 21797 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:12.536275 25762 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:12.379012 26182 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.751811 26028 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:08:12.541872 25930 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.796614 26028 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:12.533756 26196 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:12.797142 26028 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:12.532945 26047 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.407965 26074 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.445601 26083 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.524511 25395 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.799081 26074 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:12.799482 26167 consensus_queue.cc:1175] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b70fc04de37e4239a8336534f0c9c0fb" responder_term: 17 status { last_received { term: 16 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:12.548887 26282 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.563696 26032 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:12.457930 26289 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.808452 21066 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:12.510093 26094 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.516248 26224 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.525049 26162 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.524065 26242 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:12.525346 26191 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.487699 26217 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.841707 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:12.842763 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 80658.5us
I20260709 13:08:12.843071 26217 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.531721 26251 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.844609 26241 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:12.844507 26251 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.846312 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:12.846475 26144 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:12.545838 26295 raft_consensus.cc:1064] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:08:12.847200 26144 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:12.848201 26295 raft_consensus.cc:1068] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:08:12.848382 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 136950us
I20260709 13:08:12.849180 26295 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:12.848923 26144 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.849690 26295 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.458389 26226 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:12.545405 26306 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.850893 26226 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:12.851883 26306 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:12.852021 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 132972us
I20260709 13:08:12.505950 26229 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:12.450344 26261 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.829823 26054 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:12.855867 26054 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.856029 21146 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:08:12.856324 26054 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:12.850168 26295 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:12.857059 26261 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.564605 26278 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:12.858604 26278 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:12.859628 26356 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.353s	user 0.001s	sys 0.003s
I20260709 13:08:12.860337 26278 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.861986 26278 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:12.862484 21219 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:08:12.524477 26291 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.863831 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:12.728562 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:12.865166 21066 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:12.457767 26340 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:08:12.541038 26334 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:12.564850 26328 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:12.494663 26330 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:08:12.503225 26335 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.533623 26346 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.731703 21801 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:12.738225 25495 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:12.867019 26334 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:12.872133 25495 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:12.872813 26334 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:12.836829 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69397.1us
I20260709 13:08:12.874591 26334 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:12.875083 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.875000 26283 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.867223 26328 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:12.875885 26283 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.876951 26328 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.750422 25898 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:12.751003 26210 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:12.878352 26328 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:12.878804 26210 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.879186 26283 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.881011 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:12.881430 26210 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.881669 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2657) took 1403 ms. Trace:
I20260709 13:08:12.882090 21801 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:12.882164 21146 rpcz_store.cc:276] 0709 13:08:11.478342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.669642 (+191300us) service_pool.cc:224] Handling call
0709 13:08:11.688857 (+ 19215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.856540 (+1167683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.881324 (+ 24784us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540030b454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:12.881589 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2682535296}
I20260709 13:08:12.869001 26330 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:12.883891 26330 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:12.883800 26242 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.885263 26242 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:12.744772 26026 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.887712 26026 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.889009 26083 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:12.890481 26026 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.762970 21144 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:12.562237 25863 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.768469 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1583) took 1502 ms. Trace:
I20260709 13:08:12.866743 26251 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.891974 21072 rpcz_store.cc:276] 0709 13:08:11.265932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.717188 (+1451256us) service_pool.cc:224] Handling call
0709 13:08:12.717651 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.768160 (+ 50509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.768400 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:12.892486 26289 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:12.850073 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:12.775174 21215 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:12.895175 26282 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.786958 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33604 (request call id 9) took 1259 ms. Trace:
W20260709 13:08:12.895722 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45878.2us
I20260709 13:08:12.895988 21794 rpcz_store.cc:276] 0709 13:08:11.527937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.530171 (+  2234us) service_pool.cc:224] Handling call
0709 13:08:12.786741 (+1256570us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002eaed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42516853 00007f7f474e460a 00007f7f40a6a7d4 00007f7f40a6a762 00007f7f40a6a691 00007f7f40a6a60c 00007f7f40a698c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:12.786863 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2656065792}
I20260709 13:08:12.796293 26182 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.787775 26243 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:12.533516 26089 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.897516 26243 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:12.897966 26243 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:12.857750 26261 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:08:12.899380 26089 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:12.900048 26261 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:12.900449 26261 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:12.804651 21067 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:12.798554 26028 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.903853 26028 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:12.906457 21143 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
W20260709 13:08:12.829936 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 61508.9us
I20260709 13:08:12.908352 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:12.895929 26291 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:12.811949 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:12.910697 26291 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:08:12.911136 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 131863us
I20260709 13:08:12.850308 21142 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:08:12.911468 26291 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:12.863067 26224 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.741570 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1582) took 1470 ms. Trace:
I20260709 13:08:12.913272 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:12.749768 26232 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:08:12.913506 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.732462 26059 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.914227 21066 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:12.914549 26059 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.912883 21071 rpcz_store.cc:276] 0709 13:08:11.265658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.702913 (+1437255us) service_pool.cc:224] Handling call
0709 13:08:12.703374 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.731441 (+ 28067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.735711 (+  4270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:12.915608 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.915735 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:12.916136 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:12.797914 21220 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:12.917255 26059 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:12.918196 26291 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:12.884594 26330 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:12.911537 26333 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:08:12.743173 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.927953 26295 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.873425 25495 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:12.934165 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.913036 26224 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:12.896616 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:12.916270 21066 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:12.910975 26306 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:12.939000 21070 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:12.901511 26261 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:12.941681 26261 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:12.943575 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:12.943737 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:12.945785 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:12.949266 21797 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:08:12.948691 21064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.009s	sys 0.013s
I20260709 13:08:12.969612 21220 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:12.982693 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:12.983592 21070 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:12.992224 21767 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:12.992132 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 137111us
W20260709 13:08:12.994268 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.994583 21064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.010s	sys 0.014s
I20260709 13:08:13.003765 26226 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.006593 26226 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:13.007439 21069 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.601s	user 0.004s	sys 0.018s
I20260709 13:08:13.008139 21069 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:13.009976 21069 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:12.998881 26054 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.019991 26229 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:13.020673 26229 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.021059 26229 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:13.022128 21144 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: [10.9-12.11]
I20260709 13:08:13.023833 26291 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:13.024775 26291 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:13.027982 21066 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:13.030153 21066 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:13.034299 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:13.034468 26028 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:13.035192 26089 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:13.036597 26026 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:13.037065 26089 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:13.037729 26089 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.038240 26089 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:13.002125 26243 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.039522 25898 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:13.040020 25898 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:13.041448 25898 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.043566 26306 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:13.044978 26306 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.038949 26026 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:13.043018 25898 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:13.051947 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 155404us
I20260709 13:08:13.055414 21070 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:13.072340 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:13.073443 21142 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:13.078402 26226 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:13.086866 26283 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:13.087984 26283 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:13.088426 26283 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:13.088835 26155 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:08:13.089519 26283 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.090862 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:13.091816 21797 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:13.093390 21072 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:08:13.094009 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:13.096833 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:13.097802 21797 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:13.091001 26283 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:13.107757 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:13.107993 21794 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:13.112556 21794 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:08:13.115772 21797 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:13.116142 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:13.121810 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:13.123977 26265 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:13.125727 26265 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:13.129786 26226 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:13.130577 26226 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.021111 26054 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:13.138561 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:13.136924 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:13.136430 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2377) took 2200 ms. Trace:
W20260709 13:08:13.135936 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 69038.3us
I20260709 13:08:13.144152 21065 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:13.149326 21144 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:13.057359 26026 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:13.151938 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:13.026376 26291 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.143671 21064 rpcz_store.cc:276] 0709 13:08:10.936407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.513482 (+577075us) service_pool.cc:224] Handling call
0709 13:08:11.513956 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:12.233621 (+719665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:12.233661 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:12.234085 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:08:13.048600 (+814515us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:13.056850 (+  8250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.056929 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.056985 (+    56us) raft_consensus.cc:1685] finished
0709 13:08:13.057000 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.136366 (+ 79366us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":723097,"raft.queue_time_us":728156,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":19267,"thread_start_us":760583,"threads_started":1,"wal-append.queue_time_us":514761}
W20260709 13:08:13.160727 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 93828.6us
I20260709 13:08:13.171470 21070 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:13.175050 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:13.192471 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.142467 26232 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 13:08:13.198371 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.203510 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:13.213356 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.173880 26054 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:13.216516 26099 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:13.217911 26226 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:13.219688 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 58280.9us
I20260709 13:08:13.221001 26283 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:13.221165 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:13.221705 26283 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:13.224025 26099 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:13.223150 26283 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.227224 26283 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:13.215682 26199 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:13.216446 26054 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.193427 26333 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:08:13.242287 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.230180 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:13.257313 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:13.260931 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.209698 26243 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:13.260159 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64434.1us
I20260709 13:08:13.253996 26261 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:13.259807 26295 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:13.186219 26083 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:08:13.288228 26032 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.319305 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:13.320988 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 63844.2us
W20260709 13:08:13.323587 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 115562us
W20260709 13:08:13.324474 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 137690us
I20260709 13:08:13.329849 20969 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 11, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 11 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:13.338172 26054 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:13.339222 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:13.275506 26333 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:13.242049 26330 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.341362 26059 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:13.286972 26261 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:13.357486 21065 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
W20260709 13:08:13.354040 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.275928 26026 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:13.372027 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.372519 21064 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:08:13.372162 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:13.359205 26241 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.008s
I20260709 13:08:13.385952 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 853) took 1082 ms. Trace:
I20260709 13:08:13.388767 21067 rpcz_store.cc:276] 0709 13:08:12.285767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.946881 (+661114us) service_pool.cc:224] Handling call
0709 13:08:12.947226 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.999389 (+ 52163us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:13.139807 (+140418us) log.cc:844] Serialized 14 byte log entry
0709 13:08:13.289539 (+149732us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:13.289597 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:13.368230 (+ 78633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.368326 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.368499 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":120194,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":1486,"raft.queue_time_us":5233,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":43508,"thread_start_us":120122,"threads_started":1}
I20260709 13:08:13.391495 26155 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:13.391800 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 212794us
I20260709 13:08:13.384528 26226 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:13.392251 26155 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.392840 26155 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:13.393643 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:13.402763 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:13.372804 26261 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:13.394913 21185 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:13.366324 26059 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:13.415127 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:13.415599 21067 pending_rounds.cc:85] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 8
W20260709 13:08:13.423584 26372 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:08:13.403661 26224 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.427841 26224 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:13.429759 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 90649us
I20260709 13:08:13.430265 26224 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:13.436434 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.431309 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:13.415975 21067 pending_rounds.cc:107] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:08:13.368747 26032 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.448153 21067 raft_consensus.cc:2889] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:13.451844 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:13.427923 25878 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:13.454202 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.402226 26243 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:08:13.442006 26334 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.456560 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:13.465488 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 69817.2us
I20260709 13:08:13.452314 26210 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:13.472844 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:13.476151 26340 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.462098 26306 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:13.481527 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 96619.8us
W20260709 13:08:13.487752 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 85896.2us
I20260709 13:08:13.502061 26335 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:13.479274 26083 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:08:13.509011 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45333.3us
I20260709 13:08:13.504195 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1585) took 1236 ms. Trace:
W20260709 13:08:13.494721 26350 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:08:13.514896 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:13.519909 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 71393.6us
I20260709 13:08:13.501184 26291 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:13.507858 25817 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:13.514686 26278 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.003s
I20260709 13:08:13.512301 26083 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.482336 26340 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.517412 25898 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:13.484189 25878 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.503067 26334 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.488505 26059 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.540004 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:13.533279 26083 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:13.533515 26330 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:13.547079 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:13.512650 21068 rpcz_store.cc:276] 0709 13:08:12.267597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.784448 (+516851us) service_pool.cc:224] Handling call
0709 13:08:12.784787 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.785215 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:12.811894 (+ 26679us) log.cc:844] Serialized 14 byte log entry
0709 13:08:13.499614 (+687720us) log.cc:844] Serialized 14 byte log entry
0709 13:08:13.503779 (+  4165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:13.503842 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:13.503897 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.503987 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.504129 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1941,"prepare.queue_time_us":155560,"prepare.run_cpu_time_us":156,"prepare.run_wall_time_us":5210,"raft.queue_time_us":2989,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":5560,"spinlock_wait_cycles":6409984,"thread_start_us":519353,"threads_started":2,"wal-append.queue_time_us":405664}
I20260709 13:08:13.495980 26333 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.550673 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:13.519672 26229 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:13.561038 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51901.5us
I20260709 13:08:13.537876 26074 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:08:13.568176 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:13.571810 26375 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:08:13.572856 26241 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:13.577018 26083 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.581416 26334 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:13.587263 21792 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:13.552166 25898 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:13.601048 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:13.583724 26032 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:13.618399 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:13.601105 26054 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:13.599639 26224 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:13.621562 26054 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:13.628232 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 81036.6us
I20260709 13:08:13.629487 26361 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:13.629815 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:13.631506 26241 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:13.631374 26278 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:13.608737 26330 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:08:13.628558 26054 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:13.637040 26229 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:13.644040 25878 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:13.643702 26261 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.645257 26340 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:13.645912 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 77839.6us
I20260709 13:08:13.666229 21111 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:13.647687 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:13.668223 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51860.6us
I20260709 13:08:13.668524 25495 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:13.669263 26278 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:13.669776 26278 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.670177 26278 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:13.670343 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:13.647624 26361 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:13.652246 26089 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.005s
I20260709 13:08:13.671862 26089 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.672320 26361 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.674162 26089 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:13.656008 25898 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.657277 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:13.652765 26242 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.003s
W20260709 13:08:13.685935 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 96660.1us
W20260709 13:08:13.691802 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.693130 25817 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:13.693713 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:13.695187 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:13.696615 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:13.697257 21065 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:13.702498 21070 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:13.703013 26167 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:08:13.703855 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46672.2us
W20260709 13:08:13.703976 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.711357 26229 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:13.712390 26283 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:13.712831 26283 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:13.691378 26334 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:13.690994 26306 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:13.721693 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:13.729176 21217 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:13.732786 25817 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:13.712390 26167 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:13.733562 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:13.735407 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65207.8us
I20260709 13:08:13.733562 25817 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.741328 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:13.753530 26330 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:13.753809 25817 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:13.754209 26330 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.754987 21220 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:13.756395 26330 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:13.746503 26340 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:13.736876 26241 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.745790 26278 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:13.759274 26386 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.213s	user 0.000s	sys 0.002s
I20260709 13:08:13.737223 26083 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:13.752434 26306 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:13.775393 21220 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:13.777573 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:13.783567 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:13.786242 25762 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.786965 25762 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.789669 26291 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.790474 26291 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.792972 26291 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:13.789317 25762 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:13.795979 21072 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:08:13.797940 26390 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.067s	user 0.003s	sys 0.000s
W20260709 13:08:13.790405 26391 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.138s	user 0.000s	sys 0.002s
I20260709 13:08:13.805249 26026 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:13.806891 26334 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:13.811394 26330 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.765640 26232 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:08:13.820874 21065 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
W20260709 13:08:13.814240 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49579.5us
I20260709 13:08:13.814792 26224 raft_consensus.cc:2764] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 13:08:13.815970 26384 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.165s	user 0.000s	sys 0.002s
I20260709 13:08:13.815835 26167 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.775713 26241 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.822333 20969 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 9 to 12, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 12 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:08:13.817188 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45184.2us
I20260709 13:08:13.828737 26241 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:13.828019 20969 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 11 to 13, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 13 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:13.766551 26054 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.834055 21139 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:08:13.830451 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:13.772234 26089 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:08:13.832312 20969 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 9 to 12, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 12 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:13.839668 26283 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.842371 26283 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:13.843605 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:13.844202 26089 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:13.844643 26089 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:13.761540 26155 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.849677 26155 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:13.853255 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:13.822458 26306 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:13.766717 26340 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.882339 21220 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:08:13.884009 21220 pending_rounds.cc:85] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 6
I20260709 13:08:13.885066 20969 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:13.887529 21220 pending_rounds.cc:107] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:08:13.888046 21220 raft_consensus.cc:2889] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:13.888096 26251 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.852382 26059 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:13.887903 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 861) took 1218 ms. Trace:
I20260709 13:08:13.897301 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:13.898839 21036 leader_election.cc:400] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:08:13.885847 26243 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:13.902951 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:13.892140 21066 rpcz_store.cc:276] 0709 13:08:12.670037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.078324 (+408287us) service_pool.cc:224] Handling call
0709 13:08:13.079141 (+   817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:13.148645 (+ 69504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:13.148679 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:13.161946 (+ 13267us) log.cc:844] Serialized 26 byte log entry
0709 13:08:13.336690 (+174744us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b180040a260. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:13.443202 (+106512us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:13.443325 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.443524 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.758687 (+315163us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:13.887687 (+129000us) raft_consensus.cc:1685] finished
0709 13:08:13.887706 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.887844 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63366,"raft.queue_time_us":77609,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":1906,"spinlock_wait_cycles":19737600,"thread_start_us":106078,"threads_started":1,"wal-append.queue_time_us":359296}
I20260709 13:08:13.852433 26330 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.916813 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:13.921918 21217 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:13.845705 26089 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.926589 26054 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:13.926879 26388 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.277s	user 0.000s	sys 0.003s
I20260709 13:08:13.926463 20969 catalog_manager.cc:5697] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 10, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 10 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:13.904512 26278 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:13.935850 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56050.9us
I20260709 13:08:13.938522 21036 leader_election.cc:400] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12.
I20260709 13:08:13.939818 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:13.941255 26394 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:13.941761 26394 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:13.942184 26394 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:13.944711 25817 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:08:13.962677 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.896034 26251 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.968940 26340 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:13.969292 26242 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.969884 26242 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.973860 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:13.974565 21139 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:13.978557 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:13.976518 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:13.979461 21792 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:13.971751 26243 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.988983 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:13.990463 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:13.995177 26241 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.995375 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:13.995641 26361 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:13.995869 26241 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.996635 21142 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:13.996398 26361 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:13.998126 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:13.996405 21147 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:14.007323 21138 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:13.983536 25495 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:14.016017 26398 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
I20260709 13:08:14.011251 26054 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.016937 26054 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:13.973853 26210 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:08:14.020663 25817 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:14.021615 26330 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:13.990916 26089 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:13.992224 26333 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.035342 21139 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:14.036253 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:14.037613 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.044062 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:14.044063 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.045430 21141 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:13.971967 26306 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.047005 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.053141 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:14.051069 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.058811 21212 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:14.059885 21213 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:14.034435 26054 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.035604 26334 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:14.000212 26243 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.068140 21221 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:14.070367 21141 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:08:14.071377 21147 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:14.071918 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:14.072571 21214 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:14.077297 26334 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:14.077994 26397 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.202s	user 0.002s	sys 0.001s
I20260709 13:08:14.089123 26155 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.089857 26155 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.091602 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:14.093490 26155 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:14.091848 26402 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:08:14.079530 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:14.100109 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:14.074020 26241 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:14.106737 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 139515us
I20260709 13:08:14.111339 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.112238 21145 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:14.112998 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:14.113161 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.113684 21140 raft_consensus.cc:2376] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12. Re-sending same reply.
I20260709 13:08:14.113900 21145 raft_consensus.cc:2368] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:14.114722 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.115541 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.115926 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.116569 21219 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:14.116824 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.117610 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.118265 21219 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:14.118953 21140 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:14.117640 21147 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:14.119843 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.120934 21140 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:14.057626 26333 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.124663 21212 raft_consensus.cc:2410] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:14.125311 21220 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:14.125357 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:14.126255 21792 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:14.127245 21142 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:14.126080 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:14.129050 21801 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:14.129407 21218 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:08:14.095634 25495 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:14.154752 21212 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:08:14.160748 21219 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:14.161154 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.161810 21212 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:14.162635 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.164729 21185 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.165613 26330 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:14.166359 25495 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:14.167325 25495 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.168819 21185 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.169829 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.170540 21220 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:14.100169 25817 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:14.170606 25762 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:14.170873 26243 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.171370 25817 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:14.171733 26291 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:08:14.172180 26406 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:08:14.172186 26291 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:14.172477 25817 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.173446 26291 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.173810 26251 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.174901 26291 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:14.176867 21218 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:14.179368 26361 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:14.181138 26059 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:14.182101 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.182317 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:14.182756 21066 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:14.149786 21147 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:14.181787 26059 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.157501 26232 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.183715 26232 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.184180 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.129326 26089 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.184971 26059 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:14.151597 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.185650 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:14.186313 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.186556 21063 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:14.189342 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:14.190830 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.197654 21067 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
W20260709 13:08:14.198508 26401 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:08:14.198829 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.159457 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.200241 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1605) took 1146 ms. Trace:
I20260709 13:08:14.200901 21067 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:14.200829 21071 rpcz_store.cc:276] 0709 13:08:13.054181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.215017 (+160836us) service_pool.cc:224] Handling call
0709 13:08:13.215382 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:13.215857 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:13.802027 (+586170us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b180049a8c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f45f143c7 00007f7f425269a7 00007f7f425360bf
0709 13:08:13.802152 (+   125us) log.cc:844] Serialized 14 byte log entry
0709 13:08:13.905102 (+102950us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:13.928677 (+ 23575us) log.cc:844] Serialized 52 byte log entry
0709 13:08:13.928851 (+   174us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:13.928887 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.928951 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:14.199942 (+270991us) raft_consensus.cc:1685] finished
0709 13:08:14.200055 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:14.200185 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21831,"prepare.queue_time_us":142934,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":36,"raft.queue_time_us":14272,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":16792,"spinlock_wait_cycles":10856704,"thread_start_us":194618,"threads_started":2,"wal-append.queue_time_us":147251}
I20260709 13:08:14.202368 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.114792 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:14.182850 21072 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:08:14.184948 21147 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:14.191633 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.203886 21064 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:14.204281 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.204773 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.204953 21147 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:14.158842 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:14.192209 26400 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.322s	user 0.004s	sys 0.000s
I20260709 13:08:14.206609 21145 raft_consensus.cc:2368] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11.
I20260709 13:08:14.207465 26242 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.207551 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.207820 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.208580 21146 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:14.208729 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:14.209292 21145 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:14.209355 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.210168 21146 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:14.210588 21072 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:14.192720 26334 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.175005 26226 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:14.193892 21065 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:14.194113 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.220811 21138 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:14.221148 21217 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:14.194424 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.196631 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:14.222477 21144 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:14.205883 21140 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:14.205838 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.211462 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.210954 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.159457 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.224820 21142 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:14.225996 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:14.158726 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:14.215133 21214 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:14.223948 21146 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:14.185467 26089 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.231786 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.157543 26335 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.236232 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.237728 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.238312 21144 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:14.241084 21142 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:14.246976 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:14.247682 21138 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:14.247538 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.249625 21143 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:08:14.252193 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:14.253211 21140 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:14.219592 26226 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:14.254312 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:14.258687 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.271560 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.271778 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:14.272414 21146 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:14.273411 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.224542 26333 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.275527 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.233290 26074 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.295601 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.295987 26026 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.233024 26251 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:08:14.259718 26243 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:14.297104 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:14.319123 21142 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:14.318051 21071 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:14.310509 21138 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
W20260709 13:08:14.312439 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 153772us
I20260709 13:08:14.246040 26335 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.331037 21144 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:14.243667 26182 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.338507 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.339454 21067 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
W20260709 13:08:14.336319 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 88810.5us
I20260709 13:08:14.332876 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:14.341843 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.276984 26242 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:08:14.354482 21219 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:08:14.303081 26333 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:08:14.363770 21071 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:08:14.357506 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:14.319078 26026 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.357195 26182 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.361539 26155 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:14.381328 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:14.403692 21067 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:14.315055 26074 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:14.407274 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65521.4us
I20260709 13:08:14.408138 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:14.346073 26408 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:08:14.402323 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:14.364696 26199 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.414436 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:14.416472 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 59048.3us
I20260709 13:08:14.377904 21144 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:08:14.408993 26243 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:14.389278 26083 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:14.423010 21145 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:14.423220 26199 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.425508 26199 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.427850 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:14.433274 21144 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
W20260709 13:08:14.435250 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.430994 21071 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:14.446520 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:14.336498 26226 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.446794 21185 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:14.403216 26155 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:14.425592 26232 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:14.457043 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:14.444630 26378 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:08:14.470582 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.474066 21145 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [10.6-11.7]
I20260709 13:08:14.449316 26054 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:14.445176 26330 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:14.454836 26333 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:14.486110 21071 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:08:14.484226 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:14.480544 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:14.450079 26089 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.460783 26155 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:14.489768 21067 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:14.496224 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:14.489815 26340 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:14.468322 26083 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:14.482477 26226 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:14.492916 26420 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:08:14.493547 26412 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:08:14.456027 26199 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:14.482758 26232 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:08:14.494390 26414 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:08:14.461968 26306 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:08:14.484221 26054 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.486174 26242 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:08:14.496551 26394 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.008s
I20260709 13:08:14.514326 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:14.490653 26333 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:14.504308 26361 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:14.509142 26232 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.492942 26421 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:14.531311 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.539750 21145 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
W20260709 13:08:14.529757 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 83031.2us
I20260709 13:08:14.524993 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:14.509232 26420 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.509585 26334 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:14.500676 26074 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:14.552089 26074 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:14.554608 26074 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:14.509522 26199 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.556509 26199 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:14.557034 26335 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.527597 26291 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:14.558941 26291 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:14.543622 26182 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.562321 26291 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.563982 26421 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.527385 26394 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.500555 26261 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:14.544350 26413 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:08:14.544931 26083 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.583745 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.540234 26232 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:14.551455 26420 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:14.503571 26243 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:14.527591 26333 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:14.555017 26074 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:14.566473 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 109558us
I20260709 13:08:14.523566 26054 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:14.521458 26242 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.527827 26340 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.627372 26182 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:14.624418 21185 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.559723 26334 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:14.537647 26361 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.632421 26426 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 13:08:14.622879 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 97937us
I20260709 13:08:14.630524 26340 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.632278 26334 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:14.632764 26361 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:14.631655 21064 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:14.631420 26242 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:14.636535 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.640868 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.627440 26074 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:14.644459 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.644872 21797 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:08:14.650264 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:14.633569 26026 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.625465 26335 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:08:14.655777 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.653085 21801 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:14.633726 26243 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:14.652191 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:14.653592 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:14.653781 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.655509 21215 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:14.658946 21797 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:14.656474 21216 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
W20260709 13:08:14.659978 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.660431 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.647557 26429 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:14.646675 26182 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 13:08:14.675429 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.675196 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.657127 25762 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:14.681728 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.682469 21216 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:14.660529 26421 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:14.686329 26243 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:14.687903 26395 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.816s	user 0.001s	sys 0.005s
W20260709 13:08:14.692857 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56419.7us
I20260709 13:08:14.699452 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.700285 21147 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:14.679898 25762 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.707762 25762 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:14.716691 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.657788 26436 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:08:14.732772 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.733464 21138 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:14.729585 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.734545 21147 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:08:14.734261 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.735334 21138 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:08:14.738011 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.738314 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.738701 21145 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:08:14.739589 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.740191 21144 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
W20260709 13:08:14.740697 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 475171us
I20260709 13:08:14.744237 26394 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.744447 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:14.747859 21138 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:08:14.749182 21795 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:14.731196 26429 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:14.751813 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:14.754458 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.756388 26429 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.759958 26167 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.760524 26167 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.752513 21794 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:08:14.771727 21147 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:14.770574 21141 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:08:14.773944 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.786011 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.787777 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.780028 26226 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:14.788537 26306 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:14.789196 26226 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:14.790024 26226 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:14.792537 26089 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:14.794351 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.808558 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:14.810006 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:14.795118 26340 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:14.816841 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 146791us
I20260709 13:08:14.817736 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.821817 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:14.823064 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:14.821240 26226 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:14.825219 26441 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:14.825459 26436 raft_consensus.cc:1064] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
W20260709 13:08:14.826509 26439 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:08:14.827582 26432 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:14.828068 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.801934 26199 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.816567 26026 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:14.794392 26361 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.803121 26054 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.830984 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:14.806771 26155 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.847259 21140 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:08:14.844810 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:14.816756 26089 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:08:14.829030 26306 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:14.829927 26340 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:14.854789 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:14.889349 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44613.7us
I20260709 13:08:14.890200 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:14.890345 21111 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:14.836915 26026 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:14.865042 26199 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.821961 26242 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.837393 26334 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.866621 26155 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.867185 26450 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:14.901614 26435 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.901379 26155 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.902333 26435 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.885119 26167 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.835716 26441 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:14.906497 26074 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.913380 26334 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:14.897838 26026 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:14.916505 26074 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.885169 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.918144 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.919531 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.919574 26232 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.920297 21213 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:14.920665 26054 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.921895 26232 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.894225 26251 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:14.922408 26251 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.922680 26242 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.923403 21212 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:14.923842 26306 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:14.924044 26441 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:14.924407 26306 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:14.894805 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:14.925551 21138 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:14.901746 26450 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.892172 26340 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.907871 26435 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.849406 26432 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:08:14.897786 26089 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:14.921635 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.922775 26251 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:14.902624 26155 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:14.874320 21217 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
W20260709 13:08:14.932236 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 173119us
I20260709 13:08:14.935211 21213 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 15: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:14.939155 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:14.942700 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52579.9us
I20260709 13:08:14.944813 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:14.931598 26340 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:14.931325 26450 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:14.947978 25762 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.949004 26452 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:14.958416 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.950356 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.960177 21217 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:14.962788 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:14.963835 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:14.950770 26427 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.393s	user 0.005s	sys 0.000s
I20260709 13:08:14.963853 21220 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:08:14.967665 26226 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.967875 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 683) took 1194 ms. Trace:
I20260709 13:08:14.969386 26155 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:14.971530 26435 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:14.993641 21143 rpcz_store.cc:276] 0709 13:08:13.773920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.774084 (+   164us) service_pool.cc:224] Handling call
0709 13:08:13.799700 (+ 25616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:14.250123 (+450423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.967595 (+717472us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b540027a454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:14.967813 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033953024}
I20260709 13:08:14.971668 26059 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.973297 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:14.975601 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.977174 21138 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:14.984104 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 93855.8us
I20260709 13:08:14.979578 21184 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:14.992619 21147 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:15.007396 21217 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:14.972338 26420 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.995648 26226 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:14.952193 26361 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:15.011013 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:14.973058 26032 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:14.999629 21794 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:08:15.031387 21792 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:15.031718 21794 pending_rounds.cc:85] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 8
I20260709 13:08:15.031967 21794 pending_rounds.cc:107] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:08:15.032214 21794 raft_consensus.cc:2889] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:15.010257 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:15.010977 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:15.012758 21213 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
W20260709 13:08:15.015420 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 66783.1us
I20260709 13:08:15.033737 21220 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:14.953539 25762 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:14.951361 26333 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.005352 26435 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.938294 26306 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:15.030452 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.037032 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:15.043557 26232 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:15.018056 26089 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.044858 26226 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:15.045110 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.045514 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.045765 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:15.046474 26454 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:15.046861 21792 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:08:15.005589 26026 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.048676 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.089978 21071 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:15.045801 26210 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:15.034683 26241 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.007s	sys 0.012s
I20260709 13:08:15.049126 25495 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.051635 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:15.043634 21064 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:08:15.020151 21141 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:15.053439 21138 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
W20260709 13:08:15.028455 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.044584 26423 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.021550 26083 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:15.073238 26394 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.121594 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.122252 21067 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:15.018484 26428 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:08:15.073627 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:15.123391 21145 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:15.080298 21147 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:15.034868 26054 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.053311 26032 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.089128 21801 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:15.013350 26059 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.059118 26435 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:15.074431 26261 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:15.098150 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:15.072487 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:15.106509 21141 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:08:15.110162 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:15.110711 26456 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:08:15.110880 26083 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.090901 25817 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:08:15.176723 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:15.111476 26423 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:15.177420 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:15.178020 21142 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:15.092110 26089 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:15.094034 26241 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.007s	sys 0.012s
I20260709 13:08:15.116794 26251 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:15.119524 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 146329us
I20260709 13:08:15.103040 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:15.047971 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.183923 21063 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:08:15.130896 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.184531 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:14.999205 26436 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.138332 26420 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:15.086100 26155 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:08:15.141151 26333 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:15.188005 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 110903us
I20260709 13:08:15.052032 26452 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:08:15.143311 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.072250 26432 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:08:15.145179 26026 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:15.148203 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:15.153126 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 68290us
I20260709 13:08:15.172823 21216 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:15.147295 26032 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:15.185091 21071 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:15.097406 26210 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.173521 26261 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:15.189532 21065 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:15.204704 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:15.205374 21213 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:15.120718 25495 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.175371 26083 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:15.136124 26394 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.192584 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.210662 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.211243 21063 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:15.212105 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.212615 21063 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:08:15.197059 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:15.201314 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.159152 26054 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.187002 26428 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:15.190028 21212 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:08:15.300537 21072 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:15.238528 21142 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:08:15.315644 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 124066us
I20260709 13:08:15.315996 21144 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:15.318735 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.305183 26155 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.305840 26232 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.248229 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:15.208748 26261 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:15.325434 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:15.264398 26428 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:15.328084 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.406031 21064 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:08:15.334108 21213 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:15.334108 21147 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:15.340147 26429 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.341845 26210 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:15.182597 26454 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:15.318336 26032 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:15.362049 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:15.364156 26467 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:15.366086 26054 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:15.371817 26394 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:15.372836 26463 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:08:15.376211 26423 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:15.377310 26466 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:08:15.391090 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:15.411131 26432 raft_consensus.cc:2764] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:08:15.400068 21141 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
W20260709 13:08:15.408695 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83315.6us
I20260709 13:08:15.454185 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:15.351461 26340 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.318001 25495 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:15.437018 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75062.3us
W20260709 13:08:15.450536 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 110422us
I20260709 13:08:15.398064 21063 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:15.468345 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.469806 21063 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:15.435024 26210 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:15.472278 26059 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:15.414075 26232 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:15.476238 26306 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:15.477355 26232 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:08:15.472306 26210 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:15.478495 26210 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:15.480118 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.482092 26428 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.481980 20969 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.483573 26428 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:15.146502 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:15.485654 26454 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.390763 26155 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:15.493178 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 245084us
I20260709 13:08:15.494026 21767 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:15.432596 26467 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.495012 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:15.483814 20969 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 9 to 14, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 14 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:15.502080 21794 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
W20260709 13:08:15.504295 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.457496 26333 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:15.510375 21063 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:15.511776 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:15.431453 26429 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:15.502003 26450 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.006s
I20260709 13:08:15.504608 26467 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:15.508559 26463 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:15.504583 26472 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:15.506628 26251 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:15.517223 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.513135 26420 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:15.512950 26333 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.536880 26465 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:15.549363 26333 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:15.549436 26465 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:15.550550 26465 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.551944 26465 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:15.540472 26340 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:15.542992 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:15.554538 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:15.554843 26417 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.133s	user 0.002s	sys 0.001s
I20260709 13:08:15.555503 26432 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:15.555920 26432 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.556254 26432 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:15.513809 26469 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.515692 26054 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:15.545814 21063 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
W20260709 13:08:15.548002 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.557777 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:15.502610 26242 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:08:15.517982 26436 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.564050 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.571556 21068 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:08:15.559780 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:15.569681 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.574826 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.532658 26450 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.585183 26450 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:15.569345 26469 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.564185 26251 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:15.591768 26469 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:15.569170 26232 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:08:15.563715 25817 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:15.593402 26232 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:15.593840 26232 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:15.593868 25817 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.612351 21063 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:15.613734 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:15.613910 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.615420 21063 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:15.616917 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:15.635064 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.635813 21063 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:15.602593 25495 raft_consensus.cc:2764] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:08:15.577416 26054 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:15.602290 26306 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:08:15.655968 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 110132us
I20260709 13:08:15.594784 26232 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.658987 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2413) took 1470 ms. Trace:
I20260709 13:08:15.607472 26428 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:15.631568 26450 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:15.630327 26226 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:15.625612 26459 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:15.667088 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:15.668484 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:15.576089 26436 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.670492 26420 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:15.675539 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:15.675879 26232 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:15.676353 26420 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.676863 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:15.677716 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:15.678334 21221 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
W20260709 13:08:15.678825 21221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 669) took 1602 ms (client timeout 1981 ms). Trace:
I20260709 13:08:15.679035 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:15.679190 21221 rpcz_store.cc:269] 0709 13:08:14.076311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.677468 (+1601157us) service_pool.cc:224] Handling call
0709 13:08:15.678775 (+  1307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:15.679729 21217 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
W20260709 13:08:15.678411 26479 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:08:15.663861 21066 rpcz_store.cc:276] 0709 13:08:14.188677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.195487 (+  6810us) service_pool.cc:224] Handling call
0709 13:08:14.195885 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:14.196349 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:14.196380 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:14.196905 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:08:14.259865 (+ 62960us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:14.564964 (+305099us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b180049a980. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f45f143c7 00007f7f425269a7 00007f7f425360bf
0709 13:08:14.565160 (+   196us) log.cc:844] Serialized 14 byte log entry
0709 13:08:14.973510 (+408350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.973620 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:15.554271 (+580651us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:15.658755 (+104484us) raft_consensus.cc:1685] finished
0709 13:08:15.658777 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.658924 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":479793,"prepare.queue_time_us":194123,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":35,"raft.queue_time_us":439733,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":9096,"spinlock_wait_cycles":34228480,"thread_start_us":253098,"threads_started":2,"wal-append.queue_time_us":224032}
I20260709 13:08:15.690801 26458 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.691638 26458 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.668171 26450 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.669116 26226 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.704330 21068 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:15.669925 26306 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:15.715413 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 148192us
W20260709 13:08:15.717739 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 50729.7us
W20260709 13:08:15.720594 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51071us
I20260709 13:08:15.735463 21064 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:15.736980 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.737828 21066 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:15.734901 21063 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:15.709843 26450 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:15.667991 26428 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.743067 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.707907 26226 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:15.716826 25495 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:15.747429 21063 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:15.746313 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.751856 26210 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:15.752691 26435 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.577s	user 0.005s	sys 0.004s
I20260709 13:08:15.741931 26454 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.761420 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:15.688040 26459 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.776388 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.760143 26155 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:15.771404 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 50068.7us
I20260709 13:08:15.780098 21064 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:15.739903 26333 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.781384 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:15.744977 26469 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:08:15.764689 26429 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:15.745007 26054 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.795696 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:15.797147 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.801370 26232 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:15.802048 26232 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:15.804651 26232 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:15.807099 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:15.808869 26459 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:15.809417 26477 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.375s	user 0.001s	sys 0.001s
I20260709 13:08:15.770411 26454 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.812069 21217 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:15.813517 26454 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:15.815850 26261 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.783883 25817 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:15.772336 26335 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:08:15.820421 26428 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:15.794723 26251 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:15.821327 26251 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.821692 26251 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:15.824499 26261 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:15.824810 26333 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:15.826445 26210 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:15.832834 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:15.833261 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:15.820421 21065 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:15.838362 21066 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:15.840582 26436 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:15.841738 26083 consensus_meta.cc:287] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.003s
I20260709 13:08:15.841656 26458 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:15.842785 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.770264 26435 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:15.874897 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.854895 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 59276.4us
I20260709 13:08:15.872462 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:15.918677 21063 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:15.872929 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:15.919785 21801 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:15.920135 21792 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:15.918121 21797 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:15.920708 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:15.919035 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2410) took 1736 ms. Trace:
I20260709 13:08:15.921456 21065 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:15.911342 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:15.922124 21070 rpcz_store.cc:276] 0709 13:08:14.182489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.577784 (+1395295us) service_pool.cc:224] Handling call
0709 13:08:15.592445 (+ 14661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.592751 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:15.728349 (+135598us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.861484 (+133135us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.918688 (+ 57204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:15.918742 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:15.918785 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.918847 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.918977 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56879,"prepare.queue_time_us":190293,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":58845,"thread_start_us":236225,"threads_started":2}
I20260709 13:08:15.854070 26428 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:15.854408 26059 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:15.869669 26459 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:15.933959 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 79784.6us
I20260709 13:08:15.936558 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:15.939210 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 106014us
I20260709 13:08:15.939927 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:15.939034 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:15.938995 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.941725 21071 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:15.778867 26306 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.943194 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.944166 21063 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:15.838817 26226 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.941180 21797 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:15.945060 21795 raft_consensus.cc:2368] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16.
W20260709 13:08:15.946415 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 101308us
I20260709 13:08:15.947685 21069 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:08:15.948405 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.869448 26241 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:08:15.952028 21070 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:15.951149 21065 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:15.953217 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:15.874085 26083 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.957180 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:15.958300 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:15.947938 26421 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:15.964280 26251 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.967265 26333 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:15.968791 26435 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:15.964406 21072 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:15.930694 26492 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:08:15.956028 26232 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:15.955507 26394 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:15.990554 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:15.992916 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:15.943179 26467 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:15.996784 21070 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
W20260709 13:08:15.996249 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.002862 21144 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:16.005998 21072 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:16.006466 21072 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:15.993393 26334 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.007s	sys 0.000s
W20260709 13:08:16.022848 26334 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.008s	sys 0.001s
I20260709 13:08:15.992358 26426 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:16.008086 21139 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:16.039202 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:16.002393 21796 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:16.040068 21797 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:16.008296 26251 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:15.997556 26333 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:16.008874 21141 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:08:16.019062 21072 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.019524 26232 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.021778 26089 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.005839 26483 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:08:16.001623 26394 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:15.996024 26421 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:16.034117 26432 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:15.957480 26459 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:08:16.045393 21035 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:16.043002 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52551.4us
I20260709 13:08:16.047019 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:16.053267 21070 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:16.052503 26472 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.055613 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:16.056025 21215 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:16.056337 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:16.056551 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:16.056659 26333 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.058681 26182 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:16.058511 26435 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:16.061574 21063 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:16.064647 21144 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:16.065490 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:16.097882 21142 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:16.064127 21072 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:16.066455 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:16.098733 21145 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:16.067515 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:15.994443 26429 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:08:16.083930 21147 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:16.084483 21139 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:16.086444 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:16.050900 26426 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:16.087532 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:16.088297 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:16.111338 21221 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:16.094976 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:16.109643 21145 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:16.050786 26487 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:16.113629 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:16.124121 21070 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:08:16.118221 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:16.119645 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:16.120940 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:16.125976 21218 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:16.068409 26394 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:16.107686 26472 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.127362 21063 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:16.132160 21063 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:16.131968 26426 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.133003 21063 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.133405 26472 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.133935 21063 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:16.136080 26421 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.140695 26232 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:16.141023 26226 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.141422 21063 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:08:16.141835 21063 pending_rounds.cc:85] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 5
I20260709 13:08:16.142076 21063 pending_rounds.cc:107] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:08:16.142297 21063 raft_consensus.cc:2889] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:16.143061 26467 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:16.095818 26182 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:16.127024 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:16.150058 21185 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:16.155486 21221 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:16.166653 21063 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:16.167189 21063 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:16.167604 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:16.171779 21147 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:16.174403 21142 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:16.170157 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:16.178082 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:16.123957 26445 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:16.187654 26445 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:16.160401 26242 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:16.190996 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:16.166571 26494 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.098s	user 0.002s	sys 0.000s
W20260709 13:08:16.175690 26501 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.107s	user 0.000s	sys 0.002s
I20260709 13:08:16.154721 26083 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:16.163853 26226 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:16.183794 26493 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.124s	user 0.000s	sys 0.002s
I20260709 13:08:16.197839 26445 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:16.213353 26226 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.216210 26485 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:08:16.217882 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:16.218657 21799 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:16.196717 21221 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:16.176002 26487 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:16.199991 21063 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.219704 26488 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:08:16.220399 21063 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:16.220530 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:16.209584 21072 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:08:16.189913 26229 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:16.221305 21072 pending_rounds.cc:85] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 6
I20260709 13:08:16.221459 26229 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.222010 26229 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:16.222157 26467 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:16.222640 26467 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.223006 26467 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:16.223536 26489 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:08:16.160216 26452 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:08:16.185729 26334 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:08:16.193806 26335 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:16.228804 26335 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.229187 26335 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:16.229674 21063 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:08:16.230072 21063 pending_rounds.cc:85] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 6
I20260709 13:08:16.230295 21063 pending_rounds.cc:107] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:08:16.230521 21063 raft_consensus.cc:2889] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:16.189138 26435 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:08:16.185611 26484 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:08:16.237013 26450 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.237625 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:16.238054 26465 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:16.238334 21792 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:16.238519 26465 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:16.239410 26465 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.199601 26242 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.240815 26242 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:16.242812 26474 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.675s	user 0.000s	sys 0.002s
I20260709 13:08:16.237627 26489 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:16.243783 26489 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:16.244711 26489 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.245997 26489 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:16.247709 26089 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:16.248103 26089 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:16.248943 26089 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.250130 26089 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:16.190897 26155 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:16.251632 26155 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:16.252552 26155 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.254140 21792 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:08:16.260444 26428 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:16.261596 26428 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.260648 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2437) took 1236 ms. Trace:
I20260709 13:08:16.221632 21072 pending_rounds.cc:107] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:08:16.267982 21072 raft_consensus.cc:2889] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:16.268204 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:16.270184 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:16.272102 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:16.273690 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:16.275112 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 321995us
I20260709 13:08:16.275542 21218 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:16.276808 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:16.276829 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 231502us
I20260709 13:08:16.277623 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:16.265116 21067 rpcz_store.cc:276] 0709 13:08:15.023894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.811478 (+787584us) service_pool.cc:224] Handling call
0709 13:08:15.811820 (+   342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:15.812188 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.812216 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:15.812798 (+   582us) log.cc:844] Serialized 52 byte log entry
0709 13:08:15.873817 (+ 61019us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:15.942007 (+ 68190us) log.cc:844] Serialized 14 byte log entry
0709 13:08:16.260005 (+317998us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b18002e6840. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287 00007f7f42537208
0709 13:08:16.260194 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.260286 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.260333 (+    47us) raft_consensus.cc:1685] finished
0709 13:08:16.260445 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.260588 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12225,"prepare.queue_time_us":68903,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":326636,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":194,"spinlock_wait_cycles":3533568,"thread_start_us":60626,"threads_started":1,"wal-append.queue_time_us":139070}
I20260709 13:08:16.277868 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:16.279135 21072 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:16.281764 21147 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:16.281951 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:16.284294 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:16.285390 21142 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
W20260709 13:08:16.310276 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.336494 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:16.337170 21142 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:16.230887 26467 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.289417 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:16.342154 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 63549.7us
I20260709 13:08:16.346995 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:16.365103 21067 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:16.274804 26502 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:16.279660 25495 raft_consensus.cc:2764] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:08:16.371323 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:16.372205 21071 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:16.366079 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:16.280984 26435 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:16.279840 25817 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:16.294600 26458 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:16.423660 26423 raft_consensus.cc:2764] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:08:16.413780 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 219275us
I20260709 13:08:16.413486 26059 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:16.424909 26059 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.425302 26059 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:16.341799 26340 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:08:16.404249 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.345858 26261 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:16.428279 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:16.356841 26474 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:08:16.431411 21065 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:08:16.437798 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:16.438464 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1627) took 1311 ms. Trace:
I20260709 13:08:16.439519 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:16.438822 21065 rpcz_store.cc:276] 0709 13:08:15.126790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.954694 (+827904us) service_pool.cc:224] Handling call
0709 13:08:16.430720 (+476026us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:08:16.430973 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:16.438188 (+  7215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.438409 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6724480}
I20260709 13:08:16.412227 21142 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:16.444643 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:16.280395 26454 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 13:08:16.335992 26232 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:16.341092 26450 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.434252 25817 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.441753 26224 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:08:16.435974 26261 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:16.467527 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 107900us
W20260709 13:08:16.471925 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 195539us
W20260709 13:08:16.488730 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.440944 26054 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.434321 26026 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:16.517273 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:16.445255 26426 raft_consensus.cc:2764] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:08:16.523036 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:16.517127 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56217.6us
W20260709 13:08:16.540853 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 74550.5us
I20260709 13:08:16.447858 26340 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:16.553828 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:16.559773 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:16.561100 21219 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:16.561316 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:16.535477 26507 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:08:16.562179 21216 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:16.456996 26232 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.522601 26261 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:16.464524 26333 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:16.463474 26488 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:16.622179 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:16.562858 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:16.463114 26435 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:16.543181 26458 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.612898 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:16.634553 21066 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:16.624892 21219 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:16.514678 26487 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:16.623711 21063 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:08:16.463693 26498 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:16.626569 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67656us
I20260709 13:08:16.560070 26224 consensus_queue.cc:1243] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:16.607040 26232 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:16.637929 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 84543.3us
I20260709 13:08:16.622282 26488 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:16.558831 26026 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:16.657248 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 81433.2us
I20260709 13:08:16.645824 21066 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:16.608654 26426 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
W20260709 13:08:16.656591 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 201443us
I20260709 13:08:16.663594 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:16.654811 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:16.553210 26054 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.668366 25817 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:16.670280 26291 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:16.670887 26291 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:16.677043 26026 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:16.681754 26394 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.000s
I20260709 13:08:16.682219 26394 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.684289 26467 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.659854 26333 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:16.684890 26054 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:16.685716 26333 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.687381 26435 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.644214 26335 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.638800 26458 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.691730 26458 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.691792 26361 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:16.660210 26224 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:16.692471 26361 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:16.692539 26224 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:16.694250 26224 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:16.695148 26518 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:16.658008 26488 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:16.696048 26512 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:08:16.663043 26340 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.621882 26445 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.700750 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:16.701763 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.704679 21063 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:08:16.705192 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.709322 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45795.5us
I20260709 13:08:16.710536 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:16.710538 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:16.715751 26454 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:08:16.718681 26509 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:08:16.715772 26491 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:08:16.722309 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1691) took 1679 ms. Trace:
I20260709 13:08:16.700838 26442 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:08:16.651681 26498 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.682776 26242 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:16.732384 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 77661.9us
I20260709 13:08:16.729815 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:16.733032 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:16.731544 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 86537.9us
W20260709 13:08:16.736294 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.735580 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:16.722886 21212 rpcz_store.cc:276] 0709 13:08:15.042410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.319849 (+277439us) service_pool.cc:224] Handling call
0709 13:08:15.320499 (+   650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.320840 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:15.517678 (+196838us) log.cc:844] Serialized 14 byte log entry
0709 13:08:15.662590 (+144912us) log.cc:844] Serialized 14 byte log entry
0709 13:08:16.721642 (+1059052us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b180049b0a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287
0709 13:08:16.721773 (+   131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:16.721840 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:16.721909 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.722007 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.722174 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7095,"prepare.queue_time_us":139531,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":6791,"raft.queue_time_us":207696,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":13169,"spinlock_wait_cycles":7688576,"thread_start_us":368425,"threads_started":3,"wal-append.queue_time_us":230314}
I20260709 13:08:16.730487 26026 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:16.760388 26330 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:16.763865 26520 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:08:16.765985 26226 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.767482 26487 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:16.768750 26182 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.769896 26182 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:16.765774 26334 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:16.771365 26524 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:16.771844 26524 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:16.771922 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 992) took 1676 ms. Trace:
I20260709 13:08:16.772277 26524 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:16.772380 21795 rpcz_store.cc:276] 0709 13:08:15.095301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.102678 (+  7377us) service_pool.cc:224] Handling call
0709 13:08:16.771675 (+1668997us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b5400494b54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:16.771835 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1726564224}
I20260709 13:08:16.757869 26499 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:16.775467 26469 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:16.775959 26469 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:16.775847 26499 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.776602 26242 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:16.777012 26469 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.777829 26394 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.778193 26499 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.778448 26469 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:16.779034 26498 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:16.780128 26510 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:08:16.769448 26291 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.740887 26459 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:08:16.743880 26054 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:16.750236 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:16.756819 21063 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
W20260709 13:08:16.809778 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59607.9us
I20260709 13:08:16.744436 26450 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:16.812463 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.821208 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 256021us
W20260709 13:08:16.812804 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.827976 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.828091 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:16.832592 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:16.742595 26458 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:08:16.758368 26026 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:16.832963 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 82851us
W20260709 13:08:16.825141 26229 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.603s	user 0.005s	sys 0.005s
I20260709 13:08:16.840900 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:16.859297 26089 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.609s	user 0.010s	sys 0.005s
I20260709 13:08:16.865164 26515 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:16.865666 26515 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.865998 26515 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:16.868897 26059 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.833542 26502 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:16.831323 26499 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.879346 21065 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:16.830910 26054 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.796057 26489 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:08:16.917704 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:16.918581 21801 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:16.850279 26026 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:16.850023 25817 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:16.854504 26408 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:08:16.933200 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 92366.2us
I20260709 13:08:16.889086 26059 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:16.858083 26232 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.811184 26525 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
W20260709 13:08:16.942058 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 284127us
I20260709 13:08:16.952117 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:16.954308 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.953416 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:16.836809 26450 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.920925 26054 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:16.961880 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:16.963049 21215 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:16.911212 26421 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:16.941459 26522 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:08:16.861559 26229 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.944049 26059 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:16.988981 26059 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:16.940627 26499 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:16.889721 26529 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 13:08:16.983927 21184 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:16.954733 26408 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:16.997092 21215 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:08:17.002408 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.003584 21212 raft_consensus.cc:2368] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:16.974488 26531 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.962328 26232 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:17.005525 26059 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:17.007846 26089 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.002439 26459 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:17.009473 26241 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:17.018252 26504 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.868s	user 0.004s	sys 0.000s
I20260709 13:08:17.018644 26528 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:17.018914 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:17.087013 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.087200 21067 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:08:17.019860 26523 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 13:08:17.020987 26513 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:08:17.022197 26438 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:17.022691 26450 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:17.024627 26432 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:16.985901 26330 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:08:17.086638 26241 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.095985 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.096823 21147 raft_consensus.cc:2368] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:17.096704 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1625) took 2018 ms. Trace:
I20260709 13:08:17.099224 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.031419 26224 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:17.035197 26514 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.391s	user 0.000s	sys 0.001s
W20260709 13:08:17.036821 26517 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
W20260709 13:08:17.038205 26519 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.342s	user 0.001s	sys 0.000s
I20260709 13:08:17.039311 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.040818 26515 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:17.043453 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:17.044763 26408 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:17.046397 26530 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:17.047363 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:17.048424 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:16.996284 26458 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.080200 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:16.975498 26261 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.118211 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:17.083592 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4580969850001111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
I20260709 13:08:17.118656 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:17.084404 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.119614 21146 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:08:17.119035 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4274969850000616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
I20260709 13:08:17.120450 26521 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:17.046422 26527 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:08:17.121320 26458 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:17.122015 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:17.122608 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2449) took 1287 ms. Trace:
I20260709 13:08:17.123157 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:17.123719 26458 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:17.124436 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:17.123052 21070 rpcz_store.cc:276] 0709 13:08:15.835236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.129679 (+294443us) service_pool.cc:224] Handling call
0709 13:08:16.176412 (+ 46733us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:16.357349 (+180937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:16.357384 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:16.370902 (+ 13518us) log.cc:844] Serialized 52 byte log entry
0709 13:08:16.571098 (+200196us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:16.571163 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.571264 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:17.122368 (+551104us) raft_consensus.cc:1685] finished
0709 13:08:17.122398 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.122537 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106248,"raft.queue_time_us":153195,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":19390,"thread_start_us":103381,"threads_started":1,"wal-append.queue_time_us":418760}
I20260709 13:08:17.125819 26261 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:17.052090 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.126693 21144 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:08:17.126829 21797 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:17.059307 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:17.085677 26459 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:16.986291 26488 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.128324 21212 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:17.129410 26488 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:17.129557 26459 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:17.130053 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.130445 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:17.092550 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.052693 21141 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:17.131031 21797 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:17.131016 26469 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:17.011888 26445 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:17.131964 26469 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:17.090158 26528 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.132367 26469 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:17.132558 26528 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:17.133258 26469 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.134392 26469 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:17.097241 21069 rpcz_store.cc:276] 0709 13:08:15.077904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.953823 (+875919us) service_pool.cc:224] Handling call
0709 13:08:15.982340 (+ 28517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:16.276645 (+294305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:16.276680 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:16.288089 (+ 11409us) log.cc:844] Serialized 26 byte log entry
0709 13:08:16.656356 (+368267us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:16.656401 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.656476 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.717815 (+ 61339us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b4800080358. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:08:17.092447 (+374632us) raft_consensus.cc:1685] finished
0709 13:08:17.092469 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.096634 (+  4165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":293872,"raft.queue_time_us":353838,"raft.run_cpu_time_us":3499,"raft.run_wall_time_us":375153,"spinlock_wait_cycles":27336960,"thread_start_us":242485,"threads_started":1,"wal-append.queue_time_us":386068}
I20260709 13:08:17.138883 21069 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:08:17.111609 21065 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:17.092056 26438 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:17.144129 21218 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:17.116695 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69446us
I20260709 13:08:17.085970 26089 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:17.120692 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3509969850000516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
I20260709 13:08:17.121225 26521 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.012051 26436 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:17.089006 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:16.986488 26421 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:17.141906 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 158054us
I20260709 13:08:17.138912 21797 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
W20260709 13:08:17.145881 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3473969850000458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
I20260709 13:08:17.149950 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.147409 21065 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
W20260709 13:08:17.150254 21036 connection.cc:256] RPC call timeout handler was delayed by -1.34379698500004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
I20260709 13:08:17.149950 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.150780 21035 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:17.153494 26241 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:17.153677 26515 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:17.154815 26537 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:17.155539 26537 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:17.144763 26524 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.155905 21071 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:17.156383 26537 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.157012 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:17.157552 21071 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:08:17.157919 26524 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:17.150780 21146 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:17.159837 26054 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:17.160990 26421 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:17.150938 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2942969850000736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 13:08:17.161644 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2312969850000854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:08:17.161944 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0809969850000698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
I20260709 13:08:17.162194 26054 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:17.162207 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0332969850001064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2014s for a response
I20260709 13:08:17.163964 26229 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:17.164973 21036 leader_election.cc:400] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:17.165436 21036 leader_election.cc:403] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:17.164772 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:17.163995 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:17.166900 21069 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:17.151432 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:17.167882 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:17.169183 21067 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:17.171694 21184 leader_election.cc:400] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16.
I20260709 13:08:17.174763 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:17.175958 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:17.176705 21792 raft_consensus.cc:2368] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:17.178449 21065 raft_consensus.cc:2410] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:08:17.050338 26531 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.179980 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2436) took 2156 ms. Trace:
I20260709 13:08:17.182957 21146 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:17.184366 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:17.183609 21064 rpcz_store.cc:276] 0709 13:08:15.023626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.780903 (+757277us) service_pool.cc:224] Handling call
0709 13:08:15.792036 (+ 11133us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:15.834829 (+ 42793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.834860 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:15.846950 (+ 12090us) log.cc:844] Serialized 26 byte log entry
0709 13:08:16.043007 (+196057us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:16.043054 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.043128 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.871040 (+827912us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:17.179698 (+308658us) raft_consensus.cc:1685] finished
0709 13:08:17.179723 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.179921 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1384,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":3868,"spinlock_wait_cycles":31076096,"thread_start_us":113212,"threads_started":1,"wal-append.queue_time_us":132478}
I20260709 13:08:17.185616 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.186017 21064 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:17.186282 21214 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:17.187659 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.188591 21221 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:17.181164 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:17.191188 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.193022 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.193305 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.180013 26436 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:17.197870 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.193729 21215 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:17.199088 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.177201 26261 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:17.201553 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.197027 26432 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:17.187614 26306 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:17.207594 21801 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
W20260709 13:08:17.210844 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46018.7us
I20260709 13:08:17.216353 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.216980 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:17.213455 21069 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:17.217505 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:17.217993 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:17.218263 21795 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:08:17.218838 21215 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 15: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:17.213640 21221 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:17.214097 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:17.214735 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.220515 21214 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:17.215613 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.220157 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:17.221788 26394 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:17.222337 26432 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:17.222849 26436 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.221894 21219 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:08:17.227102 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.227972 21142 raft_consensus.cc:2368] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:17.202069 26499 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:17.229691 21146 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:17.229689 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.232276 25817 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:17.232656 26089 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:17.233901 26511 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.598s	user 0.001s	sys 0.002s
I20260709 13:08:17.237712 26340 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:17.241706 26521 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:17.243306 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:17.243961 26526 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.413s	user 0.000s	sys 0.002s
I20260709 13:08:17.246109 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:17.246598 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.248646 21064 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:08:17.248924 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 488996us
W20260709 13:08:17.278121 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.284533 26458 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:17.287514 26089 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:17.288820 26089 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:17.289217 26089 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:17.289852 26335 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:17.290448 26335 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.290163 26089 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:17.291587 21066 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:17.244454 26026 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:17.277217 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:17.294615 21141 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:17.249601 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:17.227511 26450 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:17.252336 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.229876 26394 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:17.262161 21071 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:17.210461 26536 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:17.264884 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:17.303889 21220 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:17.270817 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.269852 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:17.228634 26432 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.273084 26445 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:17.299644 21142 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:17.254591 26531 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:17.212432 26261 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.313277 21214 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
W20260709 13:08:17.302453 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 85547.2us
I20260709 13:08:17.305231 21212 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:17.290913 26335 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:17.315121 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:17.317786 26224 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:17.211400 26438 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.292014 26340 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:17.322933 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:17.324512 26224 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:17.326083 26528 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.314227 26089 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:17.328034 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:17.342075 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:17.336082 26502 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:17.320670 26515 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:17.346698 21219 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:08:17.347729 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.319479 26438 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:17.345454 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:17.349120 21212 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:17.292230 26306 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:08:17.336671 26433 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:17.369928 26433 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:17.370744 26433 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.357534 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.359570 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.353334 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:17.373518 21064 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:17.357136 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:17.377770 21184 leader_election.cc:400] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:17.378321 21184 leader_election.cc:403] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:08:17.375432 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:17.380158 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:17.381307 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:17.382098 21215 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:17.336118 26224 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:17.445760 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 117793us
I20260709 13:08:17.475931 21036 leader_election.cc:400] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 10 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 10. Current term is 11.
I20260709 13:08:17.446274 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:17.390996 21220 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:17.485826 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:17.486033 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 105947us
I20260709 13:08:17.486485 21801 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:17.487077 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:17.423794 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.371423 26515 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:17.488837 21064 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:08:17.365846 26261 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:17.449563 26531 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:08:17.532629 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.368707 26335 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:17.485323 26433 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:17.482990 26241 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:17.476452 26333 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:17.533757 21035 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:17.489455 26533 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.485569 26458 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:17.364472 26394 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.365986 26467 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.174s
W20260709 13:08:17.559221 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 83439us
I20260709 13:08:17.563040 21212 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:17.503587 26459 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:17.535221 26499 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.535818 26531 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.579818 21064 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:17.583315 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:17.584088 21064 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:17.584008 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:17.585287 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.586006 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:17.556627 26432 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.371155 26340 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.596943 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:17.567849 26224 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:17.574498 26054 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:08:17.608433 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:17.638162 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.601009 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:17.644345 26528 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:17.548597 26498 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:08:17.648811 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51938.7us
I20260709 13:08:17.593592 26536 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:17.588711 26531 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:17.658483 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:17.661726 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:17.570690 26241 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.635974 26059 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:17.577809 26458 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.572901 26533 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.652842 26224 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.671310 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:17.655599 26428 raft_consensus.cc:2764] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:08:17.594969 26499 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:17.670434 25495 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:17.674840 25817 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:17.706398 21036 leader_election.cc:400] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:17.707638 21036 leader_election.cc:403] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:17.710616 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:17.711511 26534 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:17.711925 26534 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:17.712306 26534 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:17.722219 26534 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:17.724387 21036 leader_election.cc:400] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:17.678462 26543 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:17.727907 21036 leader_election.cc:403] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:17.686338 26026 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:17.729826 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.731272 21144 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:17.733207 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:17.733314 21141 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:17.732470 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:17.693602 26224 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:17.734068 21145 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:17.735142 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:17.736598 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:17.744154 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.736495 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:17.771751 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.770965 21140 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:17.620577 26432 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:17.705951 26423 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:17.726310 26059 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:17.656183 26515 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:17.785037 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 48648.6us
W20260709 13:08:17.788241 26560 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:08:17.795149 26499 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:17.770109 26428 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:17.803112 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:17.806284 26545 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.487s	user 0.001s	sys 0.000s
I20260709 13:08:17.804452 21145 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:17.957077 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:17.808225 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:17.737311 26543 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:17.952999 21065 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:17.958397 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 225257us
I20260709 13:08:17.959551 21795 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:17.784940 26026 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:17.962160 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:17.971800 21139 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:08:17.991876 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:17.996018 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:17.973510 21138 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:18.008175 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:18.009119 21142 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:17.945577 26498 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.014385 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:18.015748 21138 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:17.998579 26533 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.018837 26467 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:18.008039 26536 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.021311 21072 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 13:08:18.028168 26561 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.375s	user 0.000s	sys 0.001s
I20260709 13:08:18.028396 26426 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:18.028965 26426 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.029528 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 999) took 2156 ms. Trace:
I20260709 13:08:18.029985 21793 rpcz_store.cc:276] 0709 13:08:15.872934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.922798 (+ 49864us) service_pool.cc:224] Handling call
0709 13:08:15.945411 (+ 22613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:15.952568 (+  7157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.952599 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:15.956516 (+  3917us) log.cc:844] Serialized 26 byte log entry
0709 13:08:17.371493 (+1414977us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b1800475420. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:17.560633 (+189140us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:17.560675 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.560742 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:18.029159 (+468417us) raft_consensus.cc:1685] finished
0709 13:08:18.029181 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:18.029442 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8180,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":917,"spinlock_wait_cycles":47280000,"thread_start_us":188950,"threads_started":1,"wal-append.queue_time_us":1612664}
W20260709 13:08:18.029683 26557 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
W20260709 13:08:18.035970 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.957042 26423 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.046729 21067 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:18.058379 21142 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:08:18.062199 21065 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:18.065356 21138 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:17.965998 25817 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:18.032382 26241 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.033214 26465 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:18.071748 26438 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:18.072564 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:18.031220 26426 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.074602 26445 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:18.113605 26445 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.115833 26445 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:18.075193 21795 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:18.078085 26423 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.122910 26423 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.123780 26458 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:18.124034 20966 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 258) took 44906 ms (client timeout 15000 ms). Trace:
I20260709 13:08:18.066505 26498 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:18.077399 26556 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 13:08:17.967166 26521 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:18.079254 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.024s	sys 0.024s
I20260709 13:08:18.134500 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:18.136860 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.024s	sys 0.024s
I20260709 13:08:18.137051 21146 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:18.137701 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 650625us
I20260709 13:08:18.138717 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:18.020318 26536 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.010878 26442 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.011734 26059 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:18.044867 26570 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:18.097049 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 308031us
I20260709 13:08:18.146642 21065 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:17.966701 26564 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.148524 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:18.150450 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:17.972798 26537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.024s	sys 0.031s
I20260709 13:08:18.107126 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:18.131445 26565 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.152281 26565 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.154603 26565 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.108260 21142 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:18.155412 26224 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.156685 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:18.157642 21142 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:18.157764 26224 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:18.111747 26553 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:08:18.112979 26534 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.126586 20969 catalog_manager.cc:5697] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 3 to 9, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:08:18.134204 21141 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:08:18.011914 26428 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:18.140909 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:18.164047 21765 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.538s (SENT) [suppressed 7 similar messages]
I20260709 13:08:18.097162 26432 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.109457 26438 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:18.176519 26438 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.177909 26438 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:18.158560 21064 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:18.145301 26458 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:18.160195 26089 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.831s	user 0.012s	sys 0.000s
I20260709 13:08:18.160550 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:18.162920 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:18.164450 21794 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:18.168673 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:18.147949 26333 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
I20260709 13:08:18.169661 21065 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:08:18.171069 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:18.159200 26537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.036s	sys 0.040s
I20260709 13:08:18.144696 26059 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:18.144984 26521 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.272326 26543 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.183449 26054 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:08:18.232178 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.278808 26059 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:18.229404 26573 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:08:18.279933 26333 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:18.280762 26224 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.281855 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:18.291325 21144 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:18.283079 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.184038 20969 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 9 to 10, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 10 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.204166 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:18.284219 21795 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:18.124431 20966 rpcz_store.cc:269] 0709 13:07:33.217658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.217924 (+   266us) service_pool.cc:224] Handling call
0709 13:07:34.439385 (+1221461us) spinlock_profiling.cc:243] Waited 52 ms on lock 0x7b5400002490. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f80b2 00007f7f48371a89 00007f7f48309da6 00007f7f482db088 00007f7f483d937e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc
0709 13:07:35.116478 (+677093us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b540039d5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42486090 00007f7f42524f3a 00007f7f482db1cc 00007f7f483d937e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc 00007f7f46899392 00007f7f4085748c 00007f7f408568c6
0709 13:08:18.123795 (+43007317us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:05.235022 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:05.486805 (+251783us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:05.580699 (+ 93894us) write_op.cc:435] Acquired schema lock
0709 13:08:05.585766 (+  5067us) tablet.cc:662] Decoding operations
0709 13:08:06.148835 (+563069us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:06.159215 (+ 10380us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:06.161402 (+  2187us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:08:06.235726 (+ 74324us) tablet.cc:701] Row locks acquired
0709 13:08:06.235802 (+    76us) write_op.cc:260] PREPARE: finished
0709 13:08:06.235995 (+   193us) write_op.cc:270] Start()
0709 13:08:06.236160 (+   165us) write_op.cc:276] Timestamp: P: 1783602486235920 usec, L: 0
0709 13:08:06.236205 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 13:08:06.237046 (+   841us) log.cc:844] Serialized 8737 byte log entry
0709 13:08:08.213692 (+1976646us) op_driver.cc:464] REPLICATION: finished
0709 13:08:08.992692 (+779000us) write_op.cc:301] APPLY: starting
0709 13:08:09.037717 (+ 45025us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:09.525440 (+487723us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:09.525590 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:10.526168 (+1000578us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:10.526768 (+   600us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:08:10.526820 (+    52us) write_op.cc:312] APPLY: finished
0709 13:08:10.582158 (+ 55338us) log.cc:844] Serialized 196 byte log entry
0709 13:08:10.951231 (+369073us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:11.009177 (+ 57946us) write_op.cc:454] Released schema lock
0709 13:08:11.038409 (+ 29232us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:11.188834 (+150425us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:11.189605 (+   771us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3403146,"reported_tablets":49,"spinlock_wait_cycles":122747648,"child_traces":[["op",{"apply.queue_time_us":718655,"apply.run_cpu_time_us":101581,"apply.run_wall_time_us":2176000,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96131,"num_ops":22,"prepare.queue_time_us":367723,"prepare.run_cpu_time_us":77811,"prepare.run_wall_time_us":1542769,"raft.queue_time_us":819500,"raft.run_cpu_time_us":27571,"raft.run_wall_time_us":151602,"replication_time_us":1977298,"thread_start_us":2095629,"threads_started":5,"wal-append.queue_time_us":1547553,"wal-append.run_cpu_time_us":27657,"wal-append.run_wall_time_us":3095570}]]}
W20260709 13:08:18.181571 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.241791 26426 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:18.179189 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:18.255591 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:18.285504 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.306915 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:18.308130 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 129038us
I20260709 13:08:18.312875 26537 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:18.313388 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:18.313861 20969 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.207036 26520 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.315590 26520 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.318295 26520 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.318918 26520 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:18.278466 26054 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.204018 21793 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 9.
I20260709 13:08:18.147110 26442 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:18.243750 26433 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.003s
I20260709 13:08:18.163818 26564 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:18.352229 26361 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.353287 26361 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:18.353550 26442 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:18.353996 26564 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:08:18.302755 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:18.344218 26562 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.354827 21067 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:18.354905 26562 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.356058 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:18.303846 21798 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 10: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:08:18.304175 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 133146us
I20260709 13:08:18.310066 26426 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.298477 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:18.251106 26428 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:18.307621 26458 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:18.312137 26089 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:18.333341 26536 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.334560 26521 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:18.336462 21036 leader_election.cc:400] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:18.356913 26054 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:18.338470 21146 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:18.356736 26568 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:18.340413 21795 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:18.341563 21147 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:18.309935 26224 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:18.349931 26531 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.671s	user 0.009s	sys 0.000s
I20260709 13:08:18.350834 26498 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.304816 26423 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.158737 26570 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.357627 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:18.393777 21065 raft_consensus.cc:2410] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:08:18.395020 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:18.395834 21065 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:08:18.396924 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:18.397553 21065 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:18.358762 21067 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 9: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:18.345577 21144 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:18.358356 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:18.402392 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44426.5us
I20260709 13:08:18.402918 21144 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:18.227373 26445 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:18.376194 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:18.337539 20969 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 11, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 11 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:18.196312 21138 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:18.404582 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:18.360965 21071 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:18.416431 21144 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:08:18.416934 21144 pending_rounds.cc:85] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 6
I20260709 13:08:18.417200 21144 pending_rounds.cc:107] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:08:18.417472 21144 raft_consensus.cc:2889] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:18.430097 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54005.4us
I20260709 13:08:18.429765 20969 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.431061 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:18.453954 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:18.465070 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.467413 21144 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:18.468111 21795 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:18.469726 21144 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:18.471251 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:18.475253 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:18.484298 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:18.377029 26059 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.491158 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:18.494513 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:18.495926 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.490780 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:18.497684 21795 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:18.500411 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 337973us
I20260709 13:08:18.429945 26089 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:18.503408 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:18.467763 26565 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:18.380888 26568 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.508229 21216 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:18.494540 21065 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:08:18.520418 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:18.505102 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:18.510632 21218 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:18.469151 26533 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.401427 26531 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.513419 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:18.402707 26423 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.511508 21795 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:18.466475 26210 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:18.484472 26261 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:18.485638 26520 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:18.486712 26241 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.485916 26232 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.482441 25762 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:18.525704 21072 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:18.526284 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:18.513176 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:18.526943 20969 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.507578 26458 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.531805 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:18.532681 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:18.535832 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:18.536821 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:18.496922 26453 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:18.502497 26483 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:18.564870 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:18.568310 21217 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:08:18.484079 26155 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:08:18.487506 26536 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:08:18.483846 26445 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.583609 25495 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.591511 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 892) took 1753 ms. Trace:
I20260709 13:08:18.637095 21063 rpcz_store.cc:276] 0709 13:08:16.837948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.838123 (+   175us) service_pool.cc:224] Handling call
0709 13:08:16.877049 (+ 38926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:18.122919 (+1245870us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b18003b3aa0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f42514060 00007f7f42514bfb 00007f7f4251f64b
0709 13:08:18.123081 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:18.123121 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:18.141959 (+ 18838us) log.cc:844] Serialized 26 byte log entry
0709 13:08:18.432655 (+290696us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:18.432692 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.432780 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:18.591115 (+158335us) raft_consensus.cc:1685] finished
0709 13:08:18.591138 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:18.591445 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12481,"raft.queue_time_us":97209,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":18328,"spinlock_wait_cycles":40195840,"thread_start_us":362204,"threads_started":2,"wal-append.queue_time_us":437976}
I20260709 13:08:18.591536 21216 raft_consensus.cc:2368] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
W20260709 13:08:18.603788 26558 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.956s	user 0.016s	sys 0.002s
I20260709 13:08:18.529680 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:18.736407 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 206833us
I20260709 13:08:18.737498 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:18.605335 21801 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:18.588624 20969 catalog_manager.cc:5697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 6 to 15, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 15 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:18.538975 26564 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:18.512218 26485 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:08:18.570351 26565 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:18.625855 21217 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:18.573587 26576 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:18.518141 26534 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:18.643746 21063 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:18.502688 26333 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.232s
I20260709 13:08:18.663832 21213 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:18.512077 26567 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.129s
I20260709 13:08:18.606287 26453 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.523365 26512 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:18.764539 21145 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 14. Ops: []
I20260709 13:08:18.765158 20969 catalog_manager.cc:5697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 5 to 8, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 8 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:18.770713 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:18.772691 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:18.574366 25817 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:08:18.495877 26428 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.570808 26533 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.777451 21067 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:18.777618 21063 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:18.592476 21219 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:18.780192 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:18.815220 25817 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:08:18.825444 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 45364.8us
I20260709 13:08:18.585584 25762 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.586742 26232 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.646492 26483 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:18.748365 25495 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.583863 26210 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.607921 26458 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.810848 21070 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:18.748675 26573 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:18.767810 26394 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:08:18.811685 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 166851us
I20260709 13:08:18.787441 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:18.624004 26241 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.826541 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:18.827440 21219 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:08:18.791204 26564 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:18.810619 26453 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:18.807386 26567 consensus_queue.cc:1243] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:18.830070 25817 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:18.810387 26512 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:18.849876 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:18.820971 26499 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.831198 26426 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:18.905361 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.893621 21144 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:18.833740 26026 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:08:18.900861 26584 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:08:18.900620 25817 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.902150 26423 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.897746 26512 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:18.902910 26570 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.914453 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2463) took 2064 ms. Trace:
I20260709 13:08:18.827564 26534 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.924240 21068 rpcz_store.cc:276] 0709 13:08:16.850499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.125273 (+274774us) service_pool.cc:224] Handling call
0709 13:08:17.125564 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:17.139982 (+ 14418us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:17.140014 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:17.140438 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:08:17.490073 (+349635us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:17.490127 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.490212 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:18.590878 (+1100666us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:18.914131 (+323253us) raft_consensus.cc:1685] finished
0709 13:08:18.914152 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:18.914389 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":9236224,"thread_start_us":349286,"threads_started":1,"wal-append.queue_time_us":423702}
I20260709 13:08:18.926684 21065 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:18.888787 26394 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:18.923334 25817 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:18.894237 26224 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.939041 26059 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.918313 26564 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:18.924257 26589 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:08:18.945415 26600 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:08:18.921661 26026 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:18.946549 26445 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.916326 26565 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.947356 26026 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:18.949503 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:18.944648 26361 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.591s	user 0.008s	sys 0.003s
I20260709 13:08:18.912629 26499 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.945010 26394 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.942003 26564 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:18.959568 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:18.960282 25495 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.965395 25762 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:18.967855 25495 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.968274 26232 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.968451 25495 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:18.969741 26583 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:08:18.971230 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2734) took 1015 ms. Trace:
I20260709 13:08:18.971942 26458 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.972028 25495 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:18.971621 21142 rpcz_store.cc:276] 0709 13:08:17.958691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.167883 (+209192us) service_pool.cc:224] Handling call
0709 13:08:18.170531 (+  2648us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:18.170992 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:18.171020 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:18.171581 (+   561us) log.cc:844] Serialized 52 byte log entry
0709 13:08:18.591600 (+420019us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:18.591640 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.591784 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:18.970995 (+379211us) raft_consensus.cc:1685] finished
0709 13:08:18.971016 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:18.971170 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13520,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":25152,"thread_start_us":316897,"threads_started":1,"wal-append.queue_time_us":546433}
I20260709 13:08:18.974013 26499 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.976636 26564 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:18.977200 26054 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 13:08:18.960979 26515 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.978816 26515 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.971743 26568 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:18.943586 26483 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:18.982179 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:18.988986 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:18.977808 26054 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.005695 26054 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:19.007864 26569 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.863s	user 0.002s	sys 0.000s
I20260709 13:08:19.017196 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 897) took 1765 ms. Trace:
I20260709 13:08:19.017627 21064 rpcz_store.cc:276] 0709 13:08:17.251858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.591932 (+340074us) service_pool.cc:224] Handling call
0709 13:08:19.016944 (+1425012us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54002ad0d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:19.017065 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1151188352}
I20260709 13:08:19.022428 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2724) took 1074 ms. Trace:
I20260709 13:08:19.022876 21140 rpcz_store.cc:276] 0709 13:08:17.948042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.948180 (+   138us) service_pool.cc:224] Handling call
0709 13:08:17.948568 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:18.358541 (+409973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:18.358574 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:18.369116 (+ 10542us) log.cc:844] Serialized 26 byte log entry
0709 13:08:18.497371 (+128255us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:18.497429 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.022023 (+524594us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540036c754. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:19.022073 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:19.022120 (+    47us) raft_consensus.cc:1685] finished
0709 13:08:19.022133 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:19.022362 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":644,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":14972,"spinlock_wait_cycles":941839232,"thread_start_us":124593,"threads_started":1,"wal-append.queue_time_us":360445}
I20260709 13:08:19.025434 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.026723 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.027271 21798 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:19.029095 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.029750 21801 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:18.959240 26531 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:19.044553 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55679.5us
I20260709 13:08:19.048700 26603 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:19.060236 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:19.049209 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.061205 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.062091 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.049213 26445 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:19.049717 25762 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:18.993436 26567 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:19.058924 26592 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:08:19.067381 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.081398 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2726) took 1132 ms. Trace:
I20260709 13:08:19.081702 21801 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:19.067834 21795 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:19.068145 26241 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:19.068763 21796 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:08:19.095767 21796 pending_rounds.cc:85] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 10
I20260709 13:08:19.096019 21796 pending_rounds.cc:107] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:08:19.069641 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:19.070811 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.075443 26445 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.078207 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:19.099860 21221 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:19.100144 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.094810 26333 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.081840 21141 rpcz_store.cc:276] 0709 13:08:17.948767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.948908 (+   141us) service_pool.cc:224] Handling call
0709 13:08:18.133816 (+184908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:18.166657 (+ 32841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.081149 (+914492us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540020add0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:19.081332 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424042880}
I20260709 13:08:19.096514 21800 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:08:19.096261 21796 raft_consensus.cc:2889] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:19.098400 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.123135 21792 raft_consensus.cc:2410] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:08:19.123771 21065 pending_rounds.cc:85] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 10
I20260709 13:08:19.124161 21065 pending_rounds.cc:107] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:08:19.125710 21065 raft_consensus.cc:2889] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:19.063622 26534 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:19.133445 21141 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:19.133659 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.134016 21141 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:19.134946 21141 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.135941 21141 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:19.137465 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:19.137892 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:19.138077 21140 raft_consensus.cc:2376] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17. Re-sending same reply.
I20260709 13:08:19.138293 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:19.141228 21140 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:19.144459 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.144869 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:19.145227 21140 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:19.146745 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:19.147754 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:19.148962 26465 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:19.151151 21141 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:19.147782 21213 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:19.153754 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:19.154168 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.159243 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.080786 26603 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:19.163866 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:19.121862 26155 raft_consensus.cc:2764] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:08:19.079463 26568 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:08:19.096254 25762 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.186952 21221 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:19.190343 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:19.182787 21140 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:19.160615 26241 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:08:19.099522 26567 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.126113 26333 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:19.201749 26548 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.261s	user 0.004s	sys 0.005s
I20260709 13:08:19.200817 26026 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:19.190953 26229 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:08:19.203107 26465 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:19.210065 26525 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.973s	user 0.012s	sys 0.031s
I20260709 13:08:19.188532 26604 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:08:19.191347 26602 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:08:19.212172 26534 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:19.225629 26525 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.988s	user 0.012s	sys 0.031s
W20260709 13:08:19.229449 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 72877.7us
I20260709 13:08:19.231853 26534 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.246661 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.247483 21142 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:19.247663 21144 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:19.234766 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.223453 26026 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:19.241761 21068 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:19.252383 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.244427 21139 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:19.245245 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.244079 21141 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:19.294701 21799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1005) took 2119 ms. Trace:
I20260709 13:08:19.298849 21146 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:19.300522 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.301988 26445 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:19.302196 26453 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.230433 26603 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.303556 21798 raft_consensus.cc:2368] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15.
I20260709 13:08:19.292754 26512 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.305330 26512 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:19.307488 21214 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:19.307722 25762 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:19.311702 26465 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.312237 26453 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:19.313082 26621 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:08:19.314267 21793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 1009) took 1269 ms (client timeout 1621 ms). Trace:
W20260709 13:08:19.314687 21793 rpcz_store.cc:269] 0709 13:08:18.045073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.072235 (+ 27162us) service_pool.cc:224] Handling call
0709 13:08:19.314111 (+1241876us) spinlock_profiling.cc:243] Waited 94.2 ms on lock 0x7b54000b5c54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:19.314210 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216589056}
I20260709 13:08:19.316170 26026 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.319651 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2731) took 1368 ms. Trace:
W20260709 13:08:19.320003 26612 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:08:19.320063 21145 rpcz_store.cc:276] 0709 13:08:17.950976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.967928 (+ 16952us) service_pool.cc:224] Handling call
0709 13:08:17.968222 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.807712 (+839490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.319347 (+511635us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b54002585d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:19.319574 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1830730624}
I20260709 13:08:19.321036 26534 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:19.321877 26550 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 2.004s	user 0.010s	sys 0.011s
I20260709 13:08:19.322532 26567 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:19.323078 26587 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.260s	user 0.001s	sys 0.001s
W20260709 13:08:19.324119 26609 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:08:19.324354 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2735) took 1355 ms. Trace:
I20260709 13:08:19.324705 26210 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:19.324781 21143 rpcz_store.cc:276] 0709 13:08:17.968631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.346032 (+377401us) service_pool.cc:224] Handling call
0709 13:08:18.346386 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:18.346737 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:18.346765 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:18.347240 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:08:18.517383 (+170143us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:18.517426 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.517522 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:19.324150 (+806628us) raft_consensus.cc:1685] finished
0709 13:08:19.324172 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:19.324296 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1057,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":77,"thread_start_us":169738,"threads_started":1,"wal-append.queue_time_us":612056}
I20260709 13:08:19.338263 21800 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:19.340021 21035 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:19.345674 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:19.295612 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 54522.6us
I20260709 13:08:19.346683 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:19.293586 26531 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.350731 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:19.337801 26613 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:08:19.299170 21799 rpcz_store.cc:276] 0709 13:08:17.175155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.182392 (+  7237us) service_pool.cc:224] Handling call
0709 13:08:17.200168 (+ 17776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:17.211792 (+ 11624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:17.211821 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:17.224320 (+ 12499us) log.cc:844] Serialized 572 byte log entry
0709 13:08:17.309692 (+ 85372us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:17.309730 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.309799 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:19.045510 (+1735711us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:19.249261 (+203751us) raft_consensus.cc:1685] finished
0709 13:08:19.249290 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:19.294602 (+ 45312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10239744,"thread_start_us":85069,"threads_started":1,"wal-append.queue_time_us":505247}
I20260709 13:08:19.352105 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:19.354648 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:19.356696 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:19.357412 21068 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:19.350415 26619 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.097s	user 0.000s	sys 0.002s
I20260709 13:08:19.358839 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:19.351481 25817 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:19.353904 26622 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:19.362417 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.374619 21142 raft_consensus.cc:2410] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 11 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:19.355901 26512 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.347815 26083 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:19.365216 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1658) took 1055 ms. Trace:
I20260709 13:08:19.368103 21145 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:08:19.380560 26485 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:19.381129 26512 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.382534 26488 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:19.383366 26512 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:19.373080 26241 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.384641 26241 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.386807 26241 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:19.383953 26512 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:19.388863 26512 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:19.356504 26531 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.380683 25817 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:19.390403 26465 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:19.391388 26577 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.962s	user 0.002s	sys 0.002s
I20260709 13:08:19.378207 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:19.394778 21068 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:19.380029 21063 rpcz_store.cc:276] 0709 13:08:18.309448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.824473 (+515025us) service_pool.cc:224] Handling call
0709 13:08:18.824869 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:18.832368 (+  7499us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:19.059725 (+227357us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.223600 (+163875us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:19.224159 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:08:19.224424 (+   265us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:19.224445 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.224491 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:19.365060 (+140569us) raft_consensus.cc:1685] finished
0709 13:08:19.365082 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:19.365169 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":90917,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":2138,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":18388,"thread_start_us":308874,"threads_started":2,"wal-append.queue_time_us":282774}
W20260709 13:08:19.401414 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.404479 21145 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:19.367086 26428 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:19.414621 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:19.407244 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.376543 26483 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.376173 25762 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.376358 26445 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:19.420876 26616 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 13:08:19.388880 26333 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:19.405758 26334 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.406114 26210 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.421201 26622 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:19.445492 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2459) took 2577 ms. Trace:
I20260709 13:08:19.464479 21066 rpcz_store.cc:276] 0709 13:08:16.846006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.846190 (+   184us) service_pool.cc:224] Handling call
0709 13:08:16.879314 (+ 33124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:17.291955 (+412641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.422197 (+2130242us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54000f9954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:19.423718 (+  1521us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":947420032}
I20260709 13:08:19.417615 26428 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.399163 26083 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:19.476073 21147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3112) took 2219 ms (client timeout 1943 ms). Trace:
W20260709 13:08:19.476511 21147 rpcz_store.cc:269] 0709 13:08:17.246562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.249348 (+  2786us) service_pool.cc:224] Handling call
0709 13:08:19.466271 (+2216923us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54002599d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:19.466372 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2297160064}
I20260709 13:08:19.460882 26026 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.417747 26596 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:08:19.418035 26465 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.483201 26622 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:19.495155 21217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:19.499151 21217 raft_consensus.cc:606] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.499648 21217 raft_consensus.cc:614] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:19.462466 26564 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:19.478499 26428 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:19.424216 26445 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:19.497957 26408 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.240s	user 0.033s	sys 0.080s
I20260709 13:08:19.443753 26483 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.444337 25762 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:19.506004 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 109647us
I20260709 13:08:19.462369 26210 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.466535 26334 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:19.512372 26408 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.255s	user 0.034s	sys 0.080s
I20260709 13:08:19.525022 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.465271 26512 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.474669 25817 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:19.529258 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 114962us
W20260709 13:08:19.528117 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 130254us
I20260709 13:08:19.500011 26026 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.506067 26564 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.535032 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:19.542513 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.548290 21063 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:19.550570 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.520272 26485 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:08:19.551182 21068 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:19.524555 26426 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:08:19.521499 26488 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:08:19.527155 26333 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:08:19.542276 26634 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:19.567705 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.567785 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.570044 21068 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:19.572546 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:19.541522 26622 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.570007 21070 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:19.575206 26562 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:19.578011 26625 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:08:19.580005 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.580726 21070 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:08:19.581202 26445 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:19.580233 25762 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:19.585857 21767 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:19.589852 21217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:19.590684 21217 raft_consensus.cc:606] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.591101 21217 raft_consensus.cc:614] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:19.593799 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 60951.3us
I20260709 13:08:19.594930 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:19.597849 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55377.5us
I20260709 13:08:19.598372 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.563380 26531 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:19.611510 21063 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:19.611963 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:19.604025 26617 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:08:19.613284 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.592803 26445 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:19.593348 26489 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:19.618283 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.606905 26467 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:19.635139 26467 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.635648 26467 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:19.637182 26644 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:19.637704 26644 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:19.638127 26644 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:19.619748 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:19.641630 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.620328 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.622121 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.644470 21068 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:19.623988 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.645485 21792 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:19.625423 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:19.633985 21217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6beb666b5d504215a141add24a6d632f"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:19.646763 21217 raft_consensus.cc:606] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.647244 21217 raft_consensus.cc:614] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:19.666913 26641 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:08:19.667102 26564 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:19.617309 26634 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:08:19.631326 26489 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:19.669855 26489 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:19.679909 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 63207.4us
W20260709 13:08:19.677608 26623 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:08:19.680773 26642 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:19.681143 26465 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:19.718019 26467 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.681150 26210 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:19.694173 26083 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:19.679952 26445 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:19.734014 21220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:19.739202 21220 raft_consensus.cc:606] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.692203 26485 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:08:19.718878 26489 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:19.744796 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.742939 26026 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:19.752424 26489 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:19.754410 26467 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:19.745122 26642 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:19.750084 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:19.756259 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.757695 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.776580 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.776654 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1725) took 1288 ms. Trace:
I20260709 13:08:19.789469 21220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:19.790307 21220 raft_consensus.cc:606] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.787760 21214 rpcz_store.cc:276] 0709 13:08:18.490406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.490525 (+   119us) service_pool.cc:224] Handling call
0709 13:08:18.518525 (+ 28000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:19.307950 (+789425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.765050 (+457100us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b540013cc54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:19.776586 (+ 11536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813945216}
I20260709 13:08:19.776110 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:19.794844 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.794186 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.795961 21066 raft_consensus.cc:2368] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15.
I20260709 13:08:19.799695 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.809353 26564 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:19.816278 21113 connection.cc:256] RPC call timeout handler was delayed by -1.0392856409999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
I20260709 13:08:19.795323 26657 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:19.819104 26565 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.826646 26657 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.826876 26565 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.829167 26565 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:19.793726 26642 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.830838 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.832222 21069 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:19.810223 26564 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.827069 26657 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:19.836650 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:19.835604 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 135172us
I20260709 13:08:19.836402 21797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53"
dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:33604
I20260709 13:08:19.834949 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2488) took 1318 ms. Trace:
I20260709 13:08:19.837682 21069 rpcz_store.cc:276] 0709 13:08:18.517043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.890608 (+373565us) service_pool.cc:224] Handling call
0709 13:08:19.831739 (+941131us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:08:19.831990 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:19.834680 (+  2690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.834892 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5951744}
I20260709 13:08:19.801759 26026 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:08:19.841675 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.841943 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 902) took 1716 ms. Trace:
I20260709 13:08:19.841401 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.842453 21072 rpcz_store.cc:276] 0709 13:08:18.125546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.129855 (+  4309us) service_pool.cc:224] Handling call
0709 13:08:18.130195 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.576548 (+446353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.841722 (+1265174us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54003ca854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:19.841889 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":909120768}
I20260709 13:08:19.849488 21185 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:19.853475 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:19.857097 26647 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:08:19.862342 26089 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.862989 26089 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:19.841400 26241 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:19.845707 26528 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:08:19.865284 26634 raft_consensus.cc:2955] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Committing config change with OpId 12.9: config changed from index 2 to 9, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:19.868793 26428 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.850847 26429 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:08:19.849516 26531 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:19.835388 26445 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:19.882565 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.883708 21069 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 13:08:19.884977 26654 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:08:19.892022 26661 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:08:19.870188 26485 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:19.845942 26651 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 13:08:19.879843 26652 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:08:19.849047 26632 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:08:19.898080 26564 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:19.912335 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:19.912596 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:19.913149 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.918768 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:19.913372 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:19.913813 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.914328 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:19.919972 21069 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:19.914532 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.875756 26644 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.926813 26241 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.927386 26241 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.927647 21220 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 9.
I20260709 13:08:19.928815 26657 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:19.935731 21214 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:19.915330 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.916388 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.921099 21066 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:19.914978 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:19.937942 21063 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:19.895057 26485 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.898958 26651 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:08:19.925379 26631 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.356s	user 0.004s	sys 0.000s
I20260709 13:08:19.948619 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.946489 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:19.954888 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:19.947913 21147 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:20.007314 21142 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:08:20.011075 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 170140us
W20260709 13:08:20.013396 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67114.2us
I20260709 13:08:20.013540 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:20.014226 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:20.016187 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:20.009842 26659 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:08:19.970703 26581 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:20.019985 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:19.990250 26651 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:20.017774 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.992274 25817 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:19.964756 26428 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:20.025179 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:19.974893 26485 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:20.033792 21146 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:08:20.016714 26644 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:20.015570 26570 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:20.018025 26562 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:20.037575 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:20.056829 26533 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:20.045111 26565 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:20.045049 26241 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:20.054716 26651 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.020900 26436 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:20.021283 26429 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:08:20.074083 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:20.078173 26483 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:20.015764 26642 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:08:20.082881 26512 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:20.083606 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:20.067016 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:20.084591 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:20.104986 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:20.105494 21140 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:20.106547 21146 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:20.040923 25817 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:20.028098 26658 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:08:20.067312 26572 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:20.076479 26089 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.045854 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:20.114945 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:20.106925 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:20.115722 21142 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:20.117188 21113 leader_election.cc:400] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15.
I20260709 13:08:20.117655 21113 leader_election.cc:403] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:20.034921 26445 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:08:20.107842 26432 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:20.118916 26656 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.134239 26656 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.112845 26241 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:20.126971 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 81228.2us
I20260709 13:08:20.106487 26651 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:20.132804 26089 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:20.125299 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:20.141726 26572 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:20.136278 26664 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15.
I20260709 13:08:20.132808 25817 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:20.169386 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:20.188349 21794 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:20.189291 26645 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:20.190953 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:20.192564 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:20.155661 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:20.195851 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:20.193183 26335 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:20.160483 26667 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:20.204054 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:20.206007 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:20.185114 26241 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:20.165308 26656 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:20.208269 21793 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:20.200409 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:20.196604 26664 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:20.220721 21141 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:20.223719 26564 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:20.224316 26564 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:20.224893 26564 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:20.152014 26485 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.214464 21139 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:20.229102 26488 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:20.229606 26488 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:20.234390 26334 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:20.214155 26658 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:20.237845 21140 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:08:20.205125 26089 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:20.238266 26502 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:20.238722 26502 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:20.241117 26485 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:20.242908 26432 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:20.212031 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 188863us
I20260709 13:08:20.245990 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:20.230485 26488 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:20.248965 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:20.252413 26658 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:20.252867 26658 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:20.253264 26658 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:20.258836 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5444608509999398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
I20260709 13:08:20.259552 26515 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:20.261289 25762 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:20.274780 25762 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:20.276023 25762 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:20.277678 25762 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:20.259719 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:20.263300 26658 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.279881 26483 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:20.280422 26483 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:20.281360 26483 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.282325 26658 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:20.282722 26483 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:20.267270 26485 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:08:20.286366 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:20.286635 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:20.269259 26657 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:20.291249 26627 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.780s	user 0.002s	sys 0.000s
I20260709 13:08:20.295850 26598 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:20.296711 26467 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:20.300808 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:20.328271 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:20.419350 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 149401us
I20260709 13:08:20.420917 26537 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.422287 26537 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:20.429337 26485 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:20.247078 26210 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:20.490967 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1735) took 1428 ms. Trace:
I20260709 13:08:20.247555 26656 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:20.430047 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 129297us
I20260709 13:08:20.500655 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:20.467689 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.468467 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.468669 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.479514 26667 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.480988 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:20.484803 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 284448us
I20260709 13:08:20.639020 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:20.434525 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:20.639863 21143 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:20.247270 26534 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:20.493388 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:20.441788 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 627) took 1382 ms. Trace:
I20260709 13:08:20.641012 21065 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:20.490383 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2479) took 2190 ms. Trace:
I20260709 13:08:20.647600 21067 rpcz_store.cc:276] 0709 13:08:18.299959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.595519 (+295560us) service_pool.cc:224] Handling call
0709 13:08:18.628481 (+ 32962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:18.823458 (+194977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.490145 (+1666687us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540041d554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:20.490327 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341622016}
I20260709 13:08:20.498394 21215 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:08:20.648963 21215 pending_rounds.cc:85] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 12
I20260709 13:08:20.649308 21215 pending_rounds.cc:107] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:08:20.649727 21215 raft_consensus.cc:2889] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:20.501343 21141 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:08:20.501873 21217 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:20.349115 26543 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:20.503015 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:20.491935 21218 rpcz_store.cc:276] 0709 13:08:19.062518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.062631 (+   113us) service_pool.cc:224] Handling call
0709 13:08:19.062922 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:19.204074 (+141152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:19.204121 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:19.204184 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:19.204235 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.490757 (+1286522us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54001caa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:20.490801 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:20.490914 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323939072}
I20260709 13:08:20.657074 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.428673 26657 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:20.665140 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.679068 21214 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:20.683645 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.213804 26241 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:20.666431 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.667207 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:20.427744 26450 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:20.672715 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.687528 21220 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:20.690778 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:20.691704 21138 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:20.640928 21798 rpcz_store.cc:276] 0709 13:08:19.059855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.060005 (+   150us) service_pool.cc:224] Handling call
0709 13:08:20.441537 (+1381532us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54003bb854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:20.441714 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557826304}
I20260709 13:08:20.439617 26026 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:20.461699 26488 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:20.459548 26515 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:20.494086 26642 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:20.706395 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:20.708614 21067 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:08:20.711269 21801 raft_consensus.cc:2955] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 2 to 9, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:20.502203 25817 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:20.768143 25495 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:20.769120 25495 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.772966 21143 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
W20260709 13:08:20.731918 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.015s	sys 0.040s
I20260709 13:08:20.501784 26224 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:20.774577 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.015s	sys 0.041s
I20260709 13:08:20.700304 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:20.775070 21796 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:20.752055 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.753404 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:20.671623 26543 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:20.761823 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:20.764878 21065 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:08:20.670480 26565 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:20.737807 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:20.774914 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 578510us
W20260709 13:08:20.727739 26567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.119s	user 0.042s	sys 0.085s
I20260709 13:08:20.689869 26450 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:20.708884 26533 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:20.825240 21216 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:08:20.773753 26334 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.058s	sys 0.060s
I20260709 13:08:20.706938 26657 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:20.636659 26210 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.671896 26656 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:20.685222 26534 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.440052 26502 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.791370 21217 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:20.684602 26667 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:20.831202 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:20.883270 21795 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:08:21.046332 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:20.869939 25495 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.023708 21216 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:20.983680 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 306437us
I20260709 13:08:21.050247 21065 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:08:21.051509 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:21.053054 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:20.884197 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:20.862481 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:20.983250 21144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.043s	sys 0.066s
W20260709 13:08:21.055861 21144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.050s	sys 0.066s
W20260709 13:08:20.838922 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 422448us
I20260709 13:08:20.883631 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:21.057999 21794 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:21.057216 25495 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:21.064491 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:20.840729 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:21.069579 21798 raft_consensus.cc:2368] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:08:21.072032 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:21.060729 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.065291 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:21.073665 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:20.880478 26562 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.785s	user 0.002s	sys 0.022s
W20260709 13:08:20.881585 25762 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.603s	user 0.016s	sys 0.000s
W20260709 13:08:20.818502 26567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.211s	user 0.043s	sys 0.086s
I20260709 13:08:21.073493 21219 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:08:21.049682 26651 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.903s	user 0.003s	sys 0.007s
W20260709 13:08:21.046447 26334 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.062s	sys 0.062s
I20260709 13:08:21.094228 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:20.803639 26543 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:21.109300 21795 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:20.990648 26436 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.999218 26537 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:21.105774 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 702) took 1192 ms. Trace:
I20260709 13:08:20.877135 26687 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:21.119566 21215 rpcz_store.cc:276] 0709 13:08:19.912946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.913842 (+   896us) service_pool.cc:224] Handling call
0709 13:08:19.915636 (+  1794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.650300 (+734664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.105496 (+455196us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54002e5254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:21.105701 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1323585920}
I20260709 13:08:20.877451 26694 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:21.057794 26657 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:20.879055 26241 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:21.049043 26512 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:20.831975 26533 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.885085 26693 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.116295 21797 raft_consensus.cc:606] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:21.116894 21221 raft_consensus.cc:2410] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:08:21.120250 21218 raft_consensus.cc:2410] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:08:20.821673 26594 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:20.798058 25817 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.080183 26485 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.779093 26642 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.148752 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:21.106491 26562 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:20.819530 26224 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:20.844736 26565 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:21.188022 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 481706us
W20260709 13:08:21.213414 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 158807us
I20260709 13:08:21.241290 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:08:21.204348 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 55690.7us
I20260709 13:08:21.124679 26502 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:21.129550 26568 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.259850 21184 leader_election.cc:400] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:08:21.260666 21184 leader_election.cc:403] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:08:20.879841 26450 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.174995 26533 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:21.126376 26436 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:21.314643 26520 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.010s	sys 0.064s
I20260709 13:08:21.233386 26690 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:21.259809 26537 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:21.343652 25495 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:21.168434 26693 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.355592 25495 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:21.356341 25495 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.356774 25495 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:21.148816 26687 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.233057 26485 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.348275 26690 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:21.378047 26651 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:21.385311 26210 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.263330 26568 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:21.386317 26695 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 13:08:21.433746 21794 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
W20260709 13:08:21.387912 26675 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
W20260709 13:08:21.323496 26520 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.010s	sys 0.064s
I20260709 13:08:21.162497 26694 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.413173 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2772) took 1288 ms. Trace:
I20260709 13:08:21.419663 26229 raft_consensus.cc:993] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
W20260709 13:08:21.385572 26564 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.160s	user 0.014s	sys 0.000s
I20260709 13:08:21.201195 26594 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:21.394277 26672 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.529s	user 0.001s	sys 0.000s
I20260709 13:08:21.435348 21795 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:21.441751 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:21.451099 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.445052 21146 rpcz_store.cc:276] 0709 13:08:20.124441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.139494 (+ 15053us) service_pool.cc:224] Handling call
0709 13:08:20.139964 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.140439 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:20.236832 (+ 96393us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b48001a01d8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:08:20.244771 (+  7939us) log.cc:844] Serialized 14 byte log entry
0709 13:08:20.832038 (+587267us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:20.832566 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:08:20.832738 (+   172us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:20.832785 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.832884 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:21.395619 (+562735us) raft_consensus.cc:1685] finished
0709 13:08:21.412918 (+ 17299us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.413092 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14602,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":1193,"raft.queue_time_us":29071,"raft.run_cpu_time_us":2194,"raft.run_wall_time_us":110003,"spinlock_wait_cycles":4565248,"thread_start_us":459708,"threads_started":2,"wal-append.queue_time_us":966497}
I20260709 13:08:21.451912 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:21.420023 26690 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:21.448370 21799 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.672s	user 0.010s	sys 0.001s
I20260709 13:08:21.478735 21799 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:21.479935 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:21.455433 26694 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.443334 26210 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:08:21.455482 26483 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.172s	user 0.016s	sys 0.000s
I20260709 13:08:21.471987 26656 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:21.498294 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.007s	sys 0.031s
W20260709 13:08:21.500548 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.007s	sys 0.031s
W20260709 13:08:21.499487 21792 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.022s	sys 0.009s
W20260709 13:08:21.504890 21792 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.022s	sys 0.009s
W20260709 13:08:21.506242 21796 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.000s
I20260709 13:08:21.508320 21796 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:21.509150 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1093) took 1007 ms. Trace:
I20260709 13:08:21.509615 21796 rpcz_store.cc:276] 0709 13:08:20.501728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.502058 (+   330us) service_pool.cc:224] Handling call
0709 13:08:21.509075 (+1007017us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:21.361881 26537 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.350714 26691 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.115s
I20260709 13:08:21.484916 26644 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:21.518164 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:21.525065 21220 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
W20260709 13:08:21.537874 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.509626 26689 raft_consensus.cc:1081] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:21.518477 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:21.559713 21038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.017s	sys 0.012s
W20260709 13:08:21.560498 21038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.017s	sys 0.012s
I20260709 13:08:21.515319 26694 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:21.567076 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.012s	sys 0.038s
I20260709 13:08:21.525614 26664 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:21.509392 26697 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:21.577013 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1092) took 1074 ms. Trace:
I20260709 13:08:21.525871 26210 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:21.573771 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.013s	sys 0.038s
W20260709 13:08:21.575412 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57031.3us
I20260709 13:08:21.573876 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2780) took 1414 ms. Trace:
I20260709 13:08:21.531602 26534 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.486477 26564 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.577452 21793 rpcz_store.cc:276] 0709 13:08:20.501095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.664252 (+163157us) service_pool.cc:224] Handling call
0709 13:08:20.710959 (+ 46707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.711160 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:21.229481 (+518321us) log.cc:844] Serialized 14 byte log entry
0709 13:08:21.563218 (+333737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:21.563397 (+   179us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:21.563512 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.563646 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.563828 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15853,"raft.queue_time_us":19852,"spinlock_wait_cycles":36352,"thread_start_us":726153,"threads_started":2}
I20260709 13:08:21.546339 25762 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.530961 26699 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:21.542219 26667 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.583194 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:21.584267 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 792439us
W20260709 13:08:21.585225 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 480371us
I20260709 13:08:21.554973 26656 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:21.582342 21139 rpcz_store.cc:276] 0709 13:08:20.159123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.244992 (+ 85869us) service_pool.cc:224] Handling call
0709 13:08:20.245317 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.416326 (+171009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:20.416360 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:20.432714 (+ 16354us) log.cc:844] Serialized 26 byte log entry
0709 13:08:20.482893 (+ 50179us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:20.482936 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.483005 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:21.573608 (+1090603us) raft_consensus.cc:1685] finished
0709 13:08:21.573636 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.573804 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":174170,"raft.queue_time_us":170293,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":4141,"thread_start_us":44323,"threads_started":1,"wal-append.queue_time_us":625369}
W20260709 13:08:21.588744 21187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.043s	sys 0.041s
I20260709 13:08:21.566802 26644 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:21.596213 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:21.596441 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2762) took 1514 ms. Trace:
I20260709 13:08:21.571542 26690 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:21.611234 21142 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:21.606810 25495 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.611227 26483 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:21.605644 21147 rpcz_store.cc:276] 0709 13:08:20.082194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.082327 (+   133us) service_pool.cc:224] Handling call
0709 13:08:20.082730 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.083165 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:20.083202 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:20.088923 (+  5721us) log.cc:844] Serialized 26 byte log entry
0709 13:08:20.209473 (+120550us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:20.209515 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.209590 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:21.596220 (+1386630us) raft_consensus.cc:1685] finished
0709 13:08:21.596242 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.596377 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1513,"raft.queue_time_us":549,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":5623,"spinlock_wait_cycles":1919360,"thread_start_us":105796,"threads_started":1,"wal-append.queue_time_us":603277}
I20260709 13:08:21.608520 26241 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:21.619872 26483 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.620337 26483 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:21.588186 26697 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.579598 26694 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:21.622885 26701 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:08:21.623148 26694 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.625929 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2773) took 1486 ms. Trace:
I20260709 13:08:21.585309 26664 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.626339 21144 rpcz_store.cc:276] 0709 13:08:20.139261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.139817 (+   556us) service_pool.cc:224] Handling call
0709 13:08:20.140224 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:20.140685 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:20.140738 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:20.141323 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:08:20.288858 (+147535us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b18003d7020. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:21.056420 (+767562us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:21.056462 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.056530 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:21.625598 (+569068us) raft_consensus.cc:1685] finished
0709 13:08:21.625620 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.625848 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4144,"spinlock_wait_cycles":8000384,"thread_start_us":767284,"threads_started":1,"wal-append.queue_time_us":1482384}
I20260709 13:08:21.629087 26664 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:21.629194 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1744) took 1137 ms. Trace:
I20260709 13:08:21.629839 26483 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.630303 26537 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.629654 21218 rpcz_store.cc:276] 0709 13:08:20.492046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.656846 (+164800us) service_pool.cc:224] Handling call
0709 13:08:21.628968 (+972122us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400102554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:21.629133 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062813952}
I20260709 13:08:21.594298 25762 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.608745 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:21.591003 26488 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.837s	user 0.012s	sys 0.000s
I20260709 13:08:21.617046 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:21.612880 26515 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.845s	user 0.005s	sys 0.007s
W20260709 13:08:21.601445 21187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.043s	sys 0.041s
I20260709 13:08:21.609805 25817 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:21.640609 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 399381us
I20260709 13:08:21.612298 26667 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:21.654786 26664 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:08:21.655715 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 47042us
I20260709 13:08:21.606688 21185 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:21.677238 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 70631.1us
I20260709 13:08:21.639158 26656 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:21.668565 26658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.000s	sys 0.022s
I20260709 13:08:21.616572 26565 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.607158 26562 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:08:21.615794 26642 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:21.678345 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.662066 21144 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:08:21.664099 21036 leader_election.cc:400] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:08:21.669109 26664 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.719503 26664 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:21.719647 26656 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.665413 26667 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:21.685652 26333 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:21.605227 26699 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.723533 26667 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:21.723690 26333 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.725270 26699 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:21.614125 26685 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 13:08:21.726059 26333 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:21.661868 26515 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:21.726650 26333 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:21.727459 26333 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:21.728539 26642 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.728428 26333 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.729071 26642 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.660934 26241 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:21.613740 26682 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:08:21.731170 26562 consensus_queue.cc:1243] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:08:21.656108 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 1.15542e+06us
I20260709 13:08:21.732854 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:21.734803 26642 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:21.735891 26642 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:21.738322 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:21.738993 21065 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:21.740962 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:21.741688 21068 raft_consensus.cc:2368] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
I20260709 13:08:21.742219 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:21.742450 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:21.743034 21216 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:21.743738 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1746) took 1058 ms. Trace:
I20260709 13:08:21.744463 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:21.745290 21069 raft_consensus.cc:2410] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:08:21.745622 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:21.746325 21216 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:21.746398 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:21.701118 26644 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.712538 26568 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.656257 26520 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 13:08:21.710479 26658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.000s	sys 0.022s
I20260709 13:08:21.742458 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:21.770179 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:21.746938 21072 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:08:21.709782 21036 leader_election.cc:403] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:08:21.751832 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:21.772087 21067 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:21.752779 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:21.614236 26690 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.670615 26657 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:21.757376 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 77434.3us
I20260709 13:08:21.776680 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:21.758371 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:21.759752 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:21.761725 26483 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:21.768133 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:21.773278 21068 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:21.700807 26693 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.775601 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:21.777135 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:21.787578 21217 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:21.780877 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:21.767376 26669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.013s	sys 0.021s
I20260709 13:08:21.787959 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:21.789665 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 125773us
I20260709 13:08:21.789733 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:21.778666 21069 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:21.784435 21068 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:21.793105 21219 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:21.793642 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:21.794656 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:21.744227 21212 rpcz_store.cc:276] 0709 13:08:20.685765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.713158 (+ 27393us) service_pool.cc:224] Handling call
0709 13:08:20.805365 (+ 92207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:20.805659 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:21.280346 (+474687us) log.cc:844] Serialized 14 byte log entry
0709 13:08:21.743306 (+462960us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:21.743374 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:21.743445 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.743532 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:21.743677 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65836,"prepare.run_cpu_time_us":4095,"prepare.run_wall_time_us":82572,"thread_start_us":886020,"threads_started":2}
I20260709 13:08:21.806641 21072 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:21.808892 21213 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:21.777165 26687 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.781087 26712 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:21.820708 21217 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:08:21.678946 26210 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.766582 26026 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:08:21.780236 26465 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:21.793270 26669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.013s	sys 0.022s
I20260709 13:08:21.810081 26565 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:21.831362 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:21.825076 21796 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:21.833000 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:21.839633 21219 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:21.794211 25817 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:21.847951 25817 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:21.848549 26720 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.849176 25817 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.780608 26594 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.817153 26708 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:08:21.780572 26485 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:21.841420 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:21.852600 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 370377us
W20260709 13:08:21.857017 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.800222 26699 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:08:21.873868 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.234s	user 0.014s	sys 0.037s
W20260709 13:08:21.987798 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.348s	user 0.015s	sys 0.037s
W20260709 13:08:21.988363 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 1.34929e+06us
I20260709 13:08:21.989346 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:21.991959 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:21.948726 25495 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:21.944736 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1742) took 1475 ms. Trace:
I20260709 13:08:21.893188 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1747) took 1192 ms. Trace:
I20260709 13:08:21.968582 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:21.994338 21800 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:21.921022 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33604 (request call id 10) took 2102 ms. Trace:
I20260709 13:08:21.995568 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:21.884990 21221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1741) took 1416 ms (client timeout 1609 ms). Trace:
W20260709 13:08:21.996654 21221 rpcz_store.cc:269] 0709 13:08:20.468825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.469269 (+   444us) service_pool.cc:224] Handling call
0709 13:08:21.884629 (+1415360us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b540034cbd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:21.884856 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987931520}
W20260709 13:08:21.918314 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.019s	sys 0.044s
W20260709 13:08:21.997920 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.019s	sys 0.045s
W20260709 13:08:21.845008 26709 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:08:21.847783 21067 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:21.995060 21797 rpcz_store.cc:276] 0709 13:08:19.818962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.830110 (+ 11148us) service_pool.cc:224] Handling call
0709 13:08:21.920776 (+2090666us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400494b54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42516853 00007f7f474e460a 00007f7f40a6a7d4 00007f7f40a6a762 00007f7f40a6a691 00007f7f40a6a60c 00007f7f40a698c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:21.920894 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2940547840}
I20260709 13:08:21.974642 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:21.801368 26520 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:08:21.800395 26502 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:08:21.886924 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:22.028635 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.032029 21069 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:22.033167 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:22.034795 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45570.3us
I20260709 13:08:22.036448 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.037488 21143 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:22.037503 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.039505 21147 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:22.040596 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.041747 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.042618 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:22.043861 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:22.045089 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:21.812218 26483 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:08:22.045912 21215 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:22.046037 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:22.046578 21143 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:22.047436 21212 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:22.048514 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:22.053470 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 78965.9us
W20260709 13:08:21.872582 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 167612us
I20260709 13:08:22.005337 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:21.993479 21213 rpcz_store.cc:276] 0709 13:08:20.469088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.469192 (+   104us) service_pool.cc:224] Handling call
0709 13:08:21.944554 (+1475362us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400102a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:21.944674 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2396403968}
I20260709 13:08:21.995718 21220 rpcz_store.cc:276] 0709 13:08:20.713691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.751743 (+ 38052us) service_pool.cc:224] Handling call
0709 13:08:21.892946 (+1141203us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b54004349d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:21.893105 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722628864}
W20260709 13:08:21.998430 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 924031us
I20260709 13:08:21.943382 25762 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:22.007575 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:21.919431 26525 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:21.876474 26564 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.925457 26394 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:22.056118 21800 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
W20260709 13:08:21.889660 26676 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.430s	user 0.004s	sys 0.000s
I20260709 13:08:22.038362 21144 pending_rounds.cc:85] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 8
I20260709 13:08:22.058060 21767 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:22.077800 21144 pending_rounds.cc:107] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:08:22.078119 21144 raft_consensus.cc:2889] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:21.939339 26682 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:21.987546 26699 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.874936 25817 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:22.058051 21141 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:22.059717 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:22.131119 21798 raft_consensus.cc:2376] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18. Re-sending same reply.
I20260709 13:08:22.059374 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.132506 21146 raft_consensus.cc:2368] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
W20260709 13:08:22.004099 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4441413020000482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
I20260709 13:08:22.075832 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:22.134831 25495 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:22.135459 25495 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:21.844875 26465 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:21.904629 26450 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:21.907274 26224 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.146862 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:21.930548 26697 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:21.917829 26436 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:22.078917 21139 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:22.155567 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.093101 26681 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:08:21.830240 26712 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.093071 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:22.094425 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.164876 25495 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.116525 21187 leader_election.cc:400] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 9 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
W20260709 13:08:22.167135 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 75324us
I20260709 13:08:22.118474 21069 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:22.120627 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:22.119870 21143 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:22.169798 21794 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:22.076723 26658 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:22.178026 26658 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.178458 26658 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:22.179962 26719 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:08:22.056833 26718 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:22.092671 26713 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:08:22.191001 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2787) took 1884 ms. Trace:
I20260709 13:08:21.849223 26720 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.095175 26699 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:22.031042 26687 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:22.056679 26565 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:08:22.163784 26681 consensus_queue.cc:1243] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:22.094358 26602 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:22.066354 26467 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:08:22.148985 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.143007 26155 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:22.152961 21141 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:22.312734 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:22.313195 21071 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.037s	sys 0.070s
W20260709 13:08:22.313544 21071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.037s	sys 0.070s
I20260709 13:08:22.156699 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.009203 26428 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:22.076884 26525 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:22.154524 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 94797.3us
I20260709 13:08:22.117468 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:22.174185 21792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1020) took 1537 ms. Trace:
I20260709 13:08:22.176142 21070 raft_consensus.cc:2410] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:08:22.176708 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.177500 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.119249 26682 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.191428 21140 rpcz_store.cc:276] 0709 13:08:20.306171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.638911 (+332740us) service_pool.cc:224] Handling call
0709 13:08:20.639247 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:21.039573 (+400326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.039616 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:21.067716 (+ 28100us) log.cc:844] Serialized 52 byte log entry
0709 13:08:21.610611 (+542895us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:21.618854 (+  8243us) log.cc:844] Serialized 14 byte log entry
0709 13:08:21.619068 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.619172 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:22.190773 (+571601us) raft_consensus.cc:1685] finished
0709 13:08:22.190792 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.190942 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8018,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":36,"raft.queue_time_us":67809,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":38662,"thread_start_us":885831,"threads_started":2,"wal-append.queue_time_us":696705}
I20260709 13:08:22.151459 26711 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:22.273810 26690 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:22.138852 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:22.308524 21146 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:22.156368 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 734) took 2224 ms. Trace:
I20260709 13:08:22.200181 26657 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:22.321131 21141 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:08:22.329407 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:22.328099 21792 rpcz_store.cc:276] 0709 13:08:20.636690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.636861 (+   171us) service_pool.cc:224] Handling call
0709 13:08:20.637246 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:21.510768 (+873522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.510813 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:21.538141 (+ 27328us) log.cc:844] Serialized 52 byte log entry
0709 13:08:21.873463 (+335322us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:21.873513 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:21.873608 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:22.173894 (+300286us) raft_consensus.cc:1685] finished
0709 13:08:22.173923 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.174109 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":244439,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":74,"thread_start_us":1157543,"threads_started":2,"wal-append.queue_time_us":597018}
I20260709 13:08:22.222432 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:22.455469 21072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.004s	sys 0.023s
I20260709 13:08:22.361198 26602 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:22.480852 21145 rpcz_store.cc:276] 0709 13:08:19.926641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.933048 (+  6407us) service_pool.cc:224] Handling call
0709 13:08:19.933463 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:19.933856 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:19.933896 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:19.971458 (+ 37562us) log.cc:844] Serialized 26 byte log entry
0709 13:08:20.210580 (+239122us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:20.210621 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.210689 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:21.888933 (+1678244us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:22.143260 (+254327us) raft_consensus.cc:1685] finished
0709 13:08:22.143358 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.151151 (+  7793us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31052,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":37221,"spinlock_wait_cycles":21092096,"thread_start_us":189312,"threads_started":1,"wal-append.queue_time_us":231102}
W20260709 13:08:22.482378 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 154679us
W20260709 13:08:22.486019 21072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.004s	sys 0.023s
W20260709 13:08:22.491503 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 269143us
I20260709 13:08:22.493001 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:22.340131 26428 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.311203 26687 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:22.494153 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 360704us
I20260709 13:08:22.495545 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:22.497761 21138 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:22.498608 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 359839us
I20260709 13:08:22.323418 26155 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.393074 26711 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:22.500066 26543 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.371s	user 0.009s	sys 0.005s
I20260709 13:08:22.524395 26525 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.524714 26687 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:22.527369 26155 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.551661 26658 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.554292 26658 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:22.555117 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46231.5us
W20260709 13:08:22.564254 26664 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.844s	user 0.010s	sys 0.000s
I20260709 13:08:22.564872 26664 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.566792 26488 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.570637 26488 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.573304 26488 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:22.574779 26642 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.838s	user 0.006s	sys 0.003s
I20260709 13:08:22.536595 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:22.489806 26333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.005s	sys 0.008s
W20260709 13:08:22.541128 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.577677 26712 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.578188 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:22.533761 26361 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:22.542454 26515 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:22.544512 26682 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.544399 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:22.581077 26515 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:22.581440 26515 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:22.581830 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:22.582261 26515 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:22.546912 26667 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.823s	user 0.008s	sys 0.000s
I20260709 13:08:22.584326 21220 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:22.584223 26667 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.584764 26515 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:22.537933 25762 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:08:22.586606 26667 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:22.515532 26726 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:22.587891 21213 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:22.550186 26711 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:22.551447 26708 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.569948 26436 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:22.598812 21110 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.026s	sys 0.009s
I20260709 13:08:22.587054 25762 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:22.584367 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.507965 26645 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:22.589624 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:22.520736 26651 consensus_meta.cc:287] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.100s	user 0.009s	sys 0.004s
W20260709 13:08:22.529690 26533 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.208s	user 0.010s	sys 0.001s
W20260709 13:08:22.593904 21765 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.016s	sys 0.010s
I20260709 13:08:22.510700 26634 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:22.597662 21111 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.016s	sys 0.052s
W20260709 13:08:22.610585 21765 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.017s	sys 0.010s
I20260709 13:08:22.575500 26602 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.600644 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.597958 26642 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:22.602356 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:22.594389 26333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.005s	sys 0.009s
I20260709 13:08:22.538455 26543 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.604084 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.624769 21138 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
W20260709 13:08:22.610805 21111 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.017s	sys 0.052s
I20260709 13:08:22.611756 26651 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 9. Current term is 10.
W20260709 13:08:22.626214 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 663719us
I20260709 13:08:22.626791 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.609119 26664 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:22.598555 26361 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:08:22.601328 21110 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.026s	sys 0.009s
I20260709 13:08:22.610915 25762 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:22.617493 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:22.603349 21065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.011s	sys 0.019s
W20260709 13:08:22.641831 21065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.011s	sys 0.019s
W20260709 13:08:22.611012 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 618228us
I20260709 13:08:22.631577 21219 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
W20260709 13:08:22.642585 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.643069 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.643700 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:22.643842 21141 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:22.644296 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:22.603523 26436 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.614269 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.638772 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.646525 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.600137 26708 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.611927 26645 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:22.626863 26687 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.633090 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.647210 21146 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:22.647397 26682 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.657791 26361 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:22.668263 21070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1668) took 2838 ms (client timeout 2769 ms). Trace:
I20260709 13:08:22.668506 26682 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.668344 26721 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:22.669003 26361 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.670729 26682 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:22.671074 26716 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.943s	user 0.007s	sys 0.002s
I20260709 13:08:22.648492 21212 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 11: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:22.650121 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.679062 21140 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:22.651070 21215 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:22.639926 26667 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:22.656136 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.656194 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:22.640839 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:22.688828 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 48072.1us
I20260709 13:08:22.627861 26720 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:22.660092 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:22.647826 21214 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:22.614015 26533 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.630030 26465 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.661495 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.664219 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:22.666649 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:22.693411 21067 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:22.693895 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:22.694217 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:22.694530 21069 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
I20260709 13:08:22.694756 21067 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:22.695395 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:22.667392 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:22.668777 21070 rpcz_store.cc:269] 0709 13:08:19.830067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.830477 (+   410us) service_pool.cc:224] Handling call
0709 13:08:22.660922 (+2830445us) spinlock_profiling.cc:243] Waited 2.34 s on lock 0x7b54003bef54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:22.668165 (+  7243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5386239744}
I20260709 13:08:22.697677 21068 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:22.648129 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:22.678910 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.653389 21147 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:08:22.680614 21063 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:08:22.701337 21063 pending_rounds.cc:85] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 7
I20260709 13:08:22.701643 21063 pending_rounds.cc:107] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:08:22.701929 21063 raft_consensus.cc:2889] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:22.628010 26706 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.016s	sys 0.025s
I20260709 13:08:22.647606 26712 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:22.683250 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.685878 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:22.650661 26664 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:22.709213 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:22.723870 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:22.654966 26568 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:22.718672 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:22.722219 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:22.726569 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.736037 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.736183 26645 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.736893 21215 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:22.737145 26564 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:22.737617 26564 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:22.739274 26525 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:22.739926 26525 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:22.740998 26525 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.744177 26565 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:08:22.744724 26727 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.219s	user 0.000s	sys 0.003s
I20260709 13:08:22.745302 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:22.745859 26728 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.223s	user 0.000s	sys 0.002s
I20260709 13:08:22.746268 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:22.747048 26512 raft_consensus.cc:993] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:22.747646 26725 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:22.748052 26725 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.738584 26564 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.748986 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:22.749281 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1100) took 1110 ms. Trace:
I20260709 13:08:22.660701 26707 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:08:22.713363 26718 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:22.746381 26445 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:22.736058 26740 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:08:22.754551 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.758271 21141 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:22.713665 26706 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.016s	sys 0.026s
I20260709 13:08:22.757067 21795 rpcz_store.cc:276] 0709 13:08:21.638782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.638979 (+   197us) service_pool.cc:224] Handling call
0709 13:08:21.665548 (+ 26569us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:21.671783 (+  6235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.671831 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:21.695574 (+ 23743us) log.cc:844] Serialized 52 byte log entry
0709 13:08:22.217956 (+522382us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:22.218004 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.218093 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:22.744118 (+526025us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:22.748991 (+  4873us) raft_consensus.cc:1685] finished
0709 13:08:22.749014 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.749225 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79838,"raft.queue_time_us":11131,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":98,"spinlock_wait_cycles":12092800,"thread_start_us":499168,"threads_started":1,"wal-append.queue_time_us":814507}
I20260709 13:08:22.758670 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.758787 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.806341 21220 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:22.807010 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.808179 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.754094 26725 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:22.799830 26429 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:22.764098 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2512) took 1026 ms. Trace:
I20260709 13:08:22.810464 21220 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:08:22.811806 21220 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:08:22.810714 21071 rpcz_store.cc:276] 0709 13:08:21.737387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.737516 (+   129us) service_pool.cc:224] Handling call
0709 13:08:21.737901 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:21.738267 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.738296 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:21.738811 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.313905 (+575094us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:22.313953 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.314018 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:22.745302 (+431284us) spinlock_profiling.cc:243] Waited 7.59 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:22.748471 (+  3169us) raft_consensus.cc:1685] finished
0709 13:08:22.748496 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.764031 (+ 15535us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1138,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":17454976,"thread_start_us":574714,"threads_started":1,"wal-append.queue_time_us":783541}
I20260709 13:08:22.752449 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.768414 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:22.679468 26687 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.748291 26708 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.759526 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.827499 21067 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:22.757365 26334 raft_consensus.cc:1081] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
W20260709 13:08:22.815982 26731 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:08:22.818526 21219 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:22.764333 26720 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:22.836998 26720 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.819139 21215 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
W20260709 13:08:22.838584 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:22.820097 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 51767.7us
I20260709 13:08:22.822688 21068 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:22.745814 26664 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:22.712307 26667 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.723989 26502 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:08:22.834152 26708 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:22.840533 21064 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.025s	sys 0.041s
I20260709 13:08:22.729440 26712 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:22.829398 26733 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:08:22.818521 26564 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:22.800108 26682 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:08:22.850765 26712 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.851241 21138 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:22.853938 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:22.845919 21064 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.026s	sys 0.041s
I20260709 13:08:22.855412 26708 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.856736 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.858192 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:22.861136 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.910243 21141 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:22.911159 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.911510 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.861588 26642 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.862032 26735 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.862726 26515 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.864019 26725 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.865489 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:22.802493 26520 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:22.830225 26718 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:22.831224 26428 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:22.841051 26720 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:22.973609 21219 raft_consensus.cc:2368] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13.
I20260709 13:08:22.911782 21141 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
W20260709 13:08:22.928421 25817 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.783s	user 0.012s	sys 0.001s
I20260709 13:08:22.912735 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.978144 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:22.924852 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 59316.3us
I20260709 13:08:22.925575 26580 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:22.804908 25762 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:22.908255 26656 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:22.968286 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:22.912518 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:22.992154 21071 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:22.993714 26725 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:22.975217 26520 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:22.974749 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.996227 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.996387 26520 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:22.996850 21141 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:22.997936 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.998790 21141 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:22.981408 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.966210 26718 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:22.975908 26746 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:08:23.050487 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:23.002255 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:22.926541 26515 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:23.056313 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 54180.2us
I20260709 13:08:22.974370 26543 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:23.042001 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:22.984198 26580 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:23.054845 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:22.927572 26642 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:23.063252 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:23.059307 21218 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:23.060413 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:23.057804 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:22.999526 26436 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:22.943567 26735 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:23.074863 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1752) took 2236 ms. Trace:
I20260709 13:08:22.990108 26739 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:23.083499 21214 rpcz_store.cc:276] 0709 13:08:20.827819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.827947 (+   128us) service_pool.cc:224] Handling call
0709 13:08:20.828266 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.692801 (+1864535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.063450 (+370649us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54002ee0d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:23.063686 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4184155008}
I20260709 13:08:23.056890 26502 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:08:23.088824 26502 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:23.088917 26512 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:23.089239 26502 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:23.089789 26512 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:23.090091 26502 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:23.094012 26732 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.449s	user 0.000s	sys 0.002s
I20260709 13:08:23.052784 26718 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:23.058053 26565 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:23.077281 26741 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
W20260709 13:08:23.100441 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 442357us
I20260709 13:08:23.066525 26725 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:08:23.106491 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.071732 26580 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:23.020020 26656 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:23.084874 26533 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:23.102962 26502 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:23.124137 26738 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:08:23.118971 26725 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:23.140950 26699 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.851s	user 0.012s	sys 0.000s
I20260709 13:08:23.141547 26699 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:23.113839 26686 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 13:08:23.151542 26744 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:08:23.155632 26737 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:23.155965 26735 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:23.156111 26737 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:23.188511 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:23.199043 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2519) took 1450 ms. Trace:
I20260709 13:08:23.199726 21063 rpcz_store.cc:276] 0709 13:08:21.748801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.811646 (+ 62845us) service_pool.cc:224] Handling call
0709 13:08:22.661415 (+849769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.702461 (+ 41046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.195947 (+493486us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b54003bef54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:23.198502 (+  2555us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b54003bef50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:23.198805 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948675840}
I20260709 13:08:23.202109 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.203430 21063 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:23.205401 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:23.207685 21066 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.008s	sys 0.008s
I20260709 13:08:23.208104 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2517) took 1459 ms. Trace:
I20260709 13:08:23.208766 21065 rpcz_store.cc:276] 0709 13:08:21.748143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.753317 (+  5174us) service_pool.cc:224] Handling call
0709 13:08:21.753694 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:21.765033 (+ 11339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.765076 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:21.765630 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.642186 (+876556us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:22.642240 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.642304 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.207542 (+565238us) raft_consensus.cc:1685] finished
0709 13:08:23.207590 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.207863 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20374,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":10618,"thread_start_us":876238,"threads_started":1,"wal-append.queue_time_us":1111802}
I20260709 13:08:23.210915 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:23.211572 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.211827 21067 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:23.212580 21065 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:23.214131 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.214927 21065 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:23.216584 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.217412 21065 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:23.017884 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.220363 21068 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:23.220993 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.221527 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.221833 21065 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:23.223073 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.224052 21065 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:23.223037 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.225255 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.225720 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:23.226624 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.227023 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.235018 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.027034 21071 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:23.243362 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2520) took 1494 ms. Trace:
I20260709 13:08:23.243983 21072 rpcz_store.cc:276] 0709 13:08:21.749087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.829125 (+ 80038us) service_pool.cc:224] Handling call
0709 13:08:21.829442 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:21.829829 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.829860 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:21.917958 (+ 88098us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b2c000014c8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:08:21.918483 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.489511 (+571028us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:22.489567 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.489657 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.093350 (+603693us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:23.242813 (+149463us) raft_consensus.cc:1685] finished
0709 13:08:23.242863 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.243137 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":9020,"spinlock_wait_cycles":31377408,"thread_start_us":570606,"threads_started":1,"wal-append.queue_time_us":701403}
I20260709 13:08:23.247637 21072 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 16. Current term is 17. Ops: []
W20260709 13:08:23.256655 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.259207 21111 leader_election.cc:400] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13.
I20260709 13:08:23.259712 21111 leader_election.cc:403] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:23.261605 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 221357us
W20260709 13:08:23.041059 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 72884.9us
I20260709 13:08:23.268714 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:23.270114 21138 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:23.272071 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:23.273837 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:23.274350 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:23.276265 21138 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:08:23.284549 21144 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.030s	sys 0.018s
W20260709 13:08:23.326647 21144 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.030s	sys 0.018s
W20260709 13:08:23.284535 21140 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.013s	sys 0.024s
I20260709 13:08:23.286569 21141 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:23.286769 21143 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.008s	sys 0.012s
W20260709 13:08:23.327673 21140 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.014s	sys 0.024s
W20260709 13:08:23.327837 21143 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.008s	sys 0.012s
W20260709 13:08:23.287256 21145 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.000s	sys 0.019s
W20260709 13:08:23.329113 21145 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.001s	sys 0.019s
W20260709 13:08:23.287809 21146 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.011s	sys 0.009s
W20260709 13:08:23.330246 21146 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.011s	sys 0.009s
W20260709 13:08:23.288326 21147 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.000s	sys 0.019s
W20260709 13:08:23.331178 21147 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.000s	sys 0.020s
I20260709 13:08:23.243399 21068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39498
I20260709 13:08:23.082728 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:23.306412 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:23.308507 21212 raft_consensus.cc:2410] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:08:23.313284 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:23.315101 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:23.320094 21217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.003s	sys 0.019s
W20260709 13:08:23.344851 21217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.003s	sys 0.019s
I20260709 13:08:23.321513 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:23.325001 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2955) took 1584 ms. Trace:
I20260709 13:08:23.347509 21218 rpcz_store.cc:276] 0709 13:08:21.740336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.740664 (+   328us) service_pool.cc:224] Handling call
0709 13:08:21.740913 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:23.065827 (+1324914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.323378 (+257551us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b54002c04a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42493dc0 00007f7f4251fa02 00007f7f4251bdd6 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:08:23.324500 (+  1122us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540015d454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:23.324810 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3043100416}
I20260709 13:08:23.363747 26155 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:23.364120 26333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.004s	sys 0.033s
W20260709 13:08:23.364797 26333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.004s	sys 0.033s
I20260709 13:08:23.364898 26155 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:23.367606 26361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.013s	sys 0.024s
I20260709 13:08:23.367481 25817 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:23.207005 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.395449 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:23.091012 26512 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:23.397585 26656 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:23.389415 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 645) took 1374 ms. Trace:
I20260709 13:08:23.399662 21795 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:23.400076 21801 rpcz_store.cc:276] 0709 13:08:21.987271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.992209 (+  4938us) service_pool.cc:224] Handling call
0709 13:08:22.046329 (+ 54120us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.046743 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.046784 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.058693 (+ 11909us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.492637 (+433944us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:22.493057 (+   420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.493141 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.360731 (+867590us) raft_consensus.cc:1685] finished
0709 13:08:23.360786 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.361126 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11025,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5177,"thread_start_us":367661,"threads_started":1,"wal-append.queue_time_us":551707}
W20260709 13:08:23.394933 26528 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.482s	user 0.040s	sys 0.029s
I20260709 13:08:23.099102 26739 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:23.402966 26528 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.490s	user 0.041s	sys 0.029s
I20260709 13:08:23.132787 26667 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:23.139142 26708 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:23.414407 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 45653.3us
I20260709 13:08:23.415956 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:23.418896 26565 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:23.420159 26361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.014s	sys 0.025s
I20260709 13:08:23.423138 26520 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.424275 26725 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:23.425264 26760 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.438319 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:23.441212 26690 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:23.442346 26525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.010s	sys 0.027s
W20260709 13:08:23.442772 26525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.010s	sys 0.027s
I20260709 13:08:23.136740 26687 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:23.099994 26483 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:08:23.445510 26664 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:08:23.446724 26699 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:23.445510 26361 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:08:23.450738 21795 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:08:23.453217 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:23.456415 21068 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:23.457024 21068 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:23.426224 26760 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:23.463996 26515 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:23.465736 26726 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:23.492542 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:23.467159 26429 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:23.512272 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:23.613974 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 340028us
W20260709 13:08:23.617389 26210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.000s	sys 0.018s
W20260709 13:08:23.618358 26210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.000s	sys 0.019s
W20260709 13:08:23.625888 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 308639us
W20260709 13:08:23.629277 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 131486us
W20260709 13:08:23.627221 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 321165us
I20260709 13:08:23.674741 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:23.682770 21801 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:23.696415 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:23.711241 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:23.602403 25817 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:23.699111 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 63400.6us
I20260709 13:08:23.717787 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:23.698319 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:23.694339 26520 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:23.394933 26737 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:23.733039 26699 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:23.733935 26699 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:23.734321 26520 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:23.735216 26644 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:23.737102 21063 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:23.736065 21218 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:23.760195 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.779584 21065 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:23.783679 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:23.784452 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.784862 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2527) took 1121 ms. Trace:
W20260709 13:08:23.787740 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.001s	sys 0.030s
W20260709 13:08:23.788282 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.001s	sys 0.030s
I20260709 13:08:23.789085 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2805) took 1129 ms. Trace:
I20260709 13:08:23.789361 26667 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.138s
I20260709 13:08:23.799918 26739 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:23.803753 26155 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:23.805773 26737 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:23.806344 25817 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:23.811486 26760 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:23.812212 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 197186us
I20260709 13:08:23.812403 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2510) took 2000 ms. Trace:
I20260709 13:08:23.813228 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:23.813454 26682 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:23.814159 26682 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:23.814810 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:23.816196 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 733419us
I20260709 13:08:23.783999 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2807) took 1124 ms. Trace:
I20260709 13:08:23.788084 26642 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:23.785327 21066 rpcz_store.cc:276] 0709 13:08:22.663596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.665774 (+  2178us) service_pool.cc:224] Handling call
0709 13:08:22.666041 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.666247 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:23.208720 (+542473us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.718791 (+510071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.757744 (+ 38953us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:23.784464 (+ 26720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.784586 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.784798 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":411210,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":18774,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":54380,"thread_start_us":979526,"threads_started":2}
I20260709 13:08:23.783847 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2794) took 1141 ms. Trace:
W20260709 13:08:23.826452 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 102126us
I20260709 13:08:23.833566 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:23.834301 26210 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:23.702415 26690 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:23.837528 21065 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:23.837786 26667 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:23.838245 26667 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:23.838649 26667 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:23.742810 26644 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:23.839082 21065 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:23.841576 21801 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:23.852499 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:23.859333 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 155385us
W20260709 13:08:23.872617 21067 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.660s	user 0.005s	sys 0.020s
I20260709 13:08:23.873041 21067 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:23.821883 21146 rpcz_store.cc:276] 0709 13:08:22.659822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.714853 (+ 55031us) service_pool.cc:224] Handling call
0709 13:08:22.715305 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.715586 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:23.330641 (+615055us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.699907 (+369266us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.721412 (+ 21505us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:23.743274 (+ 21862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.777943 (+ 34669us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.783920 (+  5977us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78378,"raft.queue_time_us":142893,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":170,"thread_start_us":875181,"threads_started":2,"wal-append.queue_time_us":485123}
I20260709 13:08:23.837518 26690 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:23.884362 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.831156 21145 rpcz_store.cc:276] 0709 13:08:22.642446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.645570 (+  3124us) service_pool.cc:224] Handling call
0709 13:08:22.645853 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.646064 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:23.329666 (+683602us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.721111 (+391445us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.765623 (+ 44512us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:23.783414 (+ 17791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.783548 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.783766 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53301,"prepare.queue_time_us":1075442,"raft.queue_time_us":176890,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":153,"thread_start_us":946893,"threads_started":2}
I20260709 13:08:23.845818 26737 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:08:23.883531 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 147558us
I20260709 13:08:23.839890 26533 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:23.853046 26644 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:23.903379 26669 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:23.909041 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:23.735471 26726 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:23.911543 26450 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.765s	user 0.014s	sys 0.007s
I20260709 13:08:23.913728 26726 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:23.914426 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2806) took 1254 ms. Trace:
I20260709 13:08:23.851572 26712 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:23.915023 21063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:60302
I20260709 13:08:23.915542 26712 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:23.916026 21063 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:23.916699 21063 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:23.919574 26564 consensus_meta.cc:287] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.044s	user 0.010s	sys 0.010s
I20260709 13:08:23.896129 21801 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
W20260709 13:08:23.889617 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 70389.1us
I20260709 13:08:23.789624 21140 rpcz_store.cc:276] 0709 13:08:22.659211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.683171 (+ 23960us) service_pool.cc:224] Handling call
0709 13:08:22.683548 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.683804 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:23.328295 (+644491us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.688708 (+360413us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.711700 (+ 22992us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:23.780654 (+ 68954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.788742 (+  8088us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.788994 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45318,"raft.queue_time_us":119190,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":33917,"thread_start_us":893274,"threads_started":2}
I20260709 13:08:23.924100 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2801) took 1271 ms. Trace:
W20260709 13:08:23.924942 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.925304 21793 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:23.927525 26681 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:23.928099 26767 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:08:23.928231 26681 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:23.927438 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54829.4us
I20260709 13:08:23.898039 26533 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:23.936749 26533 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:23.937366 26483 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:23.939268 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:23.939718 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:23.940518 21795 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
W20260709 13:08:23.940747 21110 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.985s (SENT) [suppressed 24 similar messages]
I20260709 13:08:23.812877 21064 rpcz_store.cc:276] 0709 13:08:21.736814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.742534 (+  5720us) service_pool.cc:224] Handling call
0709 13:08:21.742804 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:21.742999 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:22.911630 (+1168631us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.208728 (+297098us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.630707 (+421979us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.643851 (+ 13144us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:23.671817 (+ 27966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.695265 (+ 23448us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.737416 (+ 42151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1436546,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":36096,"raft.queue_time_us":432049,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":332,"thread_start_us":1792306,"threads_started":3}
I20260709 13:08:23.938179 26533 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:23.974460 21068 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.008s
I20260709 13:08:23.916061 26712 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:23.982926 26333 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:08:23.952482 26602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.016s	sys 0.020s
I20260709 13:08:23.984486 21218 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:23.975121 21068 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:23.914883 21147 rpcz_store.cc:276] 0709 13:08:22.659518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.700551 (+ 41033us) service_pool.cc:224] Handling call
0709 13:08:22.700863 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.701068 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:23.331680 (+630612us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.747006 (+415326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.777809 (+ 30803us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:23.846144 (+ 68335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.868206 (+ 22062us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.914361 (+ 46155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82021,"raft.queue_time_us":158815,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":149,"thread_start_us":898689,"threads_started":2,"wal-append.queue_time_us":470194}
W20260709 13:08:23.964411 21793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 640) took 2165 ms (client timeout 1562 ms). Trace:
W20260709 13:08:24.007802 21793 rpcz_store.cc:269] 0709 13:08:21.779417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.779658 (+   241us) service_pool.cc:224] Handling call
0709 13:08:23.927178 (+2147520us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b5400571054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:23.944289 (+ 17111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4884953344}
W20260709 13:08:23.993260 26602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.017s	sys 0.021s
I20260709 13:08:24.027841 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.018656 26333 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:24.033154 26333 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.036701 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:24.039979 26699 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.040725 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1686) took 1075 ms. Trace:
I20260709 13:08:24.041198 21067 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:08:24.041874 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1687) took 1098 ms. Trace:
I20260709 13:08:24.042204 21067 rpcz_store.cc:276] 0709 13:08:22.943849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.953477 (+1009628us) service_pool.cc:224] Handling call
0709 13:08:24.002473 (+ 48996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:24.041611 (+ 39138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.041815 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.059573 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1682) took 1117 ms. Trace:
I20260709 13:08:24.041138 21065 rpcz_store.cc:276] 0709 13:08:22.943555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.857063 (+913508us) service_pool.cc:224] Handling call
0709 13:08:23.919801 (+ 62738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:23.920129 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:23.920496 (+   367us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.018252 (+ 97756us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.018301 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:24.018355 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.018416 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.018518 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14355,"prepare.queue_time_us":637,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":41,"raft.queue_time_us":23329,"thread_start_us":74657,"threads_started":1}
I20260709 13:08:24.060103 21072 rpcz_store.cc:276] 0709 13:08:22.942288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.248923 (+306635us) service_pool.cc:224] Handling call
0709 13:08:23.249446 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:23.250017 (+   571us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:23.634167 (+384150us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.058429 (+424262us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.058794 (+   365us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:24.058950 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.059153 (+   203us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.059460 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1493,"prepare.queue_time_us":546830,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":45,"thread_start_us":644912,"threads_started":2}
I20260709 13:08:24.063320 21068 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:24.068240 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46527.8us
I20260709 13:08:24.072261 21063 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.089576 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:39498 (request call id 2545) took 1227 ms. Trace:
I20260709 13:08:24.090190 21068 rpcz_store.cc:276] 0709 13:08:22.836288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.243139 (+406851us) service_pool.cc:224] Handling call
0709 13:08:24.063920 (+820781us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.103870 26787 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:24.104627 26787 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:24.105310 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.106154 21794 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:24.105787 26787 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.109428 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.110795 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:24.111660 26682 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.112169 26690 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:24.112622 26690 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:24.112715 21071 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:24.113019 26690 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:24.113556 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 933) took 1094 ms. Trace:
I20260709 13:08:24.113948 21071 rpcz_store.cc:276] 0709 13:08:23.018814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.944669 (+925855us) service_pool.cc:224] Handling call
0709 13:08:24.040719 (+ 96050us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.113230 (+ 72511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.113490 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:24.067967 26718 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.902s	user 0.017s	sys 0.002s
I20260709 13:08:24.129082 26667 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.044222 26644 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:24.165053 26565 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.745s	user 0.012s	sys 0.001s
I20260709 13:08:24.165630 26565 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13.
I20260709 13:08:24.165920 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:24.166797 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 71738.7us
I20260709 13:08:24.168953 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:24.142766 21067 pending_rounds.cc:85] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 10
I20260709 13:08:24.171834 21067 pending_rounds.cc:107] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:08:24.172173 21067 raft_consensus.cc:2889] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:24.172680 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:24.173328 26667 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.170006 25817 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:24.178674 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2544) took 1355 ms. Trace:
W20260709 13:08:24.179872 26720 consensus_meta.cc:287] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.198s	user 0.006s	sys 0.015s
I20260709 13:08:24.180439 26721 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:24.181006 26567 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.207705 26567 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:24.209358 26781 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.291s	user 0.000s	sys 0.005s
I20260709 13:08:24.182914 26645 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:24.210067 26645 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:24.210484 26645 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:23.924554 21143 rpcz_store.cc:276] 0709 13:08:22.652283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.652963 (+   680us) service_pool.cc:224] Handling call
0709 13:08:22.653292 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.653536 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:23.328442 (+674906us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.923581 (+595139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.923677 (+    96us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:23.923760 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.923877 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.924030 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57950,"prepare.queue_time_us":1186745,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":5506,"raft.queue_time_us":138806,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":134,"thread_start_us":1129801,"threads_started":3,"wal-append.queue_time_us":474987}
I20260709 13:08:24.186136 26718 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.183013 26686 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:24.213850 26686 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:24.214273 26686 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:24.185238 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:24.190694 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 375960us
W20260709 13:08:24.197319 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 256600us
I20260709 13:08:24.197805 21801 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:24.199956 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2814) took 1335 ms. Trace:
W20260709 13:08:24.200546 26580 consensus_meta.cc:287] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.080s	user 0.012s	sys 0.011s
I20260709 13:08:24.200613 26737 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:24.201076 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 292126us
I20260709 13:08:24.201428 26564 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:24.202497 26682 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:24.219601 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1772) took 1228 ms. Trace:
I20260709 13:08:24.186091 21070 rpcz_store.cc:276] 0709 13:08:22.818674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.239952 (+421278us) service_pool.cc:224] Handling call
0709 13:08:23.240383 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:23.240774 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:23.240799 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:23.241466 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 13:08:23.711220 (+469754us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:23.711303 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.711400 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.173820 (+462420us) raft_consensus.cc:1685] finished
0709 13:08:24.173845 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.173980 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":442977,"threads_started":1,"wal-append.queue_time_us":930300}
I20260709 13:08:24.193531 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:24.215258 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 744) took 2181 ms. Trace:
I20260709 13:08:24.214668 26718 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.230921 26681 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:24.127018 26772 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.156s
I20260709 13:08:24.233537 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:24.217628 26739 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:24.234756 26739 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:24.236828 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.215873 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:24.211025 26567 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:24.206768 26721 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.262285 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:24.270040 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 188744us
I20260709 13:08:24.223592 21139 rpcz_store.cc:276] 0709 13:08:22.864724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.281409 (+416685us) service_pool.cc:224] Handling call
0709 13:08:23.282031 (+   622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:23.282477 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:23.682064 (+399587us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.199428 (+517364us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.199481 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:24.199530 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.199608 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.199717 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":329235,"prepare.queue_time_us":779134,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":7840,"raft.queue_time_us":340152,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":510075,"threads_started":2}
W20260709 13:08:24.271203 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53683.7us
I20260709 13:08:24.227162 21215 rpcz_store.cc:276] 0709 13:08:22.991366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.991524 (+   158us) service_pool.cc:224] Handling call
0709 13:08:22.991875 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:23.040888 (+ 49013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:23.040927 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:23.041498 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:08:23.325481 (+283983us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b1800402ac0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:23.772307 (+446826us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:23.821691 (+ 49384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.837992 (+ 16301us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.219170 (+381178us) raft_consensus.cc:1685] finished
0709 13:08:24.219191 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.219425 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60759,"raft.queue_time_us":58178,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":10047,"spinlock_wait_cycles":24326016,"thread_start_us":334659,"threads_started":1,"wal-append.queue_time_us":1149860}
I20260709 13:08:24.226428 26564 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:24.285595 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 119777us
I20260709 13:08:24.262820 26699 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:24.279717 21068 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:24.289817 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 940) took 1103 ms. Trace:
I20260709 13:08:24.290246 21068 rpcz_store.cc:276] 0709 13:08:23.186339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.279051 (+1092712us) service_pool.cc:224] Handling call
0709 13:08:24.279467 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.289524 (+ 10057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.289753 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.218032 26644 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:24.275426 26428 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:24.288369 26669 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:24.288450 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 72723.4us
I20260709 13:08:24.227854 21144 rpcz_store.cc:276] 0709 13:08:22.033442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.037573 (+  4131us) service_pool.cc:224] Handling call
0709 13:08:22.038001 (+   428us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:08:22.089331 (+ 51330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.089378 (+    47us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:08:22.090429 (+  1051us) log.cc:844] Serialized 104 byte log entry
0709 13:08:22.601004 (+510575us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:23.328881 (+727877us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.859327 (+530446us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.197843 (+338516us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.212169 (+ 14326us) spinlock_profiling.cc:243] Waited 9.78 ms on lock 0x7b180037a5c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:24.214185 (+  2016us) spinlock_profiling.cc:243] Waited 8.8 ms on lock 0x7b180037a5c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287 00007f7f42537208
0709 13:08:24.214589 (+   404us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.214852 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.214964 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.215021 (+    57us) raft_consensus.cc:1685] finished
0709 13:08:24.215032 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.215182 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":513178,"prepare.queue_time_us":1349677,"prepare.run_cpu_time_us":192,"prepare.run_wall_time_us":16319,"raft.queue_time_us":637631,"raft.run_cpu_time_us":702,"raft.run_wall_time_us":118120,"spinlock_wait_cycles":42726400,"thread_start_us":1472975,"threads_started":3,"wal-append.queue_time_us":639325}
I20260709 13:08:24.295831 26721 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.297027 26669 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.299751 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:24.300277 26565 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.301036 26565 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.301420 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:24.269490 26664 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:24.303650 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:24.304339 21068 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
W20260709 13:08:24.304896 26502 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.914s	user 0.013s	sys 0.005s
I20260709 13:08:24.305473 26502 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:24.305928 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 941) took 1119 ms. Trace:
I20260709 13:08:24.306002 26502 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.311769 26776 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:24.312392 26776 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:24.312821 26776 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:24.313597 26795 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:08:24.288256 26651 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:08:24.303815 26664 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.262117 26525 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:24.315042 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 935) took 1131 ms. Trace:
I20260709 13:08:24.299386 26737 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.306427 21068 rpcz_store.cc:276] 0709 13:08:23.186681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.305006 (+1118325us) service_pool.cc:224] Handling call
0709 13:08:24.305319 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.305579 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.305607 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.305647 (+    40us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:24.305697 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.305754 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.305864 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.283305 26564 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:24.216756 26800 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:24.219170 26720 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.302943 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:24.322764 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 942) took 1135 ms. Trace:
I20260709 13:08:24.302759 26712 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.330916 21068 rpcz_store.cc:276] 0709 13:08:23.186992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.321910 (+1134918us) service_pool.cc:224] Handling call
0709 13:08:24.322263 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.322437 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.322461 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.322499 (+    38us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:24.322535 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.322588 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.322703 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.288300 26634 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:24.329375 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:24.343917 21068 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:24.320955 26777 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:24.327724 26699 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:24.348383 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1700) took 1076 ms. Trace:
I20260709 13:08:24.272977 26580 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:24.352258 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 183434us
I20260709 13:08:24.357590 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.355063 21068 rpcz_store.cc:276] 0709 13:08:23.272406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.333895 (+1061489us) service_pool.cc:224] Handling call
0709 13:08:24.343723 (+  9828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.348055 (+  4332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.348305 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.336366 26634 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:24.335923 26565 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.366066 21215 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:24.367580 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 939) took 1175 ms. Trace:
I20260709 13:08:24.369222 21070 rpcz_store.cc:276] 0709 13:08:23.185999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.274108 (+1088109us) service_pool.cc:224] Handling call
0709 13:08:24.293074 (+ 18966us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.355632 (+ 62558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.355675 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.355729 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:24.361034 (+  5305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.361112 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.361238 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.375147 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:24.375903 21220 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:24.376150 21218 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:24.349920 26567 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:24.327904 26664 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:24.391386 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.363408 26634 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:24.361419 26718 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:24.386685 26788 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:24.406090 26664 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.406432 26788 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.397593 21213 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:08:24.407425 25762 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:24.398113 26759 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.405s	user 0.001s	sys 0.001s
W20260709 13:08:24.399479 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 96613.6us
I20260709 13:08:24.334158 26645 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.346858 26525 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.405587 26737 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:24.396397 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67138.3us
I20260709 13:08:24.411927 26682 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:24.412043 25762 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.413002 26699 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.421042 26634 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.426427 21793 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:24.427338 26634 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:24.430244 21070 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:24.411842 26669 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:24.413503 26720 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:24.414139 21220 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:24.392189 26543 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.442543 26543 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.420120 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.448040 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:24.450330 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:24.454222 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 937) took 1269 ms. Trace:
I20260709 13:08:24.455482 26502 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.360635 26800 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.420487 21765 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:24.454716 21067 rpcz_store.cc:276] 0709 13:08:23.185174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.047990 (+862816us) service_pool.cc:224] Handling call
0709 13:08:24.080759 (+ 32769us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:24.172992 (+ 92233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.173020 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:24.173630 (+   610us) log.cc:844] Serialized 52 byte log entry
0709 13:08:24.347532 (+173902us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:24.348048 (+   516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.348114 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.453995 (+105881us) raft_consensus.cc:1685] finished
0709 13:08:24.454013 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.454154 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57388,"raft.queue_time_us":12869,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":12444,"thread_start_us":173453,"threads_started":1,"wal-append.queue_time_us":278310}
I20260709 13:08:24.458985 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.460732 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.473129 26720 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:24.483639 26720 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:24.492074 26769 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.556s	user 0.003s	sys 0.000s
I20260709 13:08:24.492959 26720 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.494177 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2531) took 1829 ms. Trace:
I20260709 13:08:24.494851 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1761) took 1863 ms. Trace:
I20260709 13:08:24.495007 26718 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:24.500247 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:24.495235 21216 rpcz_store.cc:276] 0709 13:08:22.631718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.631860 (+   142us) service_pool.cc:224] Handling call
0709 13:08:22.632215 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.632557 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.632593 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.825740 (+193147us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b2c0011d108. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:08:22.826233 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:08:23.318520 (+492287us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:23.319252 (+   732us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.700633 (+381381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.720482 (+ 19849us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.491373 (+770891us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:24.494484 (+  3111us) raft_consensus.cc:1685] finished
0709 13:08:24.494504 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.494680 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70355,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":148049,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":8119,"spinlock_wait_cycles":70416000,"thread_start_us":832326,"threads_started":2,"wal-append.queue_time_us":987079}
I20260709 13:08:24.494648 21069 rpcz_store.cc:276] 0709 13:08:22.664721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.696240 (+ 31519us) service_pool.cc:224] Handling call
0709 13:08:22.696748 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.725048 (+ 28300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.725085 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.725577 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:08:23.237624 (+512047us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:23.237708 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.237836 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.397318 (+1159482us) spinlock_profiling.cc:243] Waited 8.19 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:24.464649 (+ 67331us) raft_consensus.cc:1685] finished
0709 13:08:24.464680 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.493975 (+ 29295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29407,"raft.queue_time_us":31806,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":18847616,"thread_start_us":488953,"threads_started":1,"wal-append.queue_time_us":1009691}
I20260709 13:08:24.503115 26580 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.504359 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.506105 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:24.460235 26525 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.457206 26712 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.499253 26656 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:08:24.508978 26656 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:24.509349 26656 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:24.510213 26656 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.474771 26737 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:24.501397 26708 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:24.462949 26788 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.516491 25762 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.526146 21212 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:08:24.439347 26690 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.540252 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:24.544402 26690 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.545117 26645 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.543013 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:24.548009 21067 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:08:24.549988 26803 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:08:24.539364 26656 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:24.555796 26800 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.557168 26776 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.559573 21068 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:24.564081 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 143673us
I20260709 13:08:24.566097 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.566243 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.566468 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.567761 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:24.569269 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 119123us
I20260709 13:08:24.569917 26776 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:24.571483 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45274.6us
I20260709 13:08:24.575338 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.579142 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.580271 26567 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:24.580955 26567 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:24.581441 26567 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:24.584838 26753 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.606s	user 0.003s	sys 0.000s
I20260709 13:08:24.584957 26739 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.586665 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 705) took 2760 ms. Trace:
I20260709 13:08:24.587100 21217 rpcz_store.cc:276] 0709 13:08:21.826499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.826772 (+   273us) service_pool.cc:224] Handling call
0709 13:08:21.827172 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:21.827619 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:21.827663 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:21.841864 (+ 14201us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.669236 (+827372us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b18004992a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:23.345731 (+676495us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:23.345816 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.345935 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.584003 (+1238068us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:24.586357 (+  2354us) raft_consensus.cc:1685] finished
0709 13:08:24.586383 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.586590 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45006,"raft.queue_time_us":48821,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":38635,"spinlock_wait_cycles":43669120,"thread_start_us":675895,"threads_started":1,"wal-append.queue_time_us":1845960}
W20260709 13:08:24.590035 26789 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 13:08:24.595752 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.590391 26686 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:24.598416 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 44155.5us
I20260709 13:08:24.596356 26543 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.646659 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:24.648057 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49951.2us
I20260709 13:08:24.647619 21217 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
W20260709 13:08:24.656780 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.663760 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:24.676190 26711 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:24.677151 26711 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.706017 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:24.728029 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.740409 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:24.697685 26564 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:24.755779 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 103428us
I20260709 13:08:24.755093 26564 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.725788 26739 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:24.766033 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:24.768564 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:24.769831 21069 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:24.771543 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:24.789326 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:24.734373 26581 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:08:24.785708 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:24.804538 21069 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:24.806332 26686 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.808130 26334 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:24.808616 26334 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:24.808969 26334 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:24.812773 26711 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:24.815229 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 54813.9us
I20260709 13:08:24.817971 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:24.818682 26699 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:24.819262 21111 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:24.821870 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.822480 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.823766 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:24.826608 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.831012 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:24.833408 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:24.834268 21212 raft_consensus.cc:2376] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13. Re-sending same reply.
I20260709 13:08:24.318572 21064 rpcz_store.cc:276] 0709 13:08:23.184112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.970138 (+786026us) service_pool.cc:224] Handling call
0709 13:08:24.018120 (+ 47982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.128966 (+110846us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:24.129464 (+   498us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.285100 (+155636us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.314686 (+ 29586us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.314742 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:24.314794 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.314861 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.314987 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56392,"prepare.queue_time_us":3578,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":89,"raft.queue_time_us":69514,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":24989,"thread_start_us":113453,"threads_started":1}
I20260709 13:08:24.845288 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:24.861266 21063 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:24.862521 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:24.863971 26818 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.124s	user 0.000s	sys 0.003s
I20260709 13:08:24.867228 26752 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:24.868300 26752 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:24.868619 26752 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:24.868685 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:24.869946 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:24.870572 21072 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:08:24.788628 26580 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:24.843027 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.842089 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:24.874889 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:24.885842 21064 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:24.880846 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:60302 (request call id 931) took 1862 ms. Trace:
I20260709 13:08:24.847016 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:24.886639 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67443.1us
I20260709 13:08:24.886597 21063 rpcz_store.cc:276] 0709 13:08:23.018229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.914799 (+896570us) service_pool.cc:224] Handling call
0709 13:08:24.861093 (+946294us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b4c00408480. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f407e8a20 00007f7f407f9a29 00007f7f407fa13f 00007f7f407be2ab 00007f7f407ebf3c 00007f7f407ec3fe 00007f7f40a7988b 00007f7f42478495 00007f7f424c930f 00007f7f42513a55 00007f7f474e3cd0
0709 13:08:24.880756 (+ 19663us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72506240}
I20260709 13:08:24.887881 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:24.885516 21796 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:24.875953 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56776.3us
I20260709 13:08:24.870519 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 938) took 1685 ms. Trace:
W20260709 13:08:24.872120 26797 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 13:08:24.885442 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 151742us
I20260709 13:08:24.892900 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:24.869535 26752 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.863452 26739 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:24.799633 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46800
I20260709 13:08:24.889142 21065 rpcz_store.cc:276] 0709 13:08:23.185569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.087351 (+901782us) service_pool.cc:224] Handling call
0709 13:08:24.122344 (+ 34993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:24.262592 (+140248us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:24.287146 (+ 24554us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.312155 (+ 25009us) spinlock_profiling.cc:243] Waited 8.8 ms on lock 0x7b18000f02e0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:24.499722 (+187567us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:24.512957 (+ 13235us) log.cc:844] Serialized 26 byte log entry
0709 13:08:24.515730 (+  2773us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:24.517075 (+  1345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.519482 (+  2407us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.835114 (+315632us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:24.870229 (+ 35115us) raft_consensus.cc:1685] finished
0709 13:08:24.870266 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.870433 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117758,"prepare.queue_time_us":21782,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":127506,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":10279,"spinlock_wait_cycles":59448448,"thread_start_us":159541,"threads_started":1,"wal-append.queue_time_us":335584}
I20260709 13:08:24.911535 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:24.911883 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:24.912622 21142 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:24.913619 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:24.915375 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.915557 26686 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:24.915649 26752 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:24.916042 26686 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:24.916366 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:24.916656 26686 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:24.919701 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.914311 21142 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:24.924018 21066 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 10.
W20260709 13:08:24.924685 26823 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:08:24.902292 21144 raft_consensus.cc:606] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Received request to transfer leadership
I20260709 13:08:24.925381 21144 raft_consensus.cc:614] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:24.926481 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:24.927037 21144 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:24.928104 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:24.929246 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:24.923019 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:24.862861 26567 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:24.892365 26564 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:24.900460 26804 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:24.945888 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:24.931565 21036 connection.cc:256] RPC call timeout handler was delayed by -1.833034818000101s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2786s for a response
I20260709 13:08:24.999663 26804 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:24.947408 26821 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:08:24.999927 21144 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:24.950367 21143 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:24.950755 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:24.952059 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:25.033126 21145 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:24.952896 26642 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:24.952836 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:24.953748 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.957881 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:25.034152 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 81373.3us
I20260709 13:08:24.958401 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:24.958525 26581 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:25.048944 26644 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:25.049417 26644 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.049793 26644 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:25.049628 26334 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:24.965927 21072 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:25.048972 21217 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:24.966732 21069 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:24.929232 26784 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:08:24.970978 21798 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:24.972440 26567 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:24.980110 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.986822 26788 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:24.991189 20969 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 166) took 30782 ms (client timeout 15000 ms). Trace:
I20260709 13:08:25.060043 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:25.059612 26788 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:25.059588 20969 rpcz_store.cc:269] 0709 13:07:54.211350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.282034 (+ 70684us) service_pool.cc:224] Handling call
0709 13:08:24.990867 (+30708833us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:19.607358 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:19.637943 (+ 30585us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:19.637993 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:08:19.638006 (+    13us) tablet.cc:662] Decoding operations
0709 13:08:19.639492 (+  1486us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:19.639579 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:19.639608 (+    29us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:08:19.640852 (+  1244us) tablet.cc:701] Row locks acquired
0709 13:08:19.640896 (+    44us) write_op.cc:260] PREPARE: finished
0709 13:08:19.641064 (+   168us) write_op.cc:270] Start()
0709 13:08:19.641220 (+   156us) write_op.cc:276] Timestamp: P: 1783602499641000 usec, L: 0
0709 13:08:19.641237 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:08:19.757280 (+116043us) log.cc:844] Serialized 6394 byte log entry
0709 13:08:20.119602 (+362322us) op_driver.cc:464] REPLICATION: finished
0709 13:08:21.298647 (+1179045us) write_op.cc:301] APPLY: starting
0709 13:08:21.325455 (+ 26808us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:22.831134 (+1505679us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:22.909236 (+ 78102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:23.920491 (+1011255us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:23.920940 (+   449us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 13:08:23.921034 (+    94us) write_op.cc:312] APPLY: finished
0709 13:08:23.921819 (+   785us) log.cc:844] Serialized 148 byte log entry
0709 13:08:24.182515 (+260696us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f3f4120ac 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b
0709 13:08:24.362543 (+180028us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:24.546509 (+183966us) write_op.cc:454] Released schema lock
0709 13:08:24.546608 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:24.547269 (+   661us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:24.551355 (+  4086us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5619067,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":1018310,"apply.run_cpu_time_us":84551,"apply.run_wall_time_us":3299528,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71039,"num_ops":16,"prepare.queue_time_us":497371,"prepare.run_cpu_time_us":32594,"prepare.run_wall_time_us":275023,"raft.queue_time_us":190120,"raft.run_cpu_time_us":22093,"raft.run_wall_time_us":362980,"replication_time_us":456041,"spinlock_wait_cycles":77589376,"thread_start_us":1271269,"threads_started":5,"wal-append.queue_time_us":633016,"wal-append.run_cpu_time_us":71441,"wal-append.run_wall_time_us":1124693}]]}
I20260709 13:08:25.063673 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:25.064296 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:25.008539 21064 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:24.949360 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2550) took 1645 ms. Trace:
I20260709 13:08:25.068392 21068 rpcz_store.cc:276] 0709 13:08:23.304332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.358525 (+1054193us) service_pool.cc:224] Handling call
0709 13:08:24.358818 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.560189 (+201371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.949130 (+388941us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54003a2fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:24.949298 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461055232}
I20260709 13:08:25.037276 21140 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:25.044078 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:25.045205 21146 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:25.047493 26642 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:25.034508 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 76709.3us
I20260709 13:08:25.073442 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:25.074259 21144 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:25.074223 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:25.048336 21147 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:25.048382 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:25.053537 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:25.065246 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:24.941303 26564 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:25.055796 21796 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:25.073071 21217 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:25.107012 21145 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:25.105298 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:25.145790 21139 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:25.128417 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:25.100411 21066 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:08:25.096560 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:25.080741 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:25.148075 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:25.151358 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:25.145536 21214 raft_consensus.cc:2376] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13. Re-sending same reply.
I20260709 13:08:25.057758 26720 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:25.215642 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:25.170823 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.216559 21068 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:08:25.216850 21064 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:08:25.217986 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 143838us
I20260709 13:08:25.219259 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:25.156433 21143 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:25.156286 21145 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:25.220044 21066 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:25.153635 21213 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:08:25.214236 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 291288us
I20260709 13:08:25.223954 21069 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:08:25.102149 26656 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.000s
I20260709 13:08:25.225028 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:25.222195 21798 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:25.231336 21071 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:25.096100 26776 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:08:25.219658 26725 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.283s	user 0.055s	sys 0.081s
I20260709 13:08:25.234292 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:25.235867 21144 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:25.144979 26564 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:25.254647 21140 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:08:25.090013 26642 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:25.108031 26667 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:25.081076 26784 consensus_queue.cc:1243] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:08:25.080545 26658 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:25.107851 26737 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.110598 26788 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:25.253700 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:25.259016 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 111019us
I20260709 13:08:25.244545 26776 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:25.265040 26335 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:25.265535 21219 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:08:25.265367 26485 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:25.267661 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:25.261404 26779 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:25.267855 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:25.108119 26686 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:25.251353 26725 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.315s	user 0.056s	sys 0.081s
W20260709 13:08:25.268524 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.269112 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:25.274605 21144 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:08:25.278134 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.255086 21140 pending_rounds.cc:85] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 10
I20260709 13:08:25.286118 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.329747 21140 pending_rounds.cc:107] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:08:25.330001 21067 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:25.330127 21140 raft_consensus.cc:2889] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:25.231102 25762 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:25.289254 21142 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:08:25.292650 26800 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
W20260709 13:08:25.315228 26828 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:08:25.299373 26658 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:25.220293 26699 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:25.311939 21187 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:25.357206 21071 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:25.317584 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:25.319919 21145 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:25.358989 21071 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:25.306622 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:25.087785 26804 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:25.353518 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.371827 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.354290 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.369917 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.380788 21142 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:25.382167 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.383023 21072 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
W20260709 13:08:25.392496 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 81942.8us
I20260709 13:08:25.397293 21143 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:25.401006 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:25.409951 21145 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: []
W20260709 13:08:25.407392 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 494433us
I20260709 13:08:25.339296 26485 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:25.364107 26779 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:25.327374 26667 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:25.438485 21796 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:08:25.446758 21215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.008s	sys 0.040s
W20260709 13:08:25.449229 21215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.008s	sys 0.040s
W20260709 13:08:25.456297 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 149943us
W20260709 13:08:25.458482 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 458561us
I20260709 13:08:25.407387 26725 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:25.367031 26737 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:25.460161 21068 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
W20260709 13:08:25.441592 26828 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:08:25.431083 26699 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:25.427220 26711 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:25.431452 26580 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:25.498270 21146 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:25.474882 26752 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.001s
I20260709 13:08:25.430732 25762 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:08:25.489470 26711 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:25.489243 26725 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:25.492897 26580 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:25.518025 25762 consensus_queue.cc:1243] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:25.408525 26776 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:25.536465 21146 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
W20260709 13:08:25.549235 21218 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.029s	sys 0.026s
W20260709 13:08:25.550665 21218 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.029s	sys 0.027s
W20260709 13:08:25.559512 21138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.030s	sys 0.086s
W20260709 13:08:25.561753 21138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.030s	sys 0.086s
W20260709 13:08:25.567560 21065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.025s	sys 0.038s
W20260709 13:08:25.575429 21065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.026s	sys 0.038s
W20260709 13:08:25.545289 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 57473.7us
I20260709 13:08:25.584975 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2563) took 1065 ms. Trace:
I20260709 13:08:25.543044 20970 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 7 to 8, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.689138 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:25.692395 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58846us
I20260709 13:08:25.552605 26485 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:25.699560 26725 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:25.634603 26686 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:25.720970 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:08:25.717415 21221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.023s	sys 0.069s
I20260709 13:08:25.724860 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:25.726018 21221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.023s	sys 0.069s
I20260709 13:08:25.516666 26658 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:25.739993 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:25.725051 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:25.724138 26645 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.752880 26645 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:25.556342 26580 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:25.767037 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:25.768850 21147 raft_consensus.cc:2410] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:08:25.765211 21801 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:25.740653 21143 raft_consensus.cc:2376] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17. Re-sending same reply.
I20260709 13:08:25.568733 26779 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:25.772656 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:25.777650 21140 raft_consensus.cc:2376] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17. Re-sending same reply.
I20260709 13:08:25.773275 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:25.779033 21145 raft_consensus.cc:2368] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 19.
I20260709 13:08:25.780339 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:25.781234 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.772291 26784 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:25.774866 26658 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:25.782452 26784 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:25.782073 20970 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 8 to 9, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.783546 26784 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:25.783546 26645 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:25.788798 26708 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:25.789003 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:25.789594 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4200817959999767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 13:08:25.789355 26708 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:25.789985 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4118348180001021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3254s for a response
W20260709 13:08:25.790328 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3768817960000206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
W20260709 13:08:25.790628 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3066817960000208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 13:08:25.790920 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2751817959999698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
I20260709 13:08:25.788501 20970 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 8 to 9, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.792487 26708 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:25.792907 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:25.793095 20970 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 9 to 10, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 10 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.795305 20970 catalog_manager.cc:5697] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.798149 20970 catalog_manager.cc:5697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:25.799593 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 83698.5us
W20260709 13:08:25.791211 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2355817960000195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 13:08:25.805871 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1986817959999598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 13:08:25.806504 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0123817959999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
I20260709 13:08:25.813073 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:25.814441 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:25.803269 20970 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 6 to 10, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 10 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.817438 26334 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:25.818425 20970 catalog_manager.cc:5697] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.820703 20970 catalog_manager.cc:5697] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 6, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 6 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.629676 26711 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:25.828020 26711 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:25.847235 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:25.849722 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.032s	sys 0.113s
W20260709 13:08:25.853055 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.032s	sys 0.114s
W20260709 13:08:25.853526 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 915638us
I20260709 13:08:25.854540 26739 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:25.855096 26739 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:25.856209 26739 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:25.808043 26667 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:25.862569 26777 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:25.857590 26739 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:25.866250 26777 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:25.882864 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:25.883580 21146 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:25.878981 26737 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:25.892855 21139 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.000s
I20260709 13:08:25.894362 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:25.895627 21146 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:25.883404 26777 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:25.896716 21139 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:25.914389 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:25.915834 21139 raft_consensus.cc:2368] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:08:25.922030 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:25.922371 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.923122 21070 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
W20260709 13:08:25.831626 25762 consensus_queue.cc:1175] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b70fc04de37e4239a8336534f0c9c0fb" responder_term: 19 status { last_received { term: 17 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: [18.14-18.14]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:25.925470 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.926772 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.928169 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:25.936126 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:25.937011 21072 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:25.955557 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 60581.3us
I20260709 13:08:25.962729 26699 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:25.963698 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1783) took 1050 ms. Trace:
I20260709 13:08:25.964215 21215 rpcz_store.cc:276] 0709 13:08:24.913047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.924502 (+ 11455us) service_pool.cc:224] Handling call
0709 13:08:24.924851 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:24.925256 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.925287 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:24.925778 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:08:25.451006 (+525228us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:25.451063 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:25.451159 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:25.963372 (+512213us) raft_consensus.cc:1685] finished
0709 13:08:25.963403 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:25.963593 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1158,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":94,"thread_start_us":524771,"threads_started":1,"wal-append.queue_time_us":959468}
W20260709 13:08:25.966342 26644 consensus_meta.cc:287] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.916s	user 0.013s	sys 0.000s
I20260709 13:08:25.967747 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:25.968499 21066 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:25.995865 21217 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.021s	sys 0.009s
W20260709 13:08:25.996373 21217 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.021s	sys 0.009s
I20260709 13:08:25.997639 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:25.978811 26543 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.010s	sys 0.037s
W20260709 13:08:26.007723 21141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.034s	sys 0.036s
W20260709 13:08:26.008190 21141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.035s	sys 0.036s
W20260709 13:08:25.998548 21798 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.023s	sys 0.006s
I20260709 13:08:25.585511 21065 rpcz_store.cc:276] 0709 13:08:24.519549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.907270 (+387721us) service_pool.cc:224] Handling call
0709 13:08:24.907598 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.907947 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:24.908201 (+   254us) log.cc:844] Serialized 14 byte log entry
0709 13:08:25.584510 (+676309us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:25.584585 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:25.584655 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:25.584744 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:25.584899 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1290,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":29,"raft.queue_time_us":12079,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":667356,"threads_started":1}
W20260709 13:08:26.009218 21798 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.023s	sys 0.006s
I20260709 13:08:25.966928 26644 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:26.007588 26543 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.010s	sys 0.037s
I20260709 13:08:26.016472 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:26.047935 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:25.998996 25762 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:26.061077 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:26.066648 21140 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:26.066782 21220 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:26.068415 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:26.071483 21139 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:26.067555 26708 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:26.084833 26644 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:26.096630 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3000) took 1011 ms. Trace:
I20260709 13:08:26.097152 21221 rpcz_store.cc:276] 0709 13:08:25.085118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.212421 (+127303us) service_pool.cc:224] Handling call
0709 13:08:25.212873 (+   452us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:25.213303 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:25.213349 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:25.214128 (+   779us) log.cc:844] Serialized 52 byte log entry
0709 13:08:25.726583 (+512455us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:25.726643 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:25.726748 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:26.096356 (+369608us) raft_consensus.cc:1685] finished
0709 13:08:26.096375 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.096561 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3151,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":12956,"thread_start_us":511923,"threads_started":1,"wal-append.queue_time_us":839606}
I20260709 13:08:26.100669 21066 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:08:26.070691 21072 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
W20260709 13:08:26.070880 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 59816.5us
I20260709 13:08:26.107750 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:26.108282 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:26.110564 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:26.111471 21072 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:08:26.071283 26564 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.769s	user 0.007s	sys 0.006s
I20260709 13:08:26.114002 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:26.115669 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:26.124738 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:26.124382 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.009s	sys 0.023s
W20260709 13:08:26.127422 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.009s	sys 0.023s
W20260709 13:08:26.130112 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 683807us
I20260709 13:08:26.128705 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.068958 25762 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.128815 26656 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:26.121048 26841 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.142591 21146 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:08:26.127442 26800 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.143082 21146 pending_rounds.cc:85] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 11
W20260709 13:08:26.141360 21218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.007s	sys 0.028s
W20260709 13:08:26.158088 21218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.008s	sys 0.028s
I20260709 13:08:26.160898 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1784) took 1247 ms. Trace:
I20260709 13:08:26.157662 21146 pending_rounds.cc:107] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:08:26.161636 21146 raft_consensus.cc:2889] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:26.145694 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.163798 26687 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.163930 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1116) took 1286 ms. Trace:
I20260709 13:08:26.164446 21801 rpcz_store.cc:276] 0709 13:08:24.874129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.874342 (+   213us) service_pool.cc:224] Handling call
0709 13:08:26.159829 (+1285487us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400494b54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:26.159973 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439169792}
I20260709 13:08:26.162832 21218 rpcz_store.cc:276] 0709 13:08:24.913420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.939430 (+ 26010us) service_pool.cc:224] Handling call
0709 13:08:24.978888 (+ 39458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.979175 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:25.552366 (+573191us) log.cc:844] Serialized 14 byte log entry
0709 13:08:26.159789 (+607423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:26.159866 (+    77us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:26.159936 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.160027 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.160814 (+   787us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":918632,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":131,"thread_start_us":1179179,"threads_started":2}
W20260709 13:08:26.137655 26720 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.893s	user 0.002s	sys 0.011s
I20260709 13:08:26.125921 26564 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:26.175458 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:26.174433 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 423815us
W20260709 13:08:26.185441 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:26.205928 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 392941us
I20260709 13:08:26.208853 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:26.210521 21139 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:26.212200 21144 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:26.171718 26720 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:26.218897 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:26.218269 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 72689us
I20260709 13:08:26.221707 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:26.163872 26800 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:26.222532 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 196595us
I20260709 13:08:26.169595 26690 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:08:26.213402 26739 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:26.242998 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.244191 26788 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:26.244737 26788 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:26.228847 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:26.170334 26687 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.235364 26854 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:26.158697 26841 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.244969 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.237773 26855 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:26.244084 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:26.255906 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:26.256446 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:26.249388 26841 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:26.251472 26834 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:26.246917 26687 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:26.264919 26708 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:26.265691 26708 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.266238 26708 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:26.246295 26800 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:26.270119 26777 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.274765 21066 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 13:08:26.275194 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2996) took 1191 ms. Trace:
I20260709 13:08:26.275619 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:26.276278 21069 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:26.277850 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:26.275621 21220 rpcz_store.cc:276] 0709 13:08:25.083587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.151086 (+ 67499us) service_pool.cc:224] Handling call
0709 13:08:26.275020 (+1123934us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b5400442fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:26.275109 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2067734912}
I20260709 13:08:26.245117 26788 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:26.237607 25762 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:26.239336 26564 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:26.247658 26854 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:26.247751 26776 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:26.290896 26779 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:26.291522 26779 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.293730 26779 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:26.261174 26699 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:26.285883 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:26.227562 21035 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:26.287403 21216 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:26.229526 26804 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:26.297070 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:26.298118 21072 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:26.298796 21220 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:26.300181 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 72762.9us
I20260709 13:08:26.302845 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:26.303450 26687 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:26.316614 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:26.303820 26863 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:26.304984 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:26.281276 26800 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.323028 26800 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.324568 21067 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:26.325261 26800 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:26.325788 26656 raft_consensus.cc:2764] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:08:26.325853 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:26.306974 26864 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.326548 21066 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:26.280228 26841 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:26.327401 26841 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:26.327811 26841 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:26.331832 21018 auto_rebalancer.cc:893] Removing replace marker failed: Replica 5e5e65640c8b4f5ca7eadc6a14312c55 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:08:26.331971 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:26.332429 21018 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:08:26.332762 21018 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:08:26.334249 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:26.326834 26864 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.336748 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:26.337381 26864 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:26.338820 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:26.344250 26777 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:26.345031 26735 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:26.346855 26690 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:26.316610 26687 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.349360 26687 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:26.351737 26863 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:26.352115 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:26.352747 26863 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.355072 21800 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:26.355574 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:26.356506 21072 raft_consensus.cc:2368] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14.
I20260709 13:08:26.357678 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:26.360558 21142 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:08:26.360136 26841 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.309872 21220 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:08:26.310081 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.312685 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:26.257709 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.366312 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:26.229313 26725 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:26.319720 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:26.367453 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47856.6us
I20260709 13:08:26.321282 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:26.370126 21066 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:26.375867 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:26.342640 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.387259 21071 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:26.390436 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:26.390456 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:26.411724 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:26.399436 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 142193us
I20260709 13:08:26.397421 21767 leader_election.cc:400] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14.
I20260709 13:08:26.414906 21767 leader_election.cc:403] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:08:26.414510 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:26.413142 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.417173 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.377717 26725 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:26.421815 21069 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:08:26.425004 21064 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:08:26.386961 26642 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:26.427103 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:26.430358 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:26.479806 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52964.1us
I20260709 13:08:26.482833 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 754) took 1565 ms. Trace:
W20260709 13:08:26.483706 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 103688us
I20260709 13:08:26.483438 21144 rpcz_store.cc:276] 0709 13:08:24.917173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.333744 (+416571us) service_pool.cc:224] Handling call
0709 13:08:25.352604 (+ 18860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:26.215879 (+863275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.482477 (+266598us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b540030af54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:26.482748 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1958566656}
I20260709 13:08:26.426180 26831 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14.
I20260709 13:08:26.396824 26686 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:08:26.397930 26543 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:08:26.478600 26667 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:26.497555 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:26.498338 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:26.493584 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:26.500104 21146 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:26.391960 26720 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:08:26.506424 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2830) took 1598 ms. Trace:
I20260709 13:08:26.506914 21143 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:26.483032 26711 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:26.511698 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:26.515232 21144 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:26.488227 26752 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:26.519140 21144 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:08:26.418241 26777 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:26.482489 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:26.410822 26804 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:26.544232 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61811.8us
I20260709 13:08:26.548259 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.508970 21141 rpcz_store.cc:276] 0709 13:08:24.907507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.083556 (+176049us) service_pool.cc:224] Handling call
0709 13:08:25.083849 (+   293us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:25.084258 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:25.084287 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:25.170443 (+ 86156us) log.cc:844] Serialized 52 byte log entry
0709 13:08:26.008603 (+838160us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:26.491513 (+482910us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b18003dcf00. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f45f143c7 00007f7f425269a7 00007f7f425360bf
0709 13:08:26.491655 (+   142us) log.cc:844] Serialized 14 byte log entry
0709 13:08:26.491891 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.491964 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:26.492002 (+    38us) raft_consensus.cc:1685] finished
0709 13:08:26.506124 (+ 14122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.506345 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":227947,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":1288,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":8870,"spinlock_wait_cycles":14733056,"thread_start_us":887460,"threads_started":2,"wal-append.queue_time_us":911235}
I20260709 13:08:26.549853 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.423014 26846 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:26.418325 26864 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:26.520740 26777 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:26.551240 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 122114us
I20260709 13:08:26.550750 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:26.551517 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 208952us
I20260709 13:08:26.554041 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:26.423017 26656 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:26.573757 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:26.574421 21066 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:26.427951 26642 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:26.428601 26708 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.558219 21071 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:26.569201 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:26.567230 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:26.567265 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:26.585151 21071 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:26.585801 21067 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:26.567598 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:26.566553 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:26.587476 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 161240us
I20260709 13:08:26.568073 21219 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:08:26.568636 21072 pending_rounds.cc:85] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 9
I20260709 13:08:26.516868 25817 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:26.562049 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64611.8us
I20260709 13:08:26.553334 26667 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:26.567936 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:26.579876 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 152871us
I20260709 13:08:26.568725 21146 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:26.584026 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:26.594205 21144 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:26.588593 21072 pending_rounds.cc:107] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:08:26.594738 21072 raft_consensus.cc:2889] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:26.577368 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:26.589159 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:26.592056 21142 raft_consensus.cc:2368] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14.
I20260709 13:08:26.592262 21794 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:08:26.568493 26843 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:08:26.595947 21069 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:26.597410 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:26.598106 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:26.599841 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:26.563438 26739 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:26.599995 21069 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:26.568207 26711 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:26.595479 21070 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:26.427881 25762 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.151s
I20260709 13:08:26.603076 21063 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:08:26.563218 26776 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:26.603139 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:26.565286 26752 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:26.588577 26642 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:26.579957 26656 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.604286 21071 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.586405 26841 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:26.623533 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:26.624109 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:26.630532 21071 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:26.604370 26847 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:26.647897 26800 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:26.648429 26800 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:26.648818 26800 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:26.649163 26831 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:26.649817 26831 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:26.548179 26725 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.651918 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:26.652827 26725 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:26.653728 26687 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.654177 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 2997) took 1570 ms. Trace:
I20260709 13:08:26.654542 21216 rpcz_store.cc:276] 0709 13:08:25.083978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.087434 (+  3456us) service_pool.cc:224] Handling call
0709 13:08:25.317379 (+229945us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:08:26.654015 (+1336636us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b54000bc2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:26.654120 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076643456}
I20260709 13:08:26.656970 26485 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:26.626591 26543 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:08:26.656674 26777 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.627876 21069 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.610575 26580 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:26.638919 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:26.669971 21146 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:26.574117 26846 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:26.679067 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64739.6us
I20260709 13:08:26.625483 26656 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:26.619544 26667 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:26.697528 21069 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:26.695348 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:08:26.695875 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 98565.3us
I20260709 13:08:26.639925 26855 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:26.704097 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 89672.5us
W20260709 13:08:26.676869 26859 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:08:26.616662 26711 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.625669 26864 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:08:26.716140 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:26.694470 26690 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:26.694126 26877 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:08:26.717085 21147 raft_consensus.cc:2376] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13. Re-sending same reply.
I20260709 13:08:26.700788 26580 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:26.707212 26855 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:26.728367 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 85057.8us
I20260709 13:08:26.712479 26485 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:26.730703 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:26.700073 26567 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:08:26.708446 26872 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:26.720943 26779 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:08:26.723198 26863 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:26.735875 21069 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:08:26.737524 21069 pending_rounds.cc:85] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 9
I20260709 13:08:26.737778 21069 pending_rounds.cc:107] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:08:26.738032 21069 raft_consensus.cc:2889] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:26.730768 26485 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:26.741880 26485 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.743233 26485 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:26.743316 26567 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:26.743986 26567 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:26.744777 26708 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:26.745462 26708 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:26.745853 26708 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.745183 26567 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.752907 26788 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:26.756199 26871 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:08:26.761200 26711 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:26.768190 21144 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:26.730988 26855 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:26.747054 26567 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:26.782882 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62637.6us
I20260709 13:08:26.746349 26708 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:26.740372 26863 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.720397 26667 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.780122 26864 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:26.809420 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45026.6us
I20260709 13:08:26.718073 26831 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:26.730091 26752 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:26.848444 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 117847us
I20260709 13:08:26.861373 26687 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:26.864066 26876 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:08:26.864434 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:26.865491 25762 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:26.865923 25762 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:26.866077 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:26.866981 25817 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:26.868018 25817 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:26.868343 26846 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:26.869019 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:26.869494 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:26.869724 26580 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.871451 25817 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:26.787111 26886 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:08:26.805195 26863 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:26.821213 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:26.819131 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 113862us
I20260709 13:08:26.875015 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:26.877713 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56631.2us
I20260709 13:08:26.800277 26804 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.878546 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:26.828738 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:26.888653 21143 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:26.882179 21069 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:26.876163 26868 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 13:08:26.896687 26859 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:08:26.922231 26687 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:26.931341 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.866746 25762 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:26.936710 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55436.1us
I20260709 13:08:26.879057 26737 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:26.879652 26739 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:26.896190 26831 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:26.932569 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 132720us
I20260709 13:08:26.845463 26877 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
W20260709 13:08:26.850788 26885 consensus_queue.cc:1175] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" responder_term: 16 status { last_received { term: 15 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:26.939451 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:26.824268 26864 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.931448 26881 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:08:26.971351 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 92894.6us
I20260709 13:08:26.925834 26854 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.950147 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 2999) took 1865 ms. Trace:
I20260709 13:08:26.902108 26564 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:08:26.938086 26788 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:26.936738 26862 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.982811 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:27.005029 21216 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:26.978755 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:26.940268 26737 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.932211 26846 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:08:26.922930 26804 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:26.987855 26779 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:26.987771 26867 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.948480 26687 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.003872 21221 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:27.003731 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 291311us
W20260709 13:08:27.014968 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 80170.8us
I20260709 13:08:26.975958 26881 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:27.010381 26737 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:27.003575 21069 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:26.947508 26887 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:08:27.043241 21216 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:27.043998 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:26.982419 21217 rpcz_store.cc:276] 0709 13:08:25.084759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.215188 (+130429us) service_pool.cc:224] Handling call
0709 13:08:25.244078 (+ 28890us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:25.297883 (+ 53805us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:25.996827 (+698944us) log.cc:844] Serialized 14 byte log entry
0709 13:08:26.230126 (+233299us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:26.232208 (+  2082us) log.cc:844] Serialized 26 byte log entry
0709 13:08:26.232646 (+   438us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:26.233392 (+   746us) log.cc:844] Serialized 14 byte log entry
0709 13:08:26.269657 (+ 36265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.269728 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:26.949843 (+680115us) raft_consensus.cc:1685] finished
0709 13:08:26.949873 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.950071 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35905,"prepare.queue_time_us":595694,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":36664,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":15525,"thread_start_us":833657,"threads_started":2,"wal-append.queue_time_us":377330}
I20260709 13:08:27.011684 25762 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:27.034759 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.002147 26889 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:27.023388 26777 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:26.970281 26874 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
W20260709 13:08:27.046766 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 109010us
I20260709 13:08:26.999715 26854 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.023631 26739 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:27.016247 26872 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:08:27.058743 21071 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:08:27.078202 21071 pending_rounds.cc:85] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 7
I20260709 13:08:27.047746 26881 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:27.038085 26779 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:27.009589 26888 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:08:27.078497 21071 pending_rounds.cc:107] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:08:27.016112 26862 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.087663 26739 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.091989 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:27.092178 21071 raft_consensus.cc:2889] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:27.068707 26699 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.051692 26788 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:27.108819 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.035554 26867 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.107568 26699 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.116591 26687 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:27.118281 26699 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:27.119370 26788 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:27.104313 26711 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:27.080349 26642 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.124014 26847 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:27.124459 26847 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:27.125206 26642 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:27.131220 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 188049us
I20260709 13:08:27.093802 26889 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:27.098788 26886 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:27.132652 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:27.132983 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.074489 26777 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.144825 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.149245 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:27.105257 26883 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:08:27.135556 26886 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:27.161465 26886 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.162534 26886 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:27.125377 26847 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.163173 21765 leader_election.cc:400] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14.
I20260709 13:08:27.161718 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:27.138878 26687 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:27.173487 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:27.173933 21800 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:27.173945 21068 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:27.177173 21795 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:08:27.183213 21071 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:08:27.181584 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 79922.7us
I20260709 13:08:27.188364 21220 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:27.193022 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:27.164043 26897 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:27.194974 21064 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:08:27.199566 21069 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:08:27.197072 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:27.164786 26901 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:08:27.144408 26873 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:08:27.166217 26739 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.164749 26903 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:27.206341 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.184371 26875 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.209597 21217 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:27.209939 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.212049 26804 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.208492 26580 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:27.214145 26580 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.217301 21069 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:27.219470 26905 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:08:27.220212 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.224156 26882 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:27.224634 26882 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:27.225136 26725 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.004s
I20260709 13:08:27.225589 26882 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.200544 26892 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:27.147208 26788 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.238219 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:27.240428 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.241279 26902 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:27.242468 26777 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.176376 26779 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.243232 26567 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:27.138432 26889 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.167265 26899 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:08:27.246717 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:27.243899 26739 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.250557 26739 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.246171 26690 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:27.253419 26739 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.254767 26737 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.243728 26567 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:27.260957 26867 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.260780 26567 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.262100 26567 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:27.249712 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.268064 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.268728 21212 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:27.269570 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:27.270473 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.208670 26887 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:08:27.234195 26875 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.275707 21798 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:27.275882 26892 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:27.273288 26777 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:27.277993 26485 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:27.278932 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:27.278649 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:27.278349 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:27.316092 21144 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: []
W20260709 13:08:27.295748 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 75626us
I20260709 13:08:27.287602 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.288568 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:27.320793 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48852.2us
I20260709 13:08:27.257822 26804 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:27.335175 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 97334us
W20260709 13:08:27.282656 26800 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.002s
W20260709 13:08:27.299477 26656 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.004s
W20260709 13:08:27.365748 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46983.4us
I20260709 13:08:27.357493 21141 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:27.359375 21036 leader_election.cc:400] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 19.
W20260709 13:08:27.414428 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78640.5us
I20260709 13:08:27.360388 21220 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:08:27.416771 21220 pending_rounds.cc:85] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 13
I20260709 13:08:27.417178 21220 pending_rounds.cc:107] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:08:27.417560 21220 raft_consensus.cc:2889] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:27.312937 26882 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:27.425063 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.297556 25817 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:27.282876 26784 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.296670 26892 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:27.312688 26776 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.300472 26863 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:27.277565 26889 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:27.306723 26891 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.303s	user 0.005s	sys 0.000s
I20260709 13:08:27.415145 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.417035 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.361196 21142 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:08:27.285000 26874 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.130s
I20260709 13:08:27.287708 26841 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.126s
I20260709 13:08:27.286430 26725 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 13:08:27.313433 26708 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.328284 26720 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:08:27.486340 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.487766 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.488651 21072 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:27.489461 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.490341 26902 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:27.439309 26737 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.299876 26777 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.444974 26784 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.493537 26777 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.277827 26564 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:27.494107 26784 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:27.285239 26735 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.144s
I20260709 13:08:27.496243 21795 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:27.496910 26708 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.471701 26892 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.446306 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:27.498253 21794 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:27.448522 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.449805 21796 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:08:27.413726 26656 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.501204 26656 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.450071 26776 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.502224 21069 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
W20260709 13:08:27.471076 26855 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.679s	user 0.018s	sys 0.001s
I20260709 13:08:27.506071 26855 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.509366 26855 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:27.459918 26913 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:08:27.350255 26804 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:27.469008 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:27.475322 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 60262.5us
I20260709 13:08:27.539047 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.472430 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.394731 26800 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:27.484138 26912 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:08:27.442152 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:27.456610 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 120524us
W20260709 13:08:27.458235 26911 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:08:27.541829 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.470522 26580 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:27.544394 21218 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:27.481495 26867 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:27.472863 26862 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.552145 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.468838 26485 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:27.555766 26737 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:27.556011 26725 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:27.555817 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 138851us
I20260709 13:08:27.545269 26777 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:27.466516 26864 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:27.551116 26914 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:08:27.561796 21071 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:27.566329 26855 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:08:27.544320 26564 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:08:27.571897 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:27.617981 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:27.607928 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.570494 26667 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:27.621507 26881 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.001s
I20260709 13:08:27.623920 26800 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:27.624194 26919 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:08:27.579061 26862 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.578019 26739 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:08:27.633205 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 61461.1us
I20260709 13:08:27.636232 26873 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:27.636878 26873 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:27.637367 26873 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:27.639048 26804 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:27.640128 26656 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:27.641393 26863 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:27.645726 26921 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:08:27.646817 21147 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:27.647137 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:27.648533 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:27.649066 21146 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:08:27.646838 21142 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:27.659945 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.663319 21145 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 13:08:27.665537 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.618103 26902 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:08:27.671995 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.672209 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.676642 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.676944 21212 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:08:27.682430 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:27.683358 21796 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:27.685497 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.650856 26776 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:27.691411 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 73610us
I20260709 13:08:27.692611 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.705181 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.707999 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.711687 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:27.715829 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:27.716548 25817 raft_consensus.cc:2764] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:08:27.716837 21036 leader_election.cc:400] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
I20260709 13:08:27.717132 21036 leader_election.cc:403] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:08:27.716558 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:27.714107 21217 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:27.633257 26881 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.675065 26875 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:27.663359 26580 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:08:27.632664 26667 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:27.692762 26711 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:27.663730 26862 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.730024 21141 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:08:27.733808 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:27.716459 26708 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:27.724232 26889 raft_consensus.cc:2764] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:08:27.740916 21215 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:27.741837 25817 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:27.749861 26881 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:27.755208 26893 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.023s	sys 0.027s
I20260709 13:08:27.721565 26924 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:08:27.760466 26862 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.725299 26915 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 12. Current term is 13.
W20260709 13:08:27.767109 26893 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.024s	sys 0.027s
I20260709 13:08:27.787012 26924 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:27.762063 26737 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 13:08:27.832329 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 98564.4us
I20260709 13:08:27.833084 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:27.780452 26739 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 13:08:27.810245 26932 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:08:27.811722 26847 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:27.813400 26667 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.816622 26902 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:27.819780 26854 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.835310 26909 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:27.830085 26881 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:08:27.827837 26658 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:27.829113 26720 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:08:27.795243 26920 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:08:27.794341 26788 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.784211 26855 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:08:27.841435 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:27.796542 26924 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:27.824234 26567 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.000s
I20260709 13:08:27.852661 21147 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:27.854225 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:27.857185 26899 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:27.860222 21217 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:27.864645 26711 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:27.865077 26909 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:27.866468 26924 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.847679 26902 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:27.868187 26915 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:27.852885 26779 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:27.873579 21793 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:27.848491 26658 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:27.877830 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:27.886061 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.887945 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:27.886132 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:27.879436 26881 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.895537 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:27.896073 25817 raft_consensus.cc:2764] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:08:27.872983 26899 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:27.899051 21069 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:27.900472 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:27.900717 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.883559 26909 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:27.913537 26909 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.860759 26788 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.905038 21072 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:08:27.905169 21063 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:27.887084 26779 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:27.895812 26875 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:08:27.891384 26906 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:27.909056 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:27.909584 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:27.910878 21142 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:27.898622 26881 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:27.918848 21138 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:27.936264 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.937228 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.938422 21138 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:27.939571 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:27.933807 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 57638us
I20260709 13:08:27.905774 26924 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:27.948611 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:27.952927 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:27.958025 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.961434 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.962452 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:27.963104 21795 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:27.964359 21795 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:27.933840 26902 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:27.970896 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.978722 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:27.981774 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:27.984860 21800 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:27.964439 21069 raft_consensus.cc:2955] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 2 to 9, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:27.974567 21213 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
W20260709 13:08:27.988008 21036 connection.cc:256] RPC call timeout handler was delayed by -2.0044180140000663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:08:27.988476 21036 connection.cc:256] RPC call timeout handler was delayed by -1.9954180140000517s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
W20260709 13:08:27.988826 21036 connection.cc:256] RPC call timeout handler was delayed by -1.9846180140000342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 13:08:27.989138 21036 connection.cc:256] RPC call timeout handler was delayed by -1.9405180139999629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
W20260709 13:08:27.989450 21036 connection.cc:256] RPC call timeout handler was delayed by -1.9198180140000431s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
W20260709 13:08:27.989782 21036 connection.cc:256] RPC call timeout handler was delayed by -1.9099180140000271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 13:08:27.990093 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8406180140000288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 13:08:27.990430 21036 connection.cc:256] RPC call timeout handler was delayed by -1.8289180140000099s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17220000000000002s for a response
W20260709 13:08:27.990756 21036 connection.cc:256] RPC call timeout handler was delayed by -1.7056180140000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 13:08:27.991091 21036 connection.cc:256] RPC call timeout handler was delayed by -1.5184180139999626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
W20260709 13:08:27.991439 21036 connection.cc:256] RPC call timeout handler was delayed by -1.4365180140000575s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 13:08:27.991775 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3888180139999804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
W20260709 13:08:27.992139 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3825180139999702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
I20260709 13:08:27.992777 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:27.993909 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:28.004319 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.998147 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:27.946465 26720 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:08:28.014415 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:28.003428 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:28.018883 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:28.023766 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:28.027815 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:28.026261 26708 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:28.032366 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:27.972689 26884 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:08:27.948280 26873 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.033669 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:28.025382 26779 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.037393 21063 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:28.037958 26779 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:27.967397 26658 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.038846 26658 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:27.999034 26872 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:08:28.040235 26788 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.040737 26872 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:28.041234 26872 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:28.042088 26872 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.045188 26873 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:28.047228 21213 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:08:27.963665 26899 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.014436 26895 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:28.058471 21221 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
W20260709 13:08:28.060904 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.076920 26884 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:28.077574 26884 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:28.079718 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:28.080988 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:28.081727 21072 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:28.061844 26895 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:28.088043 26895 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:28.111191 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 47051.5us
I20260709 13:08:28.112151 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:28.051167 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 58475us
I20260709 13:08:28.118415 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:28.128150 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.139343 26841 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:28.140189 26841 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:28.068013 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:28.162506 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 50482.3us
I20260709 13:08:28.162690 21800 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:08:28.176011 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 176162us
I20260709 13:08:28.180373 26708 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:28.189013 21217 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:08:28.185889 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:28.190083 21220 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:28.078629 26884 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.191970 26884 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:28.191676 26874 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:28.193270 26924 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:28.193676 26924 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:28.194528 26924 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.195729 26924 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:28.196255 26867 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:28.202466 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.160797 26863 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:28.205825 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:28.227171 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:28.205266 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 86973.7us
I20260709 13:08:28.235499 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:28.235679 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.232709 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:28.229585 21800 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:08:28.236528 21800 pending_rounds.cc:85] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 9
I20260709 13:08:28.236801 21800 pending_rounds.cc:107] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:08:28.242182 21800 raft_consensus.cc:2889] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:28.243949 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:28.241544 21139 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:28.244664 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:28.245419 21146 raft_consensus.cc:2368] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:28.245724 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:28.258371 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:28.258765 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:28.216578 26667 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:28.263593 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:28.265529 21138 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:28.267876 21146 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:28.237457 26841 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:08:28.271298 21142 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:28.271237 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.274642 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:28.275296 21142 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:28.269469 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.278748 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:28.279609 21139 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:28.275969 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:28.243250 26708 raft_consensus.cc:2764] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:08:28.278982 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:28.286010 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:28.247604 26867 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:28.287896 21146 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:28.241688 26863 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.293874 21216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:28.298676 21216 raft_consensus.cc:606] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.298743 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:28.299139 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.299666 21218 raft_consensus.cc:2410] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:08:28.301653 21141 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:28.302138 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:28.273795 26929 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:28.278963 26720 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:28.286882 26711 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:08:28.309166 26831 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.292798 26909 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:28.309505 26645 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:28.309970 26929 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.312359 26929 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:28.310011 26645 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:28.316521 26831 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.323626 26864 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:28.324105 26864 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:28.324975 26864 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.306466 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:28.288043 26841 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:28.316975 26881 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:28.281072 26854 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:28.331521 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 98894.2us
W20260709 13:08:28.337390 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 66221.6us
I20260709 13:08:28.341243 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.274245 26667 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:28.309237 26945 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:08:28.348764 21065 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:28.340564 26485 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:08:28.352932 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:28.353492 26485 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:28.353951 26485 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:28.354029 26667 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.355805 26831 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:28.351008 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:28.355592 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.356397 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:28.357293 26667 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:28.358407 26867 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:28.358593 26863 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.359376 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:28.351248 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.364864 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.336192 26864 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:28.368372 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.361980 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:28.337482 26841 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.384876 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.315222 26645 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.366379 26889 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:28.370275 26784 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:28.369733 26800 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:28.393925 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.354872 26485 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:28.370190 21145 raft_consensus.cc:2955] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 2 to 9, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:28.349956 26854 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.391023 26735 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:08:28.404786 26954 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:08:28.405036 26854 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.386467 26841 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:28.377982 26881 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:08:28.408849 21063 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:28.385962 26642 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:28.402895 26938 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.526s	user 0.003s	sys 0.000s
I20260709 13:08:28.402760 26485 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:28.392586 26804 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:28.423811 26804 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.424221 26804 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:28.387341 26788 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:28.427388 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.428507 26841 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:28.400806 26949 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:08:28.407215 26887 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:28.431579 26887 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:28.432559 26887 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.433844 26887 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:28.427786 26788 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.436539 26645 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:28.439692 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54247.4us
I20260709 13:08:28.440594 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:28.442114 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.442999 25817 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:28.444224 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:28.444859 21063 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:28.446017 26863 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:28.446883 26863 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:28.459678 21146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:28.460378 21146 raft_consensus.cc:606] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.460810 21146 raft_consensus.cc:614] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:28.461753 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:28.462931 26882 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:28.463367 26882 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:28.463480 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67710.1us
I20260709 13:08:28.463761 26882 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:28.464109 21142 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:28.464242 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.465385 26959 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:28.466063 26855 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:28.471761 26889 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:28.474907 25817 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:28.476965 26895 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.533687 26892 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.534300 26892 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.536473 26892 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.479802 21216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:28.538441 21216 raft_consensus.cc:606] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.540097 26788 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:28.481724 26776 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:28.541524 26776 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:28.542457 26776 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.482270 26902 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.483942 21144 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:28.543774 26895 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.544144 26902 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.546343 26902 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.485100 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.488612 26872 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:28.488948 21066 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:28.489378 26899 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:28.490576 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.550385 21796 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:28.550781 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.483901 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:28.491389 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:28.551334 21793 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:28.493144 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2610) took 1316 ms. Trace:
I20260709 13:08:28.501807 26804 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.483100 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:28.511643 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.446086 26841 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.560000 26895 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:28.560448 26895 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:28.560822 26895 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:28.561424 26804 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.561581 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:28.562196 21220 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:28.560464 26924 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:08:28.563524 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:28.554143 26788 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:28.564810 26867 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:28.560009 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:28.565279 26867 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.565682 26867 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:28.522946 26864 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.566242 26864 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.558964 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:28.567714 21217 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:28.551751 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 67939.7us
W20260709 13:08:28.552091 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67074.8us
I20260709 13:08:28.555006 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:28.568894 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:28.553568 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 70568.6us
I20260709 13:08:28.570199 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.571429 25817 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:28.556171 21214 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:08:28.557570 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:28.572150 21214 pending_rounds.cc:85] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 6
I20260709 13:08:28.572501 21214 pending_rounds.cc:107] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:08:28.572768 21214 raft_consensus.cc:2889] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:28.552078 21068 rpcz_store.cc:276] 0709 13:08:27.176479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.182101 (+  5622us) service_pool.cc:224] Handling call
0709 13:08:27.182428 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:27.273346 (+ 90918us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:27.561389 (+288043us) log.cc:844] Serialized 14 byte log entry
0709 13:08:27.630159 (+ 68770us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:27.630283 (+   124us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:27.630698 (+   415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.492911 (+862213us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b54000db954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:28.492962 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:28.493080 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14726,"prepare.queue_time_us":15221,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":231,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3219,"spinlock_wait_cycles":208584576,"thread_start_us":354532,"threads_started":2,"wal-append.queue_time_us":151908}
I20260709 13:08:28.526778 21064 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:28.578961 21069 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:08:28.580937 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.584602 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:28.567565 26889 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:28.572991 26902 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:08:28.618539 26889 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:28.619599 26889 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.603997 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:28.597766 26841 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:08:28.605580 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:28.606333 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.597935 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:28.624423 21213 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:28.602041 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:28.612763 21221 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:28.626405 21220 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:28.626770 21221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:28.627375 21221 raft_consensus.cc:606] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.603685 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:28.636868 21219 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:28.603178 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.638675 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.643162 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.643402 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:28.644986 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.655897 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.609632 26887 raft_consensus.cc:993] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:28.646727 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:28.543777 26776 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:28.687139 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 97877.2us
I20260709 13:08:28.687327 26863 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:28.600756 26864 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:28.683110 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:28.603763 26788 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:28.641404 26841 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:08:28.698148 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52939.9us
I20260709 13:08:28.656462 26854 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:08:28.700714 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44930.3us
I20260709 13:08:28.719362 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.605329 26642 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:28.711000 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 64497.5us
I20260709 13:08:28.597119 26567 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 13:08:28.716482 21219 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:28.651067 26784 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:28.730145 26863 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:08:28.765255 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 46587.7us
I20260709 13:08:28.730993 26864 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:28.731464 26708 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:28.653070 26737 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:08:28.732731 26925 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.651296 26855 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:28.609215 26883 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:08:28.606088 26959 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.099s
I20260709 13:08:28.681864 26862 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:28.609472 26888 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:08:28.681324 26800 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:28.731814 26906 raft_consensus.cc:1081] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:28.769734 21767 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.699955 26974 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:28.777527 26788 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:08:28.769320 26960 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:08:28.768385 26708 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:28.789034 26656 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:28.794556 26925 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.804741 21219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a473266451804da089da06e9c927e7ee"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:28.866854 21219 raft_consensus.cc:606] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.867389 21219 raft_consensus.cc:614] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:28.815078 26831 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:28.745466 26642 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:28.734429 26887 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:28.821337 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 86265.7us
I20260709 13:08:28.795373 26644 raft_consensus.cc:993] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:28.820106 26864 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:28.833624 26800 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.864181 26656 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.894541 21214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:28.895426 21214 raft_consensus.cc:606] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:28.895877 21214 raft_consensus.cc:614] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:28.899539 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:28.918495 25817 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:28.919113 25817 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.878371 26831 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:28.893684 26800 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:28.978545 25817 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:28.932667 25762 raft_consensus.cc:1081] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:28.996045 26959 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:29.005540 26959 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:28.933175 26956 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.008751 26956 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:29.013386 26970 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:08:29.017386 26864 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:28.999194 26656 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.001417 26708 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:29.084901 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.092888 26831 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.198143 21137 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d78b35ac92d402db8a5e3fe45be52b5 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:46800
I20260709 13:08:29.231359 26841 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.232147 26841 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.272351 26906 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.273165 26906 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:29.282351 21764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.000s	sys 0.040s
I20260709 13:08:29.282773 26956 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:29.282910 21764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.000s	sys 0.040s
W20260709 13:08:29.284318 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.003s	sys 0.022s
W20260709 13:08:29.284833 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.003s	sys 0.022s
W20260709 13:08:29.285389 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 566106us
I20260709 13:08:29.286340 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:29.300798 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 531129us
I20260709 13:08:29.301991 21767 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:29.306666 21069 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:29.309285 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.310246 21069 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 15: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:29.310318 21145 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:29.312140 21036 leader_election.cc:400] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:29.312583 21036 leader_election.cc:403] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:29.286651 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:29.312268 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:29.314366 21145 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:29.318202 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.321404 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:29.283697 26872 consensus_meta.cc:287] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.736s	user 0.050s	sys 0.119s
I20260709 13:08:29.327142 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:29.338117 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.349599 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:29.350215 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:29.351087 21069 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:29.352056 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.303478 26708 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:29.355747 21067 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.014s	sys 0.022s
I20260709 13:08:29.361250 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:29.361918 21067 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.015s	sys 0.022s
I20260709 13:08:29.290987 26974 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:29.362423 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.019s	sys 0.029s
I20260709 13:08:29.328850 26831 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:29.322108 26975 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:29.371417 21147 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.017s	sys 0.040s
W20260709 13:08:29.372787 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.019s	sys 0.030s
W20260709 13:08:29.332742 26957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.007s	sys 0.022s
I20260709 13:08:29.331084 26667 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:29.315428 26956 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:08:29.330772 26902 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:29.341015 26889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.007s	sys 0.028s
W20260709 13:08:29.355957 26899 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.808s	user 0.016s	sys 0.004s
I20260709 13:08:29.392266 26959 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:29.399585 20970 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 242) took 49791 ms (client timeout 15000 ms). Trace:
I20260709 13:08:29.352509 26903 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:08:29.384690 21798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.024s	sys 0.035s
W20260709 13:08:29.422834 21798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.025s	sys 0.036s
W20260709 13:08:29.368278 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56318.3us
I20260709 13:08:29.355543 26656 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
W20260709 13:08:29.376359 26882 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.912s	user 0.011s	sys 0.007s
W20260709 13:08:29.379011 26957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.008s	sys 0.022s
I20260709 13:08:29.124333 20964 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:29.405050 26721 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.411978 26863 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.414386 21069 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:29.385685 26975 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.417176 26909 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:29.379880 21147 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.017s	sys 0.041s
I20260709 13:08:29.415114 21212 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
W20260709 13:08:29.395421 26889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.007s	sys 0.029s
I20260709 13:08:29.396116 26841 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:29.393851 26667 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.400970 26906 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:29.398065 26902 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.451545 20964 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:29.431535 26882 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.423763 26847 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:29.474227 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:29.420142 20970 rpcz_store.cc:269] 0709 13:07:39.604025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.604197 (+   172us) service_pool.cc:224] Handling call
0709 13:08:29.395125 (+49790928us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:26.295535 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:26.339595 (+ 44060us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:26.339659 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:08:26.339687 (+    28us) tablet.cc:662] Decoding operations
0709 13:08:26.343461 (+  3774us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:26.343554 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:26.343585 (+    31us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:08:26.344428 (+   843us) tablet.cc:701] Row locks acquired
0709 13:08:26.344481 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:08:26.344638 (+   157us) write_op.cc:270] Start()
0709 13:08:26.344800 (+   162us) write_op.cc:276] Timestamp: P: 1783602506344575 usec, L: 0
0709 13:08:26.344820 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:08:26.345478 (+   658us) log.cc:844] Serialized 3668 byte log entry
0709 13:08:27.030674 (+685196us) op_driver.cc:464] REPLICATION: finished
0709 13:08:27.557018 (+526344us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f3f4120ac 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f45f52ef0
0709 13:08:27.747523 (+190505us) write_op.cc:301] APPLY: starting
0709 13:08:27.747665 (+   142us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:27.913857 (+166192us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:27.913905 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:28.045289 (+131384us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:28.045528 (+   239us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 13:08:28.045569 (+    41us) write_op.cc:312] APPLY: finished
0709 13:08:28.045826 (+   257us) log.cc:844] Serialized 89 byte log entry
0709 13:08:28.248662 (+202836us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:28.260468 (+ 11806us) write_op.cc:454] Released schema lock
0709 13:08:28.262972 (+  2504us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:28.302205 (+ 39233us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:28.302244 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1166637,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":693120,"apply.run_cpu_time_us":130990,"apply.run_wall_time_us":545084,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41765,"num_ops":9,"prepare.queue_time_us":454335,"prepare.run_cpu_time_us":47566,"prepare.run_wall_time_us":229234,"raft.queue_time_us":340423,"raft.run_cpu_time_us":53188,"raft.run_wall_time_us":793135,"replication_time_us":672256,"spinlock_wait_cycles":11806592,"thread_start_us":1660565,"threads_started":5,"wal-append.queue_time_us":476129,"wal-append.run_cpu_time_us":27497,"wal-append.run_wall_time_us":2381357}]]}
W20260709 13:08:29.493219 21213 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.854s	user 0.025s	sys 0.021s
I20260709 13:08:29.442694 26872 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.497828 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:29.487464 26903 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:29.488381 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:29.436626 26721 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.485719 26667 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:29.487650 26906 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.490968 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.506276 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:29.486369 26977 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.444093 26980 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:29.489995 26889 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:29.493089 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:29.438352 26863 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.526039 21140 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:29.494334 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:29.495776 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:29.475085 26895 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.903s	user 0.017s	sys 0.001s
I20260709 13:08:29.487025 21142 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:29.498138 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.497959 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:29.499341 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2893) took 1298 ms. Trace:
I20260709 13:08:29.488418 26925 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:29.500227 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.450626 26909 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.490968 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:29.507567 26984 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:08:29.538863 26800 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.006s
W20260709 13:08:29.508162 26986 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:08:29.499478 26776 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.009s
I20260709 13:08:29.512431 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:29.493634 21213 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:29.521018 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.533416 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:29.544289 21220 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:29.544437 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:29.533818 21069 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:29.500221 26979 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.501026 26656 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:29.503368 26915 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:29.533581 21147 rpcz_store.cc:276] 0709 13:08:28.201313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.203665 (+  2352us) service_pool.cc:224] Handling call
0709 13:08:28.232036 (+ 28371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:28.420757 (+188721us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b18004b72a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f42514060 00007f7f42514bfb 00007f7f4251f64b
0709 13:08:28.420903 (+   146us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:28.648640 (+227737us) log.cc:844] Serialized 14 byte log entry
0709 13:08:29.457925 (+809285us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:29.474281 (+ 16356us) log.cc:844] Serialized 52 byte log entry
0709 13:08:29.474421 (+   140us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:29.475025 (+   604us) log.cc:844] Serialized 14 byte log entry
0709 13:08:29.484342 (+  9317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.484492 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:29.499097 (+ 14605us) raft_consensus.cc:1685] finished
0709 13:08:29.499120 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.499256 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85936,"prepare.queue_time_us":17691,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":76,"raft.queue_time_us":71400,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":31468,"spinlock_wait_cycles":12541056,"thread_start_us":1059789,"threads_started":3,"wal-append.queue_time_us":789342}
I20260709 13:08:29.550433 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 3004) took 1572 ms. Trace:
I20260709 13:08:29.535501 26804 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:29.497692 26959 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:08:29.550863 21212 rpcz_store.cc:276] 0709 13:08:27.978194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.978458 (+   264us) service_pool.cc:224] Handling call
0709 13:08:29.550246 (+1571788us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540008fa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:29.550365 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3268428672}
I20260709 13:08:29.505398 26903 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.522970 26957 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:29.523661 26980 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.540840 26889 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.524611 26906 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.547158 26667 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:29.547374 26737 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:29.505055 26872 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.535921 26895 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.526882 26977 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.571471 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:29.544600 26800 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.589756 26656 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:29.592634 26895 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.593473 26889 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.562130 26882 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:29.597747 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 197784us
I20260709 13:08:29.615742 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:29.619020 21146 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:29.619705 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 302542us
I20260709 13:08:29.521620 20964 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 9 to 12, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:29.596644 26915 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:29.564594 26957 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.597779 26841 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:29.614208 26975 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:29.604391 20955 catalog_manager.cc:5028] TS 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): tablet 0d78b35ac92d402db8a5e3fe45be52b5 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:08:29.555672 26979 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:29.627556 20955 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 86920.6us
I20260709 13:08:29.631560 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:29.631596 21111 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:29.632246 21793 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:29.622913 26889 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.635478 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:29.629908 26976 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.166s	user 0.001s	sys 0.001s
I20260709 13:08:29.625670 26841 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.639722 26889 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.641554 26863 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.641999 26841 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:29.648793 26800 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:29.642918 26977 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.651822 26862 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:29.646041 26889 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.649133 26909 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.643153 20964 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 14 to 16, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 16 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:29.664348 26903 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.668354 26804 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:29.675608 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1093) took 1057 ms. Trace:
I20260709 13:08:29.676007 21798 rpcz_store.cc:276] 0709 13:08:28.618151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.619020 (+   869us) service_pool.cc:224] Handling call
0709 13:08:28.619649 (+   629us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:28.653928 (+ 34279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:28.653976 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:28.654625 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 13:08:29.423298 (+768673us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:29.423355 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.423440 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:29.675368 (+251928us) raft_consensus.cc:1685] finished
0709 13:08:29.675390 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.675548 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42296,"raft.queue_time_us":35295,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":12488,"spinlock_wait_cycles":4992,"thread_start_us":768300,"threads_started":1,"wal-append.queue_time_us":788069}
I20260709 13:08:29.678071 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.642222 26721 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:29.678771 21070 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:29.678906 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.680879 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:29.673255 26902 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:29.683557 26862 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:29.684640 26862 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.670641 26979 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:29.689191 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3015) took 1109 ms. Trace:
I20260709 13:08:29.675302 26841 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.728252 26957 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.728951 26841 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:29.721300 26980 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.729301 26895 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:08:29.730978 26867 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.165s	user 0.008s	sys 0.016s
I20260709 13:08:29.731477 26867 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.733731 26867 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:29.720768 21220 rpcz_store.cc:276] 0709 13:08:28.579370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.641800 (+ 62430us) service_pool.cc:224] Handling call
0709 13:08:29.544121 (+902321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:29.544689 (+   568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.688919 (+144230us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b5400442fd0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:29.689108 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2074435840}
I20260709 13:08:29.734467 21145 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:08:29.722191 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:29.723618 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:29.736244 21793 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:29.683063 26788 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:29.737222 26788 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:29.737669 26788 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:29.726253 25762 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.740717 25762 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:29.786540 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 422122us
W20260709 13:08:29.789448 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 277035us
I20260709 13:08:29.791844 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:29.812517 25762 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:29.812336 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 207008us
I20260709 13:08:29.817086 21798 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:08:29.818888 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.819672 21066 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:29.821666 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:29.822331 21141 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:29.825348 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:29.825488 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:29.826870 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:29.829272 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 148469us
I20260709 13:08:29.832315 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.833379 21066 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:29.834329 25817 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:29.834981 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:29.837440 25762 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.837890 25762 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.840595 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:29.841365 21767 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:29.811441 26909 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.842903 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:29.843942 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.845816 21068 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:29.847730 25762 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:29.812036 26980 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:29.823141 26708 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:29.850709 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:29.829010 26889 raft_consensus.cc:476] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Not starting pre-election -- already a leader
I20260709 13:08:29.854449 25817 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:29.841502 26864 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:29.837020 26995 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:29.864524 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:29.849535 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:29.870051 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:29.857358 26915 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:29.891762 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:29.859694 26872 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:29.861792 26667 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:29.901621 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:29.934949 21798 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:29.864821 26993 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:29.864550 26996 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:29.937983 25762 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:29.939357 25762 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.939828 25762 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:29.943007 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:29.943892 21794 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:29.850364 26909 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:29.877308 26906 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:29.868671 26998 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:08:29.955101 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 112290us
I20260709 13:08:29.860723 26980 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.956921 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:29.887328 26708 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:29.894222 26721 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:29.960855 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:29.979703 21070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39498
I20260709 13:08:29.980309 21070 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:29.980677 21070 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:29.977520 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:29.991338 21068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39498
I20260709 13:08:29.991678 26854 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.992012 21068 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:29.992396 21068 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:29.992213 26854 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.993554 26841 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.994415 26854 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:29.994313 26841 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.997181 26841 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:29.987852 26863 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:30.000665 21068 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.002725 21068 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.014264 25817 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:30.015695 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:29.888394 26864 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:29.937789 26862 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:29.947538 26996 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.025557 21065 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:29.952978 26924 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.664361 20964 catalog_manager.cc:5697] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 9 to 12, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:08:30.047715 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:30.030121 25762 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:30.035394 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 141805us
I20260709 13:08:30.038344 25817 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:30.048934 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:30.026304 21798 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:30.049468 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:29.976992 26915 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:30.051633 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.052870 21068 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:29.978042 26721 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.053849 26721 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:30.054260 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.056584 25762 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.059401 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:29.988051 26889 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:30.064335 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.067840 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.068569 21145 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:30.027069 26863 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:30.059437 26642 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.160s	user 0.004s	sys 0.016s
I20260709 13:08:30.074396 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:29.989919 26708 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.082345 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.082700 21065 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:08:30.083112 21140 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:30.077615 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.077239 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.037611 26996 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:30.084725 21138 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
W20260709 13:08:30.077355 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 177458us
I20260709 13:08:30.089753 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:30.092362 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:30.092945 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:30.093937 21798 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:30.120141 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:30.120913 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:30.121470 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49095.5us
I20260709 13:08:30.052091 26924 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.151670 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:30.120713 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:30.161811 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.162365 21141 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:30.038401 27002 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:08:30.157946 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.160763 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:30.163175 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.163585 21217 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:08:30.130295 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.081858 26883 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:30.069497 27005 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:30.046929 26862 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.173979 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.181713 21219 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:30.168196 21145 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:30.177510 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.179798 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:30.180840 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.184203 21146 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:30.184270 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.185910 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:30.189973 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.191545 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:30.191697 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.192883 21063 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:08:30.201884 21146 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:30.070786 26889 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.206301 21215 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:30.207015 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.088979 26903 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:08:30.211544 21146 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:30.207549 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:30.165117 26906 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.180188 26915 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:30.222509 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.223080 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.224182 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.224754 21147 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:30.095929 26642 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.172672 26883 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.221359 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.201267 26900 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.223735 21146 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:30.179901 26977 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:30.185952 26872 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:30.237799 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:30.238581 21216 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:08:30.238751 27017 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:08:30.238936 21216 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:30.239789 21216 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.201494 26887 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:30.211438 26882 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:30.209671 26867 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:30.269758 26909 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:30.270944 26788 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.533s	user 0.004s	sys 0.007s
W20260709 13:08:30.271418 26984 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:08:30.271922 26862 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.272305 21220 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:30.273952 26889 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.260660 26959 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:08:30.276361 26739 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:30.276796 26739 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:30.277796 26739 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.279162 26924 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:30.283047 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:30.242410 26887 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.249053 27030 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:30.249774 26708 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.249889 27026 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:08:30.251549 27003 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:08:30.254252 26721 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.222301 26915 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:30.279939 26882 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.258507 26996 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.259960 26980 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.288290 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:30.289379 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51818.9us
I20260709 13:08:30.292217 21147 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:30.281998 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.185403 26737 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:08:30.215746 27016 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:30.314597 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:08:30.372746 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 47643.4us
I20260709 13:08:30.228884 26667 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:30.324138 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 106109us
W20260709 13:08:30.326368 26656 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.001s
I20260709 13:08:30.237926 26883 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:30.235751 27002 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:30.255849 27004 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:08:30.261155 26998 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:30.229410 26906 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.376202 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:30.280333 26990 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:08:30.402045 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:30.363701 26882 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:30.393163 21795 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:30.395771 21794 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 15: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:30.271466 26788 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.243479 26863 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.426107 21070 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.439683 21072 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:30.441700 21070 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.443490 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.444186 21070 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:30.445473 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.446138 21070 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:30.447350 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.261864 26900 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.452261 26900 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.280784 26977 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.326138 27026 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:30.453665 27026 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.454476 26977 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.454803 27026 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:30.456395 21070 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:30.457504 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.391746 27002 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.459444 26881 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.459969 26881 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.460990 27002 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.409085 26667 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:30.462016 26881 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:30.462561 26881 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.462858 26884 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.463376 26884 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.465461 26884 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.410842 26906 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.411307 21217 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:30.412832 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:30.413363 26980 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:30.466986 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 54240.5us
I20260709 13:08:30.467077 26980 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:30.421454 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.467620 26721 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:30.467794 21110 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:30.468092 21067 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:30.468313 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:30.422451 21141 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:30.280535 26872 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.468998 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:30.325790 27030 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.470288 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:30.471431 27030 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.471642 26667 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:30.471771 26872 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.468880 21071 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:30.414759 26976 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:30.473335 26957 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:30.473927 26980 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:08:30.424510 27003 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:30.474747 26957 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:30.475464 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2895) took 1170 ms. Trace:
I20260709 13:08:30.475732 26957 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.476270 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.475845 21144 rpcz_store.cc:276] 0709 13:08:29.304918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.305036 (+   118us) service_pool.cc:224] Handling call
0709 13:08:29.305334 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:29.331405 (+ 26071us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:29.378532 (+ 47127us) log.cc:844] Serialized 14 byte log entry
0709 13:08:29.846541 (+468009us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:29.853572 (+  7031us) log.cc:844] Serialized 26 byte log entry
0709 13:08:29.861157 (+  7585us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:29.861215 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.861300 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:30.475232 (+613932us) raft_consensus.cc:1685] finished
0709 13:08:30.475253 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:30.475405 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31385,"prepare.queue_time_us":65803,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2719,"raft.queue_time_us":26148,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":23773,"thread_start_us":441583,"threads_started":1,"wal-append.queue_time_us":661932}
I20260709 13:08:30.448696 26887 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.491431 26776 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.492022 26776 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.494336 26776 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:30.496716 27028 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:08:30.401906 26998 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.499775 26998 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.387089 27020 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:30.417987 27022 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:08:30.465521 26915 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.472078 26667 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.389315 26656 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:30.481890 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:30.400122 26903 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:30.548588 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66774.1us
I20260709 13:08:30.486372 21071 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:30.511060 21142 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:30.417876 26864 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.534039 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.534816 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.553982 21138 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:30.462998 26881 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.554055 21143 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:30.537433 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.550722 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:30.555500 25817 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:30.555744 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.186s (SENT) [suppressed 11 similar messages]
I20260709 13:08:30.556123 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:30.562661 25817 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.585104 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:30.586464 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:30.536729 26887 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.395164 27016 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.543319 27030 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.599224 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.603835 21147 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:30.311378 20964 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 14, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 14 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:30.560369 26925 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:30.604350 25817 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.560781 26903 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.589665 26895 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:08:30.588219 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.590831 26915 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:30.609157 26900 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:30.599941 27019 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:30.610611 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1768) took 1048 ms. Trace:
I20260709 13:08:30.547806 26924 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.613863 26883 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.603417 26887 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.599758 26735 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:08:30.608662 27032 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:08:30.549759 26667 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:30.622872 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.611860 20964 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:30.624481 21069 rpcz_store.cc:276] 0709 13:08:29.562372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.562549 (+   177us) service_pool.cc:224] Handling call
0709 13:08:29.572320 (+  9771us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:29.588899 (+ 16579us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:29.605777 (+ 16878us) log.cc:844] Serialized 14 byte log entry
0709 13:08:29.983701 (+377924us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b1800050f80. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:08:30.309017 (+325316us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:30.318337 (+  9320us) log.cc:844] Serialized 26 byte log entry
0709 13:08:30.318565 (+   228us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:30.318616 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:30.318728 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:30.605696 (+286968us) raft_consensus.cc:1685] finished
0709 13:08:30.610379 (+  4683us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:30.610535 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16992,"prepare.queue_time_us":16609,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":1432,"raft.queue_time_us":184759,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":13817,"spinlock_wait_cycles":5540096,"thread_start_us":316275,"threads_started":2,"wal-append.queue_time_us":603214}
I20260709 13:08:30.538467 27020 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.617895 26925 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.567759 26854 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:08:30.634053 26735 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.629673 27019 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.655385 21218 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:30.666688 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:30.630497 26895 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:30.662582 26925 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:30.658862 26887 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.656132 26735 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:30.685496 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 99126.5us
I20260709 13:08:30.616291 27030 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.655949 26883 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:30.655215 26739 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:30.666798 27019 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:30.643929 26924 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:30.695815 21185 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:30.714692 26996 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:30.717047 27037 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:08:30.719014 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:30.722842 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.723258 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.724048 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.721261 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:30.724965 21219 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:30.724967 21214 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:30.727263 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.728006 21215 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:30.729530 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:30.730126 21144 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:30.732244 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:30.734457 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.729809 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:30.735589 21219 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:30.720357 26884 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:30.722119 26906 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:30.735925 21219 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:30.759326 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.762028 21213 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
W20260709 13:08:30.762586 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.016s	sys 0.063s
W20260709 13:08:30.762934 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.016s	sys 0.063s
W20260709 13:08:30.763355 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 558515us
I20260709 13:08:30.767601 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.761602 21219 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.770821 21144 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:30.787715 21219 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:30.734457 27045 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:30.759822 26895 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.812477 27045 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.812984 27045 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:30.813076 27020 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.814247 26899 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:08:30.815265 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.817078 27030 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.818456 26667 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:30.823172 27040 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:08:30.829628 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:30.830051 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.831610 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:30.832829 26881 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:30.833012 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:30.833662 21796 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:30.833674 21216 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:30.834954 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:30.841245 26924 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:30.772773 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 101638us
I20260709 13:08:30.738102 26884 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.760667 26906 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.695060 26882 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.661187 20964 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:30.891543 26800 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:30.892083 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:30.891705 27045 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.891786 26735 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.893272 26642 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.893554 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.949290 21066 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:30.894372 26903 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.891324 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.951442 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.896217 26862 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.953112 26862 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:30.903107 21219 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:30.904089 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.904536 26667 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:30.904655 21142 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:08:30.905683 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.960492 21139 raft_consensus.cc:2368] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:30.906780 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:30.961514 21143 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:30.906862 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:30.908036 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.964744 21138 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:30.908267 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.890751 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:30.905558 21218 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:30.967374 21796 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:30.909260 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.971660 21146 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:30.909626 26854 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:30.973202 26854 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:30.973690 26854 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:30.974720 26854 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:30.975992 21145 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:30.976653 26854 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:30.978770 26862 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:30.910326 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:30.910652 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:30.980664 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 70109.1us
I20260709 13:08:30.984278 26667 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:30.984788 26667 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:30.985195 26667 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:30.913157 26995 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.026s	user 0.011s	sys 0.003s
I20260709 13:08:30.917155 27019 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.006829 21797 raft_consensus.cc:2368] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:31.006779 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:31.008723 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.009609 21071 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:30.942101 26884 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:30.906860 21216 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:30.924245 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:31.016099 21793 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:31.017009 26735 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:30.932329 26906 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:30.935855 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.936829 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.021046 21063 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:31.020323 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.939987 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:30.942515 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 386757us
I20260709 13:08:31.021943 21138 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:31.021847 21069 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:30.943516 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:31.023394 21212 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:30.895323 26956 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:30.997123 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.025821 21065 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:08:30.998404 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 106485us
I20260709 13:08:31.000608 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:30.920455 26925 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.011761 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.031060 21066 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:31.012734 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:31.031996 21217 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:31.033094 27005 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:31.033639 27005 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.034102 27005 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:31.022028 21067 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:31.039119 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:30.983896 21143 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:31.028558 21145 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:31.026795 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:30.947820 21795 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:31.045145 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:31.045878 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:31.046427 21212 raft_consensus.cc:2410] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:08:31.027395 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:31.004398 26995 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:31.067891 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:31.069567 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.003633 26903 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:31.065914 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:30.949781 26788 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:31.070117 21035 leader_election.cc:400] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:31.072353 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.088884 21035 leader_election.cc:403] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:31.073961 27019 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:31.076115 27045 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:31.108983 21067 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:31.056442 26956 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.083529 21068 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:31.113195 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:31.114261 21142 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:31.114971 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:31.086021 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.115577 21145 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:31.084765 21065 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:31.071219 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:30.986545 26800 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:31.116714 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45609us
I20260709 13:08:31.118749 27031 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:31.119217 27031 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:31.120033 27031 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:31.121992 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:31.122660 21066 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:31.123993 26887 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:31.125036 26887 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:31.125387 26887 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:31.126179 26887 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.128137 26887 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:31.128947 26925 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:31.129175 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:31.129786 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:31.130312 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.131012 27016 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:31.117818 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:31.132872 26644 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:31.133371 26644 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:31.134166 26644 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.136408 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.091933 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.081804 26902 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:31.110304 21215 raft_consensus.cc:2368] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:31.110994 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:31.160735 21070 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:08:31.136792 20964 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:31.161285 21070 pending_rounds.cc:85] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 11
I20260709 13:08:31.091784 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:31.161916 21139 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:08:31.091825 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:31.163230 21070 pending_rounds.cc:107] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:08:31.164024 21070 raft_consensus.cc:2889] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:31.150534 21068 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:31.150760 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:31.109946 26735 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:31.114312 26656 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:08:31.154870 21145 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:31.066352 27002 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:31.157793 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:31.149425 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:31.171638 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:31.069018 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:31.166174 21063 raft_consensus.cc:2410] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:08:31.170742 21066 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:31.166311 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:31.087954 26847 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:31.088593 27027 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:31.158751 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:31.189244 21065 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:31.174479 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 83167.8us
W20260709 13:08:31.177641 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 108717us
I20260709 13:08:31.178993 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:31.194561 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:31.166571 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:31.191030 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:31.159443 26902 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.212692 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:31.213548 21143 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:31.176410 26735 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.221853 26902 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:31.207154 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:31.223500 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:31.224130 21147 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:31.225663 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:31.228731 21143 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:31.230502 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:31.224303 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:31.191041 27039 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:31.231851 26667 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:31.221887 27002 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:31.236102 27002 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:31.236933 21139 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:31.199342 26485 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:31.188972 27027 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.198408 27019 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:31.189124 27045 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:31.188604 26957 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:31.253772 25762 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:31.254827 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:31.255206 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:31.276319 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52108.9us
I20260709 13:08:31.277465 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:31.279232 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:31.258735 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:31.201622 26881 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:31.205785 26739 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:31.282159 26847 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:08:31.257217 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 65561us
I20260709 13:08:31.305543 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:31.261976 27020 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:31.231989 26642 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:31.284083 25762 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:31.286892 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:31.232563 27016 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:31.252877 26485 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.222636 27008 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:08:31.264333 27027 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:31.264827 27019 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.300621 21069 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:08:31.272977 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1774) took 1170 ms. Trace:
I20260709 13:08:31.290970 26998 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:31.303336 27030 raft_consensus.cc:2764] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:08:31.322299 25762 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.327457 21068 rpcz_store.cc:276] 0709 13:08:30.088819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.089487 (+   668us) service_pool.cc:224] Handling call
0709 13:08:31.259253 (+1169766us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b54002ad5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:31.259427 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2283740416}
I20260709 13:08:31.301261 26881 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.301434 26956 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:31.311228 27020 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.301690 26882 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:31.335031 21147 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
W20260709 13:08:31.342541 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.326324 27002 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.332180 26883 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:31.327068 20964 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:31.365123 20964 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:31.367494 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:31.323715 26485 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:31.326043 26642 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.326651 26847 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:31.330796 27019 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:31.310720 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:31.332785 27016 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:31.366891 20964 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 9 to 11, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 11 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:31.395231 21069 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:31.396867 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2660) took 1416 ms. Trace:
I20260709 13:08:31.396132 20964 catalog_manager.cc:5697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 10, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 10 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:31.397265 21064 rpcz_store.cc:276] 0709 13:08:29.980041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.981279 (+  1238us) service_pool.cc:224] Handling call
0709 13:08:29.981563 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:30.781289 (+799726us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b180043d9e0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f42514060 00007f7f42514bfb 00007f7f4251f64b
0709 13:08:30.781420 (+   131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:30.781443 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:30.781959 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:08:31.167207 (+385248us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:31.167258 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.371222 (+203964us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b5400232dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:31.371298 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:31.396642 (+ 25344us) raft_consensus.cc:1685] finished
0709 13:08:31.396664 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:31.396811 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31183,"raft.queue_time_us":154968,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":24276,"spinlock_wait_cycles":96242688,"thread_start_us":547036,"threads_started":2,"wal-append.queue_time_us":521265}
I20260709 13:08:31.398063 20964 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:31.340270 26881 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:31.412346 21140 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
W20260709 13:08:31.415820 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 105234us
I20260709 13:08:31.434010 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:31.434311 21219 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:31.391793 26642 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:31.418906 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:31.424034 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:31.426949 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:31.354949 27020 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:31.438285 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:31.402040 27039 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:31.473889 21036 leader_election.cc:400] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
W20260709 13:08:31.476034 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49195.3us
I20260709 13:08:31.477512 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:31.479135 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:31.439803 26847 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:31.402009 26906 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.475200 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:31.496044 21184 consensus_peers.cc:597] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a0bb40f8e3428899935fdfa915dfd0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:31.424860 26800 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:31.504261 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:31.496426 26959 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:31.428160 26708 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:31.519490 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:31.528237 26863 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:31.528841 26863 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:31.529337 26863 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:31.523404 26998 raft_consensus.cc:2764] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:08:31.497068 27026 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:31.529487 26906 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:31.430154 26735 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:31.467054 26915 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:31.544960 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65910.6us
I20260709 13:08:31.506839 27008 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:31.555135 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:31.556193 21068 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:31.555323 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.558284 21036 leader_election.cc:400] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:31.533784 26889 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:31.525115 26959 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.497061 26737 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 13:08:31.585319 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 358693us
I20260709 13:08:31.546718 26739 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:31.600905 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:31.601629 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:31.601961 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:31.604044 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:31.605065 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:31.607750 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.609427 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.631114 21065 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:31.596760 21068 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:31.668973 27050 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:31.669859 27050 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:31.676879 27049 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:31.677712 27049 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:31.584820 26959 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:31.635035 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67402.8us
I20260709 13:08:31.707053 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:31.716063 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.011s	sys 0.031s
I20260709 13:08:31.705732 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:31.730644 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.012s	sys 0.031s
W20260709 13:08:31.733038 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 542086us
I20260709 13:08:31.614274 26887 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.598510 27039 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:31.739317 21063 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.016s	sys 0.028s
W20260709 13:08:31.739787 21063 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.016s	sys 0.028s
I20260709 13:08:31.741359 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:31.745289 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:31.747256 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:31.748778 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:31.762805 21146 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:08:31.766397 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:31.767343 21800 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:31.769997 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:31.771952 21147 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:31.564692 27016 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:31.773103 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 479417us
W20260709 13:08:31.783335 21766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.020s	sys 0.035s
W20260709 13:08:31.784715 21766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.020s	sys 0.035s
I20260709 13:08:31.798992 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:31.800132 21794 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:31.682075 26800 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 13:08:31.805560 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:31.741233 27021 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:31.779471 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:31.767683 27039 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:31.817028 27056 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:08:31.671054 27050 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.819608 26882 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:31.820468 26882 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:31.823547 27021 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:31.826660 27021 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:31.748486 26887 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.837353 26887 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:31.838451 27040 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:08:31.838985 26737 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:08:31.803562 27031 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.003s
I20260709 13:08:31.839609 26737 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:31.839864 27031 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.840561 26737 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:31.770720 27026 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:31.844323 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:31.845198 26800 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:31.845556 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:31.847442 26899 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:08:31.770802 27005 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.002s
W20260709 13:08:31.848959 21218 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.022s	sys 0.044s
W20260709 13:08:31.849857 21218 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.022s	sys 0.044s
I20260709 13:08:31.851903 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.855026 21795 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:31.860169 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:31.861003 21147 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:31.817876 27039 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:31.859058 26884 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.792s	user 0.009s	sys 0.002s
I20260709 13:08:31.854318 26887 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:31.835008 26881 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:31.859944 27021 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:31.906922 21071 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:08:31.919384 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.863818 26899 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:31.854156 27005 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:31.938108 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.937932 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:31.890547 27026 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:31.941211 26887 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:32.052816 27026 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.053287 26887 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:32.003913 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 66043.3us
I20260709 13:08:32.004499 26906 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:32.059820 26906 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:32.060724 26906 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.063001 26979 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:32.063640 27019 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.639s	user 0.009s	sys 0.001s
I20260709 13:08:32.063764 26979 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.004895 27021 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:32.065857 27021 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.066534 27008 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.067090 27008 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.005275 27005 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:31.888547 27039 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:08:32.010336 26902 consensus_meta.cc:287] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.779s	user 0.007s	sys 0.004s
I20260709 13:08:32.018464 21797 raft_consensus.cc:2368] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19.
I20260709 13:08:32.019572 26899 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:32.022188 26656 consensus_meta.cc:287] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.846s	user 0.009s	sys 0.003s
W20260709 13:08:32.022619 27058 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.183s	user 0.002s	sys 0.001s
I20260709 13:08:32.025157 26667 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:32.033921 27059 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:08:32.040889 26881 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.044305 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:32.052124 26863 consensus_meta.cc:287] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.522s	user 0.013s	sys 0.000s
I20260709 13:08:32.064093 27019 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:32.017683 21140 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:08:32.068595 27039 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:31.892220 26884 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.865334 26485 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:31.937711 27031 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:31.949268 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:32.055096 27026 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.097388 21064 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:32.121116 27039 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:32.122565 27055 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.828s	user 0.005s	sys 0.005s
I20260709 13:08:32.122630 26485 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:31.936807 27049 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.130720 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:32.134600 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:32.136029 21064 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:31.893055 26882 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.140820 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1786) took 1050 ms. Trace:
I20260709 13:08:32.145215 21035 leader_election.cc:400] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19.
I20260709 13:08:32.145804 21035 leader_election.cc:403] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:31.936863 26948 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:32.104161 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:32.110266 27008 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:31.941922 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:31.894487 27037 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:08:31.928244 26998 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:32.075008 27020 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.004s
I20260709 13:08:32.176972 27020 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.101462 21066 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:08:32.109814 26863 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.067098 26979 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.127235 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:31.928687 27061 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:32.105789 26656 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:32.196452 21066 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:32.141386 21072 rpcz_store.cc:276] 0709 13:08:31.090540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.133900 (+ 43360us) service_pool.cc:224] Handling call
0709 13:08:31.134163 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:31.134471 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:31.134498 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:31.134969 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 13:08:31.553032 (+418063us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:31.553074 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.553149 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:32.033219 (+480070us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:32.123614 (+ 90395us) raft_consensus.cc:1685] finished
0709 13:08:32.123634 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:32.140730 (+ 17096us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1208,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":25734400,"thread_start_us":414665,"threads_started":1,"wal-append.queue_time_us":667159}
I20260709 13:08:32.153998 21065 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:32.198619 21068 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:32.198781 21220 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 19: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:32.072588 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:32.074970 27005 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:08:32.113024 27036 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:32.121158 27019 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:32.075852 26902 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:32.172397 21186 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.011s	sys 0.011s
W20260709 13:08:32.133155 26644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.020s	sys 0.019s
I20260709 13:08:31.936668 27066 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:32.179319 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:32.130177 27002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.001s	sys 0.042s
I20260709 13:08:32.186386 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1868) took 1152 ms. Trace:
W20260709 13:08:32.202497 26642 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.697s	user 0.008s	sys 0.004s
I20260709 13:08:32.204731 27026 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19.
I20260709 13:08:32.204170 21767 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:32.205279 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:32.156661 26900 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 13:08:32.198808 27027 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.864s	user 0.007s	sys 0.007s
I20260709 13:08:32.239936 21064 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:32.228646 21217 rpcz_store.cc:276] 0709 13:08:31.034892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.035345 (+   453us) service_pool.cc:224] Handling call
0709 13:08:31.036115 (+   770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:31.060662 (+ 24547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:31.060696 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:31.061158 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:08:31.428741 (+367583us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:31.428779 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.428854 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:32.021837 (+592983us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:32.178069 (+156232us) raft_consensus.cc:1685] finished
0709 13:08:32.178091 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:32.186315 (+  8224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37929,"raft.queue_time_us":26552,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":6981,"spinlock_wait_cycles":20078336,"thread_start_us":355752,"threads_started":1,"wal-append.queue_time_us":702115}
W20260709 13:08:32.138007 25762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.011s	sys 0.017s
I20260709 13:08:32.232620 21070 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:32.243649 25762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.011s	sys 0.017s
I20260709 13:08:32.184809 26998 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:32.231738 27002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.001s	sys 0.043s
W20260709 13:08:32.258590 21184 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.013s	sys 0.015s
I20260709 13:08:32.239836 21065 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:32.246359 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:32.228219 26644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.023s	sys 0.022s
I20260709 13:08:32.234750 21067 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:32.255857 21796 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:08:32.265545 21069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.026s	sys 0.038s
W20260709 13:08:32.259089 21184 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.013s	sys 0.015s
W20260709 13:08:32.267640 21069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.026s	sys 0.038s
I20260709 13:08:32.262814 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:32.262022 25762 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:32.171305 26948 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.273149 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:32.275547 21064 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:32.273890 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:32.306241 21063 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:32.241083 27039 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:08:32.321218 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58486.5us
I20260709 13:08:32.321286 25762 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:32.323159 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:32.324050 25762 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:32.328724 26735 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:32.341415 26735 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:32.258955 27027 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.325775 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:32.345803 21064 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:32.356812 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 741) took 1101 ms. Trace:
I20260709 13:08:32.357368 21220 rpcz_store.cc:276] 0709 13:08:31.255001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.255313 (+   312us) service_pool.cc:224] Handling call
0709 13:08:32.356533 (+1101220us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54000bc7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:32.356723 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016656640}
I20260709 13:08:32.268277 27008 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:32.370060 21110 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.027s	sys 0.041s
I20260709 13:08:32.232543 26642 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:32.371069 21110 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.027s	sys 0.041s
W20260709 13:08:32.371706 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 605410us
I20260709 13:08:32.372777 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 1788) took 1281 ms. Trace:
I20260709 13:08:32.373170 21067 rpcz_store.cc:276] 0709 13:08:31.091079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.129547 (+ 38468us) service_pool.cc:224] Handling call
0709 13:08:32.372616 (+1243069us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003ca354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:32.372722 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2475398016}
I20260709 13:08:32.375190 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:32.444909 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:32.445794 21793 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:32.449513 21070 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:32.345754 27027 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.231566 27066 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:32.383836 20964 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.019s	sys 0.019s
W20260709 13:08:32.370855 26959 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.001s
W20260709 13:08:32.455878 26847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.026s	sys 0.069s
I20260709 13:08:32.481361 21215 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:32.364776 27039 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:32.342612 26735 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.313239 26882 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.144s
I20260709 13:08:32.489070 26959 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:32.510430 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.006s	sys 0.011s
W20260709 13:08:32.517804 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.007s	sys 0.011s
W20260709 13:08:32.518496 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 712987us
W20260709 13:08:32.511153 26996 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.016s	sys 0.000s
I20260709 13:08:32.456496 26881 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:32.475962 26957 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:32.465719 26788 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:32.480088 20964 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.019s	sys 0.020s
I20260709 13:08:32.475857 26884 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:32.490000 26847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.026s	sys 0.072s
I20260709 13:08:32.481052 26841 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.514034 27027 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.438125 27008 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.533077 21140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
W20260709 13:08:32.520974 26996 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.017s	sys 0.000s
I20260709 13:08:32.549448 21139 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:32.554059 21140 raft_consensus.cc:606] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:32.554464 21140 raft_consensus.cc:614] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:32.566349 27008 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.568024 27027 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:32.575111 27079 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:08:32.536561 26863 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.537256 26881 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.558338 26841 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.577672 26881 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:32.578091 26841 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.498879 21064 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:32.597669 26998 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.598254 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 738) took 1344 ms. Trace:
I20260709 13:08:32.597532 27039 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.598660 21212 rpcz_store.cc:276] 0709 13:08:31.254116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.254285 (+   169us) service_pool.cc:224] Handling call
0709 13:08:31.254715 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:31.708258 (+453543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:31.708286 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:31.708732 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:08:32.187228 (+478496us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:32.187274 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:32.343349 (+156075us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b540015c054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:32.343396 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:32.597928 (+254532us) raft_consensus.cc:1685] finished
0709 13:08:32.597947 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:32.598195 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2066,"raft.queue_time_us":8054,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":3827,"spinlock_wait_cycles":71607168,"thread_start_us":873246,"threads_started":2,"wal-append.queue_time_us":806181}
I20260709 13:08:32.600605 26959 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.602131 26642 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.602430 27027 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.549889 26957 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:32.604656 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1793) took 1005 ms. Trace:
I20260709 13:08:32.589376 26979 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.605613 26902 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.605427 26957 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.552282 26882 consensus_queue.cc:1243] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:32.605041 21069 rpcz_store.cc:276] 0709 13:08:31.598988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.610576 (+ 11588us) service_pool.cc:224] Handling call
0709 13:08:31.610929 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:31.614676 (+  3747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:31.614724 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:31.615343 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 13:08:32.269639 (+654296us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:32.269683 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:32.269757 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:32.604431 (+334674us) raft_consensus.cc:1685] finished
0709 13:08:32.604452 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:32.604598 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4298,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":60981,"thread_start_us":615195,"threads_started":1,"wal-append.queue_time_us":912587}
I20260709 13:08:32.605372 26979 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.610162 26979 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.553304 26889 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.621973 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:32.622740 21797 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:32.630175 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:32.636735 26998 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:32.579425 27008 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.640319 21793 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:32.641721 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:32.544231 26788 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.647567 21140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:32.648367 21140 raft_consensus.cc:606] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:32.648751 21140 raft_consensus.cc:614] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:32.611370 26889 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.647132 27008 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.637312 26998 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.653558 27008 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.653823 26998 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.654076 21143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:32.661690 21143 raft_consensus.cc:606] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:32.639940 27027 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:32.658113 27008 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:32.658565 26889 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.651059 27030 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:32.664038 27030 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:32.662236 21143 raft_consensus.cc:614] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:32.665383 26788 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.648856 27066 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.670885 21143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:32.671761 21143 raft_consensus.cc:606] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:32.663196 27027 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.672260 21143 raft_consensus.cc:614] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:32.672384 27027 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:32.676043 21218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:32.676812 21218 raft_consensus.cc:606] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:32.677253 21218 raft_consensus.cc:614] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:32.677907 26998 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:32.678982 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:32.681071 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:08:32.682037 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 129767us
I20260709 13:08:32.683724 21765 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:32.678615 26998 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.713248 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:32.680032 26737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.015s	sys 0.019s
I20260709 13:08:32.665407 27030 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.691939 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 1113) took 1268 ms. Trace:
I20260709 13:08:32.669534 26788 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:32.685568 27050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.027s	sys 0.006s
I20260709 13:08:32.717226 21796 rpcz_store.cc:276] 0709 13:08:31.423448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.423738 (+   290us) service_pool.cc:224] Handling call
0709 13:08:32.691717 (+1267979us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b5400380254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:32.691874 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806652800}
I20260709 13:08:32.725452 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:32.717212 26737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.015s	sys 0.019s
I20260709 13:08:32.671521 27008 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.728291 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:32.725898 27050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.028s	sys 0.006s
I20260709 13:08:32.715974 26739 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.737735 27005 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:32.738262 27005 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:32.739164 27005 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.740046 27020 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.725759 26788 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.730160 27085 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:32.744235 27101 ts_tablet_manager.cc:933] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.749116 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:32.771260 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.771706 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:32.772118 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:32.749758 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.733505 26903 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:32.750069 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.773581 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:32.732944 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 410351us
I20260709 13:08:32.743007 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3204) took 1038 ms. Trace:
I20260709 13:08:32.760933 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.765488 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:32.775153 21139 rpcz_store.cc:276] 0709 13:08:31.704743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.704847 (+   104us) service_pool.cc:224] Handling call
0709 13:08:31.705078 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:32.554269 (+849191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:32.742753 (+188484us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b540036d654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:32.742942 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1941330816}
I20260709 13:08:32.733908 26902 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:32.733734 27060 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:32.779753 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.781847 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.783762 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:32.782545 21212 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:32.784749 21139 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:32.783592 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51653us
I20260709 13:08:32.791407 21139 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:32.746914 27027 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.802079 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.731274 26882 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:08:32.807678 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:32.809885 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:32.801388 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:32.805635 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:32.814121 21139 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:32.814350 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:32.815388 21065 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:32.814129 21070 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:08:32.786418 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:32.819710 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:32.748421 26739 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.774466 26889 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:32.774333 26863 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.809211 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:32.782550 26485 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.782480 26902 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:32.792737 26737 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.826725 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:32.828022 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.828539 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:32.835633 21212 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 19: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:32.828626 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.825429 21147 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:32.795917 27082 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.793040 26956 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.794884 27100 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:32.833738 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 47388.7us
I20260709 13:08:32.837512 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:32.804488 27005 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:32.803131 26959 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:32.848588 21146 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:32.847445 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:32.860047 21139 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:32.868398 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:32.869235 21218 raft_consensus.cc:2368] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17.
I20260709 13:08:32.870113 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:32.870429 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:32.871572 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.871948 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:32.872907 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.875208 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.874804 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:32.876381 21141 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:08:32.877437 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:32.880364 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:32.881304 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:32.881850 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:32.882604 21793 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:32.883626 21795 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:32.839566 26902 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:32.838611 26644 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.889492 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:32.890671 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:32.893781 26739 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.895428 26788 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.855656 26863 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.897948 26863 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.845919 27066 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:32.857481 27100 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.901297 27100 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:32.903949 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:32.905126 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:32.838539 26485 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.908362 26485 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:32.908985 26644 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.909451 26948 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:32.910244 21213 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:08:32.892184 21063 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:08:32.917337 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 76136.5us
I20260709 13:08:32.921911 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:32.922613 21139 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:32.883875 27076 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:32.830824 26889 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.932133 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:32.983348 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 61490.6us
I20260709 13:08:32.932960 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:32.846750 26737 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:32.848289 26956 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.858781 26959 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:32.996189 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:32.998397 21139 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:32.914551 26644 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:32.994405 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:33.004222 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:32.846207 27082 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:33.003178 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:33.016707 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 145993us
I20260709 13:08:33.024726 21064 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:32.930322 27114 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:33.018759 27019 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:08:32.962584 27058 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:08:33.020372 21795 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:33.020756 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:33.018217 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 134315us
I20260709 13:08:33.024324 21072 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:33.043699 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:32.962275 27085 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.028888 21141 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:33.036108 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:33.024849 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:33.105474 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 80703.8us
I20260709 13:08:33.106206 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:33.106475 26884 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:33.107010 26884 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:33.044907 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 260520us
I20260709 13:08:33.107952 26884 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:33.023721 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 89791.2us
I20260709 13:08:33.079851 21143 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:33.112085 27076 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:33.112362 27016 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:33.112649 27076 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:33.113511 27076 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:33.114303 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:33.114933 21142 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:33.115401 27027 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.084501 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.117285 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.001799 27026 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.119838 27082 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:33.119619 27026 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.121179 26889 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.121691 27026 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.122800 26959 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:33.126646 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:33.130471 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:33.131332 21795 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:33.131652 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:33.132309 21792 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
W20260709 13:08:33.081372 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 56948.4us
I20260709 13:08:33.079216 27008 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:33.114499 27111 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:08:33.133370 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.040169 27114 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:33.107309 27109 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:08:33.134860 26956 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:33.135923 27114 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.136224 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.114600 26902 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.136883 21145 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:33.137589 21793 raft_consensus.cc:2410] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:08:33.137794 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:33.100067 21215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 3036) took 1499 ms (client timeout 1548 ms). Trace:
I20260709 13:08:33.139779 26902 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:33.105696 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:33.139886 21215 rpcz_store.cc:269] 0709 13:08:31.600766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.600913 (+   147us) service_pool.cc:224] Handling call
0709 13:08:33.099792 (+1498879us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b540013b854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:33.099977 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011208960}
I20260709 13:08:33.110980 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:33.141279 27114 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:33.141607 21146 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:33.141815 27114 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:33.142872 21146 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:33.145928 21146 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:33.149050 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:33.151499 21139 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:33.152603 21145 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:33.088064 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:33.154234 21144 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:33.147886 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:33.038899 26998 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.157636 21145 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:33.162668 21139 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
W20260709 13:08:33.173058 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 88640us
I20260709 13:08:33.174145 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.165725 21141 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:33.171336 21146 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:33.179240 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.141222 26737 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.181012 21184 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.187680 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.191749 21796 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:08:33.192232 21796 pending_rounds.cc:85] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 12
I20260709 13:08:33.192495 21796 pending_rounds.cc:107] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:08:33.192790 21796 raft_consensus.cc:2889] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:33.195531 21142 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
W20260709 13:08:33.196943 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 63663.3us
W20260709 13:08:33.200450 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 112562us
I20260709 13:08:33.203428 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:33.203641 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:33.203970 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:33.205559 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:33.179988 26957 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:33.212531 21070 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:33.212806 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:33.213235 21067 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:33.100502 27085 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.209177 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.216442 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.217180 21071 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:33.206779 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:33.218238 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:33.218833 21036 leader_election.cc:400] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17.
I20260709 13:08:33.218820 21072 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:33.219326 21036 leader_election.cc:403] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:08:33.190920 26900 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:33.215814 21213 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:33.190274 26925 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:33.224387 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.196436 27019 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:33.226215 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:33.224310 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 50271.4us
I20260709 13:08:33.231479 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:33.203233 26956 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:33.234978 21147 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:33.194856 27123 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:33.235262 21111 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.142881 27114 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:33.215188 26979 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:33.193351 26485 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:08:33.230069 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:33.254453 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.223465 26708 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:33.264489 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.243484 21146 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:33.245965 21221 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:33.223953 27005 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:33.196449 27067 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:08:33.249221 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.250468 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.304385 21141 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
W20260709 13:08:33.304774 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54416.7us
I20260709 13:08:33.305776 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.249284 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.230924 26983 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17.
I20260709 13:08:33.277508 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.220919 27125 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:33.242046 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.263024 21147 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:33.252808 26979 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:33.323872 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 128336us
I20260709 13:08:33.225670 26642 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:08:33.247838 27123 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:33.331892 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 82716.8us
I20260709 13:08:33.338656 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:33.339202 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:33.339383 21792 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:33.340420 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:33.318761 21139 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:33.289368 27027 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:33.308038 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.235359 27120 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:33.263885 27082 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:33.244983 26883 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:08:33.318753 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:33.274714 26708 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:33.323042 26863 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:08:33.324700 26957 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:08:33.343164 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 53436.7us
I20260709 13:08:33.274955 27036 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:33.242451 26841 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 13:08:33.350167 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:33.350028 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 94260.5us
I20260709 13:08:33.340382 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:33.350132 27089 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:08:33.318701 26902 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:33.326781 26979 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:33.326882 26667 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:33.323547 27125 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:33.375747 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.376029 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.376726 21147 raft_consensus.cc:2368] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:08:33.377525 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:33.378391 21142 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:33.380295 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:33.353157 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:33.381245 21146 raft_consensus.cc:2368] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:08:33.356480 26708 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:33.347430 26737 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:33.329380 27101 tablet_copy_client.cc:323] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:08:33.356599 26959 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:33.344930 27123 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:33.372259 27084 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.812s	user 0.011s	sys 0.006s
I20260709 13:08:33.384837 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.346073 27020 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:08:33.389230 21138 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:33.389165 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53497.5us
I20260709 13:08:33.396497 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:33.374428 27129 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:08:33.397614 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:33.398373 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 79750.2us
I20260709 13:08:33.360846 26735 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:08:33.401825 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.404907 26957 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:33.384943 27008 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.412113 27085 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:33.414067 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.414723 26642 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:33.414815 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:33.419739 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.419927 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.386740 26959 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:33.379221 27125 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:33.387522 27120 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:33.436560 21144 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:08:33.437557 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:33.439707 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 87747.5us
I20260709 13:08:33.401167 27063 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:33.443866 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:33.446485 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:33.385594 26902 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.429802 26863 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:08:33.437688 27098 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.872s	user 0.009s	sys 0.009s
I20260709 13:08:33.428745 26735 consensus_queue.cc:1243] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:33.455889 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.462570 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.440887 26959 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:33.468842 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.468863 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.476580 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.477334 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.475696 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.439977 27031 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:33.427526 27036 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:08:33.439797 27120 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:33.484018 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.450837 26485 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:33.489101 21111 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.433600 27096 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:08:33.461704 26983 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:33.467028 27082 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.466470 26863 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:33.432935 27008 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.465646 26642 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:33.460286 26957 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:33.477993 27067 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:33.463685 27138 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:33.495731 27031 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:33.495522 26925 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:33.487792 27120 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:33.510730 26863 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:33.534063 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69205.2us
I20260709 13:08:33.512864 26642 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:33.494367 27030 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:08:33.500267 27114 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:08:33.557057 26708 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:33.558333 21072 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:33.559657 26708 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:33.509032 27082 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.566440 26642 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:33.551033 26957 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:33.523418 27138 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:33.583876 27138 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:33.588861 27139 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:08:33.664280 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:33.663300 26899 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:08:33.654273 27143 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.072s	user 0.003s	sys 0.000s
I20260709 13:08:33.667752 27050 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:33.668344 27050 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:33.669050 26902 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:33.669236 27050 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:33.670760 27050 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:33.673965 27132 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.348s	user 0.000s	sys 0.002s
I20260709 13:08:33.672089 26642 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:33.678309 27082 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.602617 27031 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.657246 27016 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:08:33.658459 27146 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.076s	user 0.003s	sys 0.000s
I20260709 13:08:33.563094 26979 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:33.688408 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 105944us
W20260709 13:08:33.690212 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 200351us
I20260709 13:08:33.693807 21765 leader_election.cc:400] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:08:33.704921 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:33.694383 27060 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:33.694638 27016 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:33.707681 27016 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:33.712446 27123 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.713351 21765 leader_election.cc:400] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:08:33.713810 21765 leader_election.cc:403] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:08:33.713291 26863 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.684926 26642 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:33.738382 27123 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.738722 26863 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.740866 27125 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:33.733110 26924 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:33.743024 27125 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:33.716676 27005 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:33.727883 27039 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:33.749188 27120 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.766417 26979 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.782351 27138 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:33.785465 27120 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.788143 27138 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:33.792311 27039 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:08:33.824402 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:33.841486 21069 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:33.844430 21063 pending_rounds.cc:85] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 10
I20260709 13:08:33.844954 21063 pending_rounds.cc:107] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:08:33.844813 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:33.845523 21063 raft_consensus.cc:2889] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:33.846290 21064 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:08:33.853639 21068 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:33.855126 21066 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:33.856091 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:33.859578 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:33.865882 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 376682us
I20260709 13:08:33.866647 21143 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 13: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:33.866590 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:33.870895 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:33.874640 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 185048us
W20260709 13:08:33.887917 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.888608 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:33.888945 21797 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:33.890085 21794 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:08:33.875759 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 171450us
I20260709 13:08:33.881626 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.890903 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:33.882859 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet b7a0bb40f8e3428899935fdfa915dfd0 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:08:33.892664 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet b7a0bb40f8e3428899935fdfa915dfd0 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-b7a0bb40f8e3428899935fdfa915dfd0
I20260709 13:08:33.891875 21793 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:33.899163 21216 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:33.897055 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:33.908556 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.911819 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:33.912796 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:33.917268 26642 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:33.924769 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:33.931885 26889 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:34.005209 21793 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:34.031684 21064 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
W20260709 13:08:34.064596 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 202233us
W20260709 13:08:34.063136 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 155395us
I20260709 13:08:34.069676 26644 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:08:34.087909 27020 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:08:34.088804 27122 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:34.091562 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:34.094179 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 379594us
I20260709 13:08:34.100358 21796 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:34.102607 27155 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:34.108381 21070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.005s	sys 0.007s
I20260709 13:08:34.108788 27067 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:08:34.110191 26959 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:34.111797 27005 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
W20260709 13:08:34.122767 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52783.3us
I20260709 13:08:34.122819 26902 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:08:34.112612 26889 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:34.129150 26889 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:34.130643 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 206159us
I20260709 13:08:34.131647 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:34.132910 26956 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:34.134434 26959 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:34.134238 26644 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:34.134704 27133 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:34.105302 27141 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:34.133688 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:34.105258 21147 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.004s	sys 0.008s
W20260709 13:08:34.142694 21147 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.004s	sys 0.008s
I20260709 13:08:34.145349 21795 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:34.146291 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:34.147334 21144 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:34.130558 26889 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:34.147111 21139 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:34.168581 27141 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:34.169937 27095 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:08:34.178920 21144 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:34.161219 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:34.187371 21071 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:34.189057 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:34.189173 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:34.190085 21064 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:34.195456 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:34.226922 27138 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.227949 27138 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:34.230706 21793 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:34.251761 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:34.254542 21138 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:34.231083 27138 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:34.263571 27076 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:34.264317 27063 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.264603 27076 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:34.265106 27063 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:34.267598 26906 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:34.231112 26900 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:08:34.268361 26906 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:34.275274 26906 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:34.279822 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:34.291260 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 157732us
W20260709 13:08:34.292202 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 55390.6us
I20260709 13:08:34.296557 21141 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:34.296579 26485 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:34.297562 26485 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:34.298080 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3217) took 1075 ms. Trace:
I20260709 13:08:34.298528 21143 rpcz_store.cc:276] 0709 13:08:33.223114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.224121 (+  1007us) service_pool.cc:224] Handling call
0709 13:08:34.295968 (+1071847us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b5400423bd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:34.297970 (+  2002us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466818432}
I20260709 13:08:34.300559 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:34.301452 21143 raft_consensus.cc:2368] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:08:34.301414 27076 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:34.305193 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:34.305212 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:34.306747 27147 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.653s	user 0.004s	sys 0.000s
I20260709 13:08:34.305927 21793 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:34.309577 27120 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:34.310353 27120 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:34.311626 26485 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:34.313035 27063 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:34.314869 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:34.321621 26656 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:34.322395 26656 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:34.323757 21793 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:08:34.328842 27160 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:08:34.328752 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:34.332486 26485 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:34.333402 26485 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:34.335323 27141 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:34.335804 26656 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:34.336462 26485 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:34.295259 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:34.345079 21138 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:34.332783 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:34.353739 21143 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:08:34.356552 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:34.361095 21139 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:34.371002 21064 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:34.392328 21143 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:08:34.393535 27008 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:34.402743 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:34.405449 27016 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:34.362758 27120 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:34.425968 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:34.425997 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1813) took 1223 ms. Trace:
I20260709 13:08:34.407423 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:34.427382 21066 rpcz_store.cc:276] 0709 13:08:33.202611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.204748 (+  2137us) service_pool.cc:224] Handling call
0709 13:08:33.205044 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:33.855790 (+650746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.425659 (+569869us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54002ad5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:34.425927 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477079936}
I20260709 13:08:34.428710 21036 leader_election.cc:400] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:08:34.429100 21036 leader_election.cc:403] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:08:34.429849 27030 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:34.431295 27002 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:34.432152 27002 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:34.435123 27002 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:34.413362 26899 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.747s	user 0.017s	sys 0.011s
I20260709 13:08:34.421417 27114 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:34.440670 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:34.441100 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:34.444355 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56439.1us
I20260709 13:08:34.472574 27123 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:08:34.473883 21140 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:34.460619 21035 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:34.475073 27123 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:34.475553 21140 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:34.477351 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 804) took 1099 ms. Trace:
I20260709 13:08:34.477909 21141 rpcz_store.cc:276] 0709 13:08:33.377833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.378007 (+   174us) service_pool.cc:224] Handling call
0709 13:08:34.296201 (+918194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:34.297300 (+  1099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.477023 (+179723us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b5400423bd0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:34.477255 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2110724480}
I20260709 13:08:34.478775 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:34.489169 21069 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:34.458930 27138 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:34.491640 27138 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:34.492059 27138 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:34.462836 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:34.502310 27164 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:34.503893 27060 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:34.504369 27060 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:34.505391 27060 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:34.507146 27060 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:34.538240 21138 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
W20260709 13:08:34.536839 21184 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.035s	sys 0.070s
W20260709 13:08:34.543058 21184 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.035s	sys 0.071s
W20260709 13:08:34.543797 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 629293us
I20260709 13:08:34.546731 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:34.548117 26957 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:34.548923 26957 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:34.548861 26956 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:34.550204 26956 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:34.550715 26956 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:34.550248 26957 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:34.543063 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:34.554757 27164 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:34.559783 26956 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:34.561398 26956 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:34.576084 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2752) took 1038 ms. Trace:
I20260709 13:08:34.576609 21070 rpcz_store.cc:276] 0709 13:08:33.537881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.538307 (+   426us) service_pool.cc:224] Handling call
0709 13:08:33.538630 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:33.539008 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:33.539041 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:33.548990 (+  9949us) log.cc:844] Serialized 26 byte log entry
0709 13:08:34.109284 (+560294us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:34.109368 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.109456 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:34.575795 (+466339us) raft_consensus.cc:1685] finished
0709 13:08:34.575820 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:34.576016 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1216,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":33641,"thread_start_us":559608,"threads_started":1,"wal-append.queue_time_us":1020402}
I20260709 13:08:34.580854 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:34.582171 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:34.586550 26735 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:34.587126 27141 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:34.587191 26644 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:34.587752 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:34.588743 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46004.2us
W20260709 13:08:34.597545 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 228729us
I20260709 13:08:34.598515 21143 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 13:08:34.609424 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71322.7us
I20260709 13:08:34.615162 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:34.631776 27027 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:08:34.570937 20964 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 167) took 20870 ms (client timeout 15000 ms). Trace:
I20260709 13:08:34.632891 27027 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:08:34.670670 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.682169 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:34.698006 26883 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:08:34.682029 27027 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:34.700047 27027 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:34.701620 27027 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:34.616554 27060 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:34.718897 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:34.767481 26863 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:34.768304 26863 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:34.768590 26841 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
W20260709 13:08:34.752061 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 109110us
W20260709 13:08:34.779186 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 97093.8us
I20260709 13:08:34.780233 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:34.718199 27008 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:34.645309 26889 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 13:08:34.749744 26485 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:34.645571 26656 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.126s
I20260709 13:08:34.833137 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:34.842516 27123 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:34.836541 21213 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:34.790184 27005 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.104s
W20260709 13:08:34.928884 27016 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.005s
I20260709 13:08:34.917520 26485 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:34.973641 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:34.930963 27008 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:34.900767 27073 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:34.845367 27060 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:35.029912 27180 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:08:35.030887 27080 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:35.032493 27080 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:35.032966 26884 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.325s
I20260709 13:08:35.033769 27060 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:35.033543 20967 catalog_manager.cc:5697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.031764 27019 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:35.038205 27019 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:35.038856 27182 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.439s	user 0.001s	sys 0.001s
I20260709 13:08:34.769693 26863 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:35.034824 27080 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:35.044340 27019 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:35.046084 26881 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:35.046591 26881 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:35.047484 27073 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:35.049885 27073 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:35.050165 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:35.051045 27030 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.004s
I20260709 13:08:35.051548 27030 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:35.051590 26881 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.053151 21072 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:35.053572 26881 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:34.952874 26899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:45190 (request call id 0) took 4035 ms. Trace:
W20260709 13:08:35.095974 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 306324us
I20260709 13:08:35.003716 27114 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:35.074540 26956 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.004s
I20260709 13:08:35.119913 20967 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:35.174443 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:35.135613 27138 consensus_meta.cc:287] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.000s
I20260709 13:08:35.074432 27080 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:35.197791 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 92181.7us
I20260709 13:08:35.207693 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:35.209805 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:35.125576 26889 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:35.232426 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 452257us
I20260709 13:08:35.116163 27002 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:35.106868 26656 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:35.159432 27120 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:35.165953 27030 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:35.253031 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2990) took 1071 ms. Trace:
I20260709 13:08:35.253997 27080 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:35.255270 26485 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.096128 26899 rpcz_store.cc:276] 0709 13:08:30.896912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.899706 (+  2794us) service_pool.cc:224] Handling call
0709 13:08:30.970614 (+ 70908us) log.cc:844] Serialized 513 byte log entry
0709 13:08:34.925046 (+3954432us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18700,"raft.run_cpu_time_us":2101,"raft.run_wall_time_us":159552,"thread_start_us":879056,"threads_started":2,"wal-append.queue_time_us":616997,"wal-append.run_cpu_time_us":6939,"wal-append.run_wall_time_us":1220409}
I20260709 13:08:35.261530 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 2991) took 1078 ms. Trace:
I20260709 13:08:35.233565 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:35.219791 20967 catalog_manager.cc:5697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:35.291195 21145 rpcz_store.cc:276] 0709 13:08:34.181335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.181493 (+   158us) service_pool.cc:224] Handling call
0709 13:08:34.181883 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:34.217134 (+ 35251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:34.217180 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:34.252072 (+ 34892us) log.cc:844] Serialized 26 byte log entry
0709 13:08:34.536563 (+284491us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:34.536623 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.536717 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:35.252683 (+715966us) raft_consensus.cc:1685] finished
0709 13:08:35.252704 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:35.252835 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28336,"raft.queue_time_us":35256,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":19296,"thread_start_us":236877,"threads_started":1,"wal-append.queue_time_us":972139}
I20260709 13:08:35.311954 21070 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:35.306566 21144 rpcz_store.cc:276] 0709 13:08:34.183531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.190666 (+  7135us) service_pool.cc:224] Handling call
0709 13:08:34.203708 (+ 13042us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:34.222051 (+ 18343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:34.222090 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:34.222610 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:08:34.294448 (+ 71838us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:34.294492 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.294550 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:35.261190 (+966640us) raft_consensus.cc:1685] finished
0709 13:08:35.261227 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:35.261439 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":454,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":71038,"threads_started":1,"wal-append.queue_time_us":1036481}
I20260709 13:08:35.319403 27030 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.320500 26656 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.321774 27030 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:35.322599 26656 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:35.323336 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:35.324740 27080 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:35.325039 21219 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:35.303996 27120 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:35.326134 27211 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:35.327935 27120 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:35.213951 27114 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.330139 27114 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:35.333590 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:35.334362 27002 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.335619 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:35.336658 27002 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:35.337652 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:35.337946 26889 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.213109 27138 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.348914 26899 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:35.349408 26899 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:35.346253 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 136483us
I20260709 13:08:35.361212 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:35.363376 21068 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:08:35.364748 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 131287us
I20260709 13:08:35.370414 21064 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:35.372793 21072 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:35.368050 27211 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:35.385421 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:35.385861 27211 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:35.387226 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:35.388358 27211 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:35.376827 27080 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:35.350584 26899 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:35.389580 26889 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:35.396029 20967 catalog_manager.cc:5697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.380246 26656 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:35.419816 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:35.426878 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 61250.2us
I20260709 13:08:35.482753 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:35.479802 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:35.408437 27076 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:35.505405 27214 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:35.434823 27210 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.519543 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:35.495606 20967 catalog_manager.cc:5697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 12 to 17, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 17 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.523509 20967 catalog_manager.cc:5697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:35.526177 20967 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 7 to 15, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 15 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.421655 27008 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.090s
I20260709 13:08:35.566099 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:35.570863 27172 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:35.571698 27172 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:35.574473 27172 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:35.575583 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:35.583935 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:35.632071 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:35.641811 27095 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:35.643512 26979 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.644207 26979 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:35.506192 27214 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:35.614697 20967 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 9 to 12, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 12 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:35.662335 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:35.534142 27076 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.510668 26656 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:35.625401 26899 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:35.642669 27095 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:35.682859 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 177707us
I20260709 13:08:35.647063 26979 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:35.684504 20967 catalog_manager.cc:5697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:35.709990 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 47770.1us
I20260709 13:08:35.750347 21147 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:08:35.750828 21147 pending_rounds.cc:85] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 12
I20260709 13:08:35.751600 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:35.750411 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 84866.3us
I20260709 13:08:35.751147 21147 pending_rounds.cc:107] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:08:35.753674 21147 raft_consensus.cc:2889] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:35.756845 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:35.757187 26889 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:35.666858 27073 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.541990 27210 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:35.667078 26957 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:35.754923 20967 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:08:35.760984 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 107575us
I20260709 13:08:35.712738 27138 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:35.716279 26841 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:35.713232 26956 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:35.776141 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:35.757834 26889 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.775511 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:35.771634 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 539360us
I20260709 13:08:35.803537 20967 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 10 to 13, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 13 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:08:35.777374 27027 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.075s	user 0.016s	sys 0.002s
I20260709 13:08:35.772541 27008 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:35.827757 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:35.827889 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 75196.7us
W20260709 13:08:35.778169 27005 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.003s	sys 0.035s
W20260709 13:08:35.831342 27005 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.004s	sys 0.035s
I20260709 13:08:35.809448 27123 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.771641 27073 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.844810 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:35.845476 21794 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.043s	sys 0.126s
W20260709 13:08:35.847206 21794 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.044s	sys 0.126s
I20260709 13:08:35.848073 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:35.850178 21035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.014s	sys 0.047s
W20260709 13:08:35.850656 21035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.014s	sys 0.047s
W20260709 13:08:35.852142 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.854984 20967 catalog_manager.cc:5697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:35.856581 27199 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.366s	user 0.000s	sys 0.002s
W20260709 13:08:35.856710 21764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.043s	sys 0.063s
W20260709 13:08:35.867221 21764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.044s	sys 0.064s
I20260709 13:08:35.840608 27209 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.809861 26956 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:35.869704 21038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.030s	sys 0.010s
W20260709 13:08:35.887054 21038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.030s	sys 0.010s
I20260709 13:08:35.890707 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:35.907989 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:35.919772 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:35.921171 27095 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:35.924772 27063 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:35.925100 27005 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:35.925429 27063 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:35.925624 27005 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:35.928135 27222 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:08:35.934724 21792 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:08:35.929767 27208 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:08:35.858382 27123 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:36.003566 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 172470us
I20260709 13:08:35.929033 21145 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:35.885519 27211 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 13:08:36.019466 21793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.022s	sys 0.026s
W20260709 13:08:36.027521 21793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.023s	sys 0.026s
I20260709 13:08:36.026542 20967 catalog_manager.cc:5697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:36.046707 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.050226 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1141) took 1332 ms. Trace:
I20260709 13:08:35.913242 27209 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.050807 21793 rpcz_store.cc:276] 0709 13:08:34.708419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.708594 (+   175us) service_pool.cc:224] Handling call
0709 13:08:34.752800 (+ 44206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:34.900142 (+147342us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:08:35.393298 (+493156us) log.cc:844] Serialized 14 byte log entry
0709 13:08:36.040287 (+646989us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:36.040354 (+    67us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:36.040408 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.040481 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.040638 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":336484,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"thread_start_us":1007305,"threads_started":2}
W20260709 13:08:36.058151 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.004067 27002 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:08:36.028295 27076 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:36.088251 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3005) took 1507 ms. Trace:
W20260709 13:08:36.075421 21797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.016s	sys 0.042s
W20260709 13:08:36.094522 21797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.016s	sys 0.042s
W20260709 13:08:36.101122 27219 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.364s	user 0.001s	sys 0.001s
I20260709 13:08:36.102074 27229 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:36.102824 27229 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.105794 27229 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:36.058993 26656 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:36.111480 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 48433.7us
I20260709 13:08:36.112082 20967 catalog_manager.cc:5697] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 4 to 9, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 9 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:36.042200 27005 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.059427 27027 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:36.069728 27138 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:36.132521 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.092782 21142 rpcz_store.cc:276] 0709 13:08:34.581284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.584344 (+  3060us) service_pool.cc:224] Handling call
0709 13:08:34.584688 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:34.585459 (+   771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:34.585489 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:34.586090 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 13:08:34.816676 (+230586us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:34.891311 (+ 74635us) log.cc:844] Serialized 14 byte log entry
0709 13:08:34.891543 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.891634 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:35.249196 (+357562us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b1800407740. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:36.066031 (+816835us) raft_consensus.cc:1685] finished
0709 13:08:36.087970 (+ 21939us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.088176 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3299,"prepare.queue_time_us":74995,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":6678,"raft.queue_time_us":43595,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":16750,"spinlock_wait_cycles":42270592,"thread_start_us":230151,"threads_started":1,"wal-append.queue_time_us":779122}
I20260709 13:08:36.133558 21068 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
W20260709 13:08:36.137948 21141 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.009s	sys 0.018s
W20260709 13:08:36.152899 21141 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.010s	sys 0.018s
I20260709 13:08:36.092252 27214 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.125542 20967 catalog_manager.cc:5697] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 5 to 9, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 9 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:36.173087 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:36.174921 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3001) took 1595 ms. Trace:
I20260709 13:08:36.175525 21138 rpcz_store.cc:276] 0709 13:08:34.579820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.586877 (+  7057us) service_pool.cc:224] Handling call
0709 13:08:34.587211 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:34.594519 (+  7308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:34.594563 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:34.645956 (+ 51393us) log.cc:844] Serialized 26 byte log entry
0709 13:08:34.904701 (+258745us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:34.904758 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.904854 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:36.174642 (+1269788us) raft_consensus.cc:1685] finished
0709 13:08:36.174671 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.174843 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6911,"raft.queue_time_us":2698,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4441,"thread_start_us":123199,"threads_started":1,"wal-append.queue_time_us":806359}
I20260709 13:08:36.185812 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.186777 21068 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:36.161507 27210 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:36.132477 27218 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.278s
I20260709 13:08:36.172003 27060 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:36.178308 27008 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:36.181324 27080 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:36.201426 27063 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.215394 27215 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.152454 27138 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.160720 27027 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.179096 26889 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:36.218427 26881 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.164s	user 0.016s	sys 0.001s
I20260709 13:08:36.219983 27123 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.221131 27218 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:08:36.222020 27223 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
W20260709 13:08:36.222826 27224 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:08:36.226377 20967 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:36.224969 27060 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:36.231555 26956 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:36.257535 21066 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.009s	sys 0.013s
I20260709 13:08:36.235132 27073 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:36.257998 21066 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.009s	sys 0.013s
W20260709 13:08:36.254380 21065 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.013s	sys 0.008s
I20260709 13:08:36.232134 27080 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:36.261020 21065 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.013s	sys 0.008s
I20260709 13:08:36.242202 27218 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:36.281189 21067 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.005s	sys 0.017s
W20260709 13:08:36.281973 21067 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.005s	sys 0.017s
W20260709 13:08:36.286900 21072 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.006s	sys 0.017s
W20260709 13:08:36.288007 21072 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.006s	sys 0.017s
W20260709 13:08:34.674360 20964 rpcz_store.cc:269] 0709 13:08:13.700040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.700432 (+   392us) service_pool.cc:224] Handling call
0709 13:08:34.570624 (+20870192us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:32.610940 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:32.611640 (+   700us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:32.623092 (+ 11452us) write_op.cc:435] Acquired schema lock
0709 13:08:32.638033 (+ 14941us) tablet.cc:662] Decoding operations
0709 13:08:32.654903 (+ 16870us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:32.661600 (+  6697us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:32.661642 (+    42us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:08:32.686576 (+ 24934us) tablet.cc:701] Row locks acquired
0709 13:08:32.686614 (+    38us) write_op.cc:260] PREPARE: finished
0709 13:08:32.686817 (+   203us) write_op.cc:270] Start()
0709 13:08:32.686966 (+   149us) write_op.cc:276] Timestamp: P: 1783602512686751 usec, L: 0
0709 13:08:32.686995 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:08:32.687866 (+   871us) log.cc:844] Serialized 5618 byte log entry
0709 13:08:33.115505 (+427639us) op_driver.cc:464] REPLICATION: finished
0709 13:08:33.283889 (+168384us) write_op.cc:301] APPLY: starting
0709 13:08:33.283983 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:33.597124 (+313141us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:33.656124 (+ 59000us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:33.793840 (+137716us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:33.794041 (+   201us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 13:08:33.794070 (+    29us) write_op.cc:312] APPLY: finished
0709 13:08:33.794311 (+   241us) log.cc:844] Serialized 129 byte log entry
0709 13:08:33.794509 (+   198us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:33.794654 (+   145us) write_op.cc:454] Released schema lock
0709 13:08:33.794704 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:33.795096 (+   392us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:33.795116 (+    20us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1875316,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":168111,"apply.run_cpu_time_us":23117,"apply.run_wall_time_us":511186,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62675,"num_ops":14,"prepare.queue_time_us":764373,"prepare.run_cpu_time_us":17668,"prepare.run_wall_time_us":112694,"raft.queue_time_us":223613,"raft.run_cpu_time_us":19077,"raft.run_wall_time_us":127035,"replication_time_us":428323,"thread_start_us":950925,"threads_started":4,"wal-append.queue_time_us":94303,"wal-append.run_cpu_time_us":19057,"wal-append.run_wall_time_us":2130092}]]}
W20260709 13:08:36.314628 21064 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.002s	sys 0.022s
I20260709 13:08:36.272881 27080 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:36.311493 20967 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:36.316071 21064 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.002s	sys 0.023s
W20260709 13:08:36.317212 21146 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.007s	sys 0.015s
W20260709 13:08:36.326789 21146 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.007s	sys 0.015s
W20260709 13:08:36.317392 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.005s	sys 0.029s
W20260709 13:08:36.327652 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.005s	sys 0.030s
I20260709 13:08:36.318305 27207 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:36.305601 27223 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:08:36.249419 27008 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.284199 26956 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:36.337123 20967 catalog_manager.cc:5697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 9 to 12, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 12 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:36.320390 27120 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.036s	sys 0.043s
W20260709 13:08:36.351208 21140 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.021s	sys 0.006s
W20260709 13:08:36.351787 21140 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.021s	sys 0.006s
I20260709 13:08:36.247803 27215 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.352710 27164 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:08:36.359390 26956 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.359476 27210 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.361207 27210 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.361819 26956 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.363539 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.368705 27008 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:36.369359 21110 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.029s	sys 0.044s
W20260709 13:08:36.369827 21110 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.029s	sys 0.044s
W20260709 13:08:36.348433 27120 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.036s	sys 0.043s
I20260709 13:08:36.373557 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.336179 27214 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:36.374044 20967 catalog_manager.cc:5697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:36.382385 21069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.004s	sys 0.020s
W20260709 13:08:36.382802 21069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.004s	sys 0.020s
W20260709 13:08:36.378773 21071 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.007s	sys 0.047s
W20260709 13:08:36.385578 21071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.007s	sys 0.047s
I20260709 13:08:36.386638 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2772) took 1052 ms. Trace:
I20260709 13:08:36.383983 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2765) took 1052 ms. Trace:
I20260709 13:08:36.387234 21071 rpcz_store.cc:276] 0709 13:08:35.333889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.335379 (+  1490us) service_pool.cc:224] Handling call
0709 13:08:35.335739 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.335996 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:35.766194 (+430198us) log.cc:844] Serialized 14 byte log entry
0709 13:08:36.386199 (+620005us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:36.386273 (+    74us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:36.386341 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.386429 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.386570 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":720672,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":127,"spinlock_wait_cycles":22656,"thread_start_us":1012128,"threads_started":2}
W20260709 13:08:36.381486 26899 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.664s	user 0.006s	sys 0.006s
W20260709 13:08:36.407989 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.030s	sys 0.058s
W20260709 13:08:36.422756 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.031s	sys 0.058s
I20260709 13:08:36.425107 20967 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:36.377388 27210 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:36.379629 27209 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.374918 27005 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.397009 21069 rpcz_store.cc:276] 0709 13:08:35.331573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.332210 (+   637us) service_pool.cc:224] Handling call
0709 13:08:35.332499 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.332736 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:35.765995 (+433259us) log.cc:844] Serialized 14 byte log entry
0709 13:08:36.383476 (+617481us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:36.383553 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:36.383610 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.383698 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.383914 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":701228,"prepare.run_cpu_time_us":1132,"prepare.run_wall_time_us":12708,"raft.queue_time_us":119,"thread_start_us":1007497,"threads_started":2}
W20260709 13:08:36.428297 21063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.017s	sys 0.031s
W20260709 13:08:36.434535 21063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.017s	sys 0.031s
I20260709 13:08:36.348544 27207 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:36.440152 21794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.026s	sys 0.012s
W20260709 13:08:36.440651 21794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.026s	sys 0.012s
I20260709 13:08:36.437590 20967 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 8 to 11, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 11 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:36.447126 21035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.008s	sys 0.019s
W20260709 13:08:36.447592 21035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.009s	sys 0.019s
W20260709 13:08:36.448263 21038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.008s	sys 0.018s
W20260709 13:08:36.448608 21038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.008s	sys 0.018s
I20260709 13:08:36.386919 27214 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.427310 26957 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:36.450845 21143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.010s	sys 0.022s
W20260709 13:08:36.473250 21143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.010s	sys 0.022s
I20260709 13:08:36.428951 27008 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.438674 27164 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.379257 27060 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:36.474305 21187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.018s	sys 0.017s
W20260709 13:08:36.480737 21187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.019s	sys 0.018s
I20260709 13:08:36.477926 20967 catalog_manager.cc:5697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:36.481046 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 709731us
W20260709 13:08:36.485184 21144 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.018s	sys 0.064s
W20260709 13:08:36.485702 21144 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.018s	sys 0.065s
W20260709 13:08:36.489074 21184 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.013s	sys 0.027s
W20260709 13:08:36.489485 21184 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.013s	sys 0.027s
I20260709 13:08:36.458459 27141 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:36.451604 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.016s	sys 0.012s
W20260709 13:08:36.447923 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 555831us
I20260709 13:08:36.499217 27251 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:08:36.504985 27236 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:08:36.500324 27073 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:36.505168 27251 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.506237 27073 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:36.457595 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.006s	sys 0.026s
W20260709 13:08:36.507879 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.007s	sys 0.026s
I20260709 13:08:36.470292 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1147) took 1220 ms. Trace:
I20260709 13:08:36.509980 26889 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:36.509073 21794 rpcz_store.cc:276] 0709 13:08:35.249527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.250593 (+  1066us) service_pool.cc:224] Handling call
0709 13:08:35.251070 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.251620 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:35.852576 (+600956us) log.cc:844] Serialized 14 byte log entry
0709 13:08:36.451071 (+598495us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:36.451146 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:36.470021 (+ 18875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.470105 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.470234 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1982,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":34861,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":13020,"thread_start_us":1136756,"threads_started":2}
I20260709 13:08:36.487967 27210 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.500422 27008 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.511029 27210 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.511622 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:36.513262 27008 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.514153 27174 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:36.514578 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.515038 27141 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.516072 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:36.517063 26881 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:36.517333 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:36.518332 27211 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.518603 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:36.508595 27073 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:36.518908 27211 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.518957 27209 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.519783 27020 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:36.520295 27209 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.520665 27020 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:36.522156 27211 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:36.522416 27209 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.510673 26889 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.507756 27226 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:36.524839 27060 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:36.528757 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.529322 26881 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:08:36.508543 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 756955us
I20260709 13:08:36.522755 27211 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.532311 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:36.532650 27211 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.533740 26881 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:36.534883 27174 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:36.535357 27234 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.535424 27174 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:36.536118 27234 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.536796 27174 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.538029 27174 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:36.539062 27234 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.526072 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.545704 27218 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.517361 27141 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:36.546420 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.546819 26900 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:36.439236 27005 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.561973 27005 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.547493 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 754) took 1601 ms. Trace:
I20260709 13:08:36.547091 26889 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:36.546478 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:36.563320 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.563511 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.564208 21794 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:36.550050 21231 tablet_copy_source_session.cc:215] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:36.479498 27027 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:36.552268 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.526901 27226 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.531700 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.573721 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:36.567996 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.016s	sys 0.012s
I20260709 13:08:36.490155 27138 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:36.575085 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:36.576181 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 693381us
I20260709 13:08:36.564352 21068 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
W20260709 13:08:36.585919 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 86026.8us
W20260709 13:08:36.449383 27217 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.010s	sys 0.017s
I20260709 13:08:36.510797 26863 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:36.568709 26485 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.032s	sys 0.056s
I20260709 13:08:36.579766 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.589358 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.590128 21231 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:54620 (request call id 751) took 2688 ms. Trace:
I20260709 13:08:36.591952 21139 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:36.563017 21213 rpcz_store.cc:276] 0709 13:08:34.892555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.892720 (+   165us) service_pool.cc:224] Handling call
0709 13:08:34.933535 (+ 40815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:35.231718 (+298183us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:35.246600 (+ 14882us) log.cc:844] Serialized 14 byte log entry
0709 13:08:35.770528 (+523928us) log.cc:844] Serialized 14 byte log entry
0709 13:08:35.832081 (+ 61553us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:35.853759 (+ 21678us) log.cc:844] Serialized 26 byte log entry
0709 13:08:35.853937 (+   178us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:35.853975 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:35.854060 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:35.932684 (+ 78624us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b180047b9c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f463f058f 00007f7f3f42f571 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd
0709 13:08:36.164725 (+232041us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b180047b9c0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:36.494082 (+329357us) raft_consensus.cc:1685] finished
0709 13:08:36.494123 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.494263 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":324089,"prepare.queue_time_us":2060,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"raft.queue_time_us":343793,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":19062,"spinlock_wait_cycles":23992448,"thread_start_us":412639,"threads_started":1,"wal-append.queue_time_us":768329}
I20260709 13:08:36.503070 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2773) took 1168 ms. Trace:
I20260709 13:08:36.558310 27234 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:36.597815 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.486382 26957 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.534898 27211 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.607131 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:36.599109 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:36.486114 27164 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.513281 27210 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:36.570005 26881 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:36.602857 21792 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:36.606433 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:36.615437 21212 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:36.595219 27005 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:36.589329 27114 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:36.609716 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:36.590047 27217 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.010s	sys 0.018s
I20260709 13:08:36.617139 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:36.608466 21214 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
W20260709 13:08:36.597832 26485 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.032s	sys 0.056s
I20260709 13:08:36.596622 21231 rpcz_store.cc:276] 0709 13:08:33.881421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.882452 (+  1031us) service_pool.cc:224] Handling call
0709 13:08:36.541601 (+2659149us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b54001252d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425263f7 00007f7f474bc036 00007f7f474c5a3e 00007f7f3ed8927b 00007f7f474c59ba 00007f7f474c0acc 00007f7f474bbc7b 00007f7f474b08d1 00007f7f465644b4 00007f7f46564442
0709 13:08:36.569241 (+ 27640us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5362464640}
I20260709 13:08:36.590996 26899 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:36.623081 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.622159 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.598217 21063 rpcz_store.cc:276] 0709 13:08:35.334211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.336528 (+  2317us) service_pool.cc:224] Handling call
0709 13:08:35.336831 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.337089 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:35.773053 (+435964us) log.cc:844] Serialized 14 byte log entry
0709 13:08:36.502701 (+729648us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:36.502757 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:36.502806 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.502876 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.503009 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14642,"prepare.queue_time_us":751851,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":74184,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":1043678,"threads_started":2}
I20260709 13:08:36.585479 26656 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:08:36.647146 21221 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:36.603255 27218 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:36.609205 21035 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:36.667323 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:36.667920 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 58794.2us
I20260709 13:08:36.617247 27005 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:36.618031 27008 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.667048 27247 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.678108 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.678440 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.678251 27247 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:36.679211 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.679785 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.680581 27247 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.680981 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.681632 21139 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:36.675601 27209 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:36.682174 27209 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.682554 27209 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:36.682787 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.683961 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.684544 21063 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:36.685031 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.685459 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.686283 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.686650 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.675751 26881 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.687841 26889 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.687801 26881 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.688493 26889 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.690447 26881 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:36.690613 26889 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.692003 27215 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.692862 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:36.685484 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.694530 27211 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:36.694736 27214 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.694953 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.696171 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:36.696286 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.691202 26889 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:36.697759 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.696882 26957 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:36.697678 26889 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.698741 27226 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.682551 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.699976 21139 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:36.700450 27164 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:36.700982 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.701342 27207 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.667224 27251 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.703176 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.613782 27212 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.603830 26863 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.698781 21063 raft_consensus.cc:2410] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:08:36.710163 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:36.711665 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:36.716189 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.716909 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.611016 27234 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.718040 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:36.719538 21141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.008s	sys 0.008s
W20260709 13:08:36.729010 21141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.008s	sys 0.008s
I20260709 13:08:36.719094 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.719254 21796 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
W20260709 13:08:36.725692 21145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.011s	sys 0.005s
I20260709 13:08:36.658447 26956 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:36.732136 21145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.012s	sys 0.005s
I20260709 13:08:36.676929 27005 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:36.738977 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.676097 27217 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:36.727998 21139 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
W20260709 13:08:36.717422 27228 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.001s	sys 0.046s
I20260709 13:08:36.653986 27114 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.715956 27227 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:36.751540 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.755061 21139 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:36.722522 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:36.745240 27251 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:36.733421 27141 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.758095 21799 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:36.758193 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3010) took 1321 ms. Trace:
I20260709 13:08:36.734081 27027 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:36.757714 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:36.763810 21070 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.007s	sys 0.040s
I20260709 13:08:36.668701 26899 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.704933 27212 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.730268 27008 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.764726 27251 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.783093 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:36.860049 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 137609us
I20260709 13:08:36.890832 27027 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.891031 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:36.757725 27247 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.891605 27256 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.891891 27247 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.813302 21141 rpcz_store.cc:276] 0709 13:08:35.437028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.437185 (+   157us) service_pool.cc:224] Handling call
0709 13:08:35.486008 (+ 48823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.486207 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:36.162030 (+675823us) log.cc:844] Serialized 14 byte log entry
0709 13:08:36.756955 (+594925us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:36.757033 (+    78us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:36.757111 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.757188 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.758131 (+   943us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8454,"prepare.queue_time_us":964002,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":4552,"thread_start_us":1209497,"threads_started":2}
I20260709 13:08:36.894454 27247 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:36.888865 27249 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:08:36.817178 27073 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:08:36.881201 27250 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.189s	user 0.002s	sys 0.000s
I20260709 13:08:36.895937 27073 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.898566 27123 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:36.797158 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.900003 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.741811 27080 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:36.813061 21070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.007s	sys 0.041s
I20260709 13:08:36.812232 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2778) took 1318 ms. Trace:
W20260709 13:08:36.757550 21035 consensus_peers.cc:597] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a0bb40f8e3428899935fdfa915dfd0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:36.769968 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:36.749900 27228 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.001s	sys 0.046s
W20260709 13:08:36.916008 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 163244us
I20260709 13:08:36.900677 21068 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:36.858856 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2780) took 1340 ms. Trace:
I20260709 13:08:36.910544 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.918505 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:36.889462 27251 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:36.929857 26889 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:36.930111 27050 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:36.930306 27008 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.930800 27050 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.931537 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.932374 21801 raft_consensus.cc:2410] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:08:36.932955 27050 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:36.934048 27255 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:36.934574 27255 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.935654 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.936270 21063 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:36.936872 27255 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:36.937863 27073 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:36.940274 27252 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:36.940735 27252 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.941058 27252 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:36.945816 26863 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:36.946396 26863 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.947257 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:36.948024 21800 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:36.949057 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:36.949210 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:36.864835 27248 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 13:08:36.864022 27242 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:08:36.962320 21138 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:36.963547 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:36.955206 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:36.891382 27027 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:36.899740 27141 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.917423 21065 rpcz_store.cc:276] 0709 13:08:35.518390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.587154 (+ 68764us) service_pool.cc:224] Handling call
0709 13:08:35.587541 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.587798 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:36.273214 (+685416us) log.cc:844] Serialized 14 byte log entry
0709 13:08:36.858414 (+585200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:36.858489 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:36.858550 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.858638 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.858789 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124416,"prepare.queue_time_us":594769,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":47,"raft.queue_time_us":88170,"spinlock_wait_cycles":31820672,"thread_start_us":1102494,"threads_started":2}
I20260709 13:08:36.960222 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:36.822563 27260 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:36.914146 27172 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.946962 26863 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.862061 27114 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:36.935021 27209 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.942863 27005 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:36.768065 26956 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.875761 27253 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:36.984020 21793 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:36.899147 27123 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:36.965845 27207 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:08:37.025739 21072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.015s	sys 0.040s
W20260709 13:08:37.026543 21072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.015s	sys 0.040s
W20260709 13:08:37.029480 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 47263.5us
I20260709 13:08:37.033373 27005 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.033653 26956 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.035519 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3008) took 1602 ms. Trace:
I20260709 13:08:37.035882 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:37.036888 21138 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:37.036074 21140 rpcz_store.cc:276] 0709 13:08:35.433276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.433458 (+   182us) service_pool.cc:224] Handling call
0709 13:08:35.482437 (+ 48979us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:35.498641 (+ 16204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:35.498676 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:35.611352 (+112676us) log.cc:844] Serialized 52 byte log entry
0709 13:08:36.352269 (+740917us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:36.352317 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.352386 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:37.035210 (+682824us) raft_consensus.cc:1685] finished
0709 13:08:37.035239 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.035444 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57451,"raft.queue_time_us":21662,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":19343,"thread_start_us":681074,"threads_started":1,"wal-append.queue_time_us":941862}
I20260709 13:08:37.038297 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:37.041028 27209 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:37.038870 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:37.039044 21138 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:36.762037 27217 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:37.046725 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:37.054566 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:37.024590 27253 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:36.947402 27256 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:37.058854 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.028622 27008 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:37.076126 27008 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:37.076570 27008 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:37.076980 27123 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.030777 27207 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:37.082275 21066 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.035s	sys 0.049s
W20260709 13:08:37.082655 21066 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.035s	sys 0.049s
I20260709 13:08:37.073686 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1842) took 1468 ms. Trace:
I20260709 13:08:37.071020 26889 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:37.055912 21069 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:37.073240 21065 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:08:37.082741 21146 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.009s	sys 0.043s
I20260709 13:08:36.990268 27172 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:08:37.089323 21146 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.009s	sys 0.043s
I20260709 13:08:37.055522 26956 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:37.072328 27209 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:08:37.095770 21797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.006s	sys 0.034s
I20260709 13:08:37.071475 27253 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:37.098004 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.099221 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2774) took 1617 ms. Trace:
W20260709 13:08:37.102779 21797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.007s	sys 0.034s
W20260709 13:08:37.096196 21147 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.007s	sys 0.037s
I20260709 13:08:37.104549 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:37.074704 27073 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:37.025915 27260 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:37.083973 27207 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:37.114142 21147 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.007s	sys 0.037s
W20260709 13:08:37.110282 21142 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.021s	sys 0.029s
I20260709 13:08:37.091686 26889 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:37.123798 21142 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.021s	sys 0.029s
W20260709 13:08:37.095937 27120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.011s	sys 0.035s
W20260709 13:08:37.127457 21144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.045s
I20260709 13:08:37.102622 27209 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:37.083616 21072 rpcz_store.cc:276] 0709 13:08:35.605343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.654114 (+ 48771us) service_pool.cc:224] Handling call
0709 13:08:35.669135 (+ 15021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.669395 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:36.288590 (+619195us) log.cc:844] Serialized 14 byte log entry
0709 13:08:37.072966 (+784376us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.073046 (+    80us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:37.073112 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.073195 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.073610 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61595,"prepare.queue_time_us":519989,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":40297,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":10631,"thread_start_us":1286902,"threads_started":2}
I20260709 13:08:37.113333 21066 rpcz_store.cc:276] 0709 13:08:35.481399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.481559 (+   160us) service_pool.cc:224] Handling call
0709 13:08:35.481932 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.496408 (+ 14476us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:36.258492 (+762084us) log.cc:844] Serialized 14 byte log entry
0709 13:08:37.098811 (+840319us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.098869 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:37.098924 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.098997 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.099154 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94847,"prepare.queue_time_us":625277,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":3671,"raft.queue_time_us":20416,"thread_start_us":1381433,"threads_started":2}
I20260709 13:08:37.107721 27138 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:37.148988 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:37.149453 21144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.000s	sys 0.045s
I20260709 13:08:37.148765 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:37.195838 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:37.147815 27120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.011s	sys 0.035s
I20260709 13:08:37.127449 27234 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.138046 26889 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:37.254067 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.107988 26956 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:37.216914 27209 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:37.137655 26899 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:37.256749 21064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.015s	sys 0.019s
I20260709 13:08:37.223773 27114 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.223410 26957 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:08:37.285328 21064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.015s	sys 0.019s
I20260709 13:08:37.148430 27261 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:37.226212 27227 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:37.286703 21187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.023s	sys 0.014s
I20260709 13:08:37.288913 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.290174 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2775) took 1808 ms. Trace:
I20260709 13:08:37.293934 21072 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
W20260709 13:08:37.295450 21187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.024s	sys 0.014s
I20260709 13:08:37.151190 27073 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:37.257565 27080 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:37.295315 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.538s (SENT) [suppressed 1 similar messages]
I20260709 13:08:37.290856 26957 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:37.310002 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.246474 27138 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:37.312161 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.291097 27261 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:37.289537 27212 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:37.321885 21110 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.022s	sys 0.027s
W20260709 13:08:37.322283 21110 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.022s	sys 0.027s
I20260709 13:08:37.330482 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.336668 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:37.298928 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.000s	sys 0.036s
W20260709 13:08:37.316869 21794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.007s	sys 0.030s
W20260709 13:08:37.350247 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.000s	sys 0.037s
W20260709 13:08:37.350416 21794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.007s	sys 0.030s
I20260709 13:08:37.319224 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.315737 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:37.321439 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.008s	sys 0.035s
W20260709 13:08:37.352222 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.008s	sys 0.035s
I20260709 13:08:37.318499 26957 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:37.320820 27218 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.000s
I20260709 13:08:37.293129 27114 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:37.316602 27257 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:37.350759 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 666385us
I20260709 13:08:37.367362 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:37.304617 27227 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:37.356319 27259 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.316s	user 0.001s	sys 0.000s
I20260709 13:08:37.381927 21221 raft_consensus.cc:2410] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:08:37.321714 27261 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:37.338500 27141 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:37.429790 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:37.435480 21212 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:08:37.437204 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.371688 27123 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:36.915896 21067 rpcz_store.cc:276] 0709 13:08:35.494304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.585382 (+ 91078us) service_pool.cc:224] Handling call
0709 13:08:35.585765 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.586042 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:36.282666 (+696624us) log.cc:844] Serialized 14 byte log entry
0709 13:08:36.811750 (+529084us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:36.811816 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:36.811871 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.811980 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:36.812157 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60423,"prepare.queue_time_us":550905,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":45202,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":84361,"thread_start_us":1100278,"threads_started":2}
I20260709 13:08:37.294406 21064 rpcz_store.cc:276] 0709 13:08:35.481795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.485132 (+  3337us) service_pool.cc:224] Handling call
0709 13:08:35.506896 (+ 21764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:35.507149 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:36.335240 (+828091us) log.cc:844] Serialized 14 byte log entry
0709 13:08:37.289612 (+954372us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.289678 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:37.289938 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.290001 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.290120 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2125,"prepare.queue_time_us":603843,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":54,"raft.queue_time_us":79,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":5341,"thread_start_us":1682119,"threads_started":2}
W20260709 13:08:37.454123 21112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.015s	sys 0.022s
W20260709 13:08:37.499822 27276 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:08:37.504102 27260 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:37.516745 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3009) took 2083 ms. Trace:
W20260709 13:08:37.459825 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.490890 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3024) took 1046 ms. Trace:
I20260709 13:08:37.545616 21145 rpcz_store.cc:276] 0709 13:08:36.444501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.950486 (+505985us) service_pool.cc:224] Handling call
0709 13:08:37.031073 (+ 80587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:37.096019 (+ 64946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:37.096066 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.096127 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:37.096176 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.490614 (+394438us) spinlock_profiling.cc:243] Waited 57.5 ms on lock 0x7b540036e054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:37.490665 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.490820 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132299520}
W20260709 13:08:37.522846 21112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.015s	sys 0.023s
I20260709 13:08:37.475301 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.525111 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:37.420212 27217 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:37.423091 26863 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.455170 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3241) took 1732 ms. Trace:
I20260709 13:08:37.463418 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:37.459863 27269 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:08:37.460378 27268 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 13:08:37.460865 27258 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:08:37.543679 21146 rpcz_store.cc:276] 0709 13:08:35.433644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.479635 (+ 45991us) service_pool.cc:224] Handling call
0709 13:08:35.500075 (+ 20440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:35.595879 (+ 95804us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:36.327424 (+731545us) log.cc:844] Serialized 14 byte log entry
0709 13:08:37.097572 (+770148us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:37.104015 (+  6443us) log.cc:844] Serialized 52 byte log entry
0709 13:08:37.104211 (+   196us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:37.104258 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.104351 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:37.516383 (+412032us) raft_consensus.cc:1685] finished
0709 13:08:37.516412 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.516655 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14766,"prepare.queue_time_us":900719,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":99138,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":33249,"thread_start_us":1346118,"threads_started":2,"wal-append.queue_time_us":983931}
I20260709 13:08:37.438707 27256 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:37.482128 27265 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:08:37.569208 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:37.491828 27267 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
W20260709 13:08:37.575373 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 100126us
I20260709 13:08:37.576682 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:37.372435 27257 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:37.553177 21143 rpcz_store.cc:276] 0709 13:08:35.722963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.723081 (+   118us) service_pool.cc:224] Handling call
0709 13:08:35.723469 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:35.757187 (+ 33718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:35.757219 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:35.757708 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:08:36.474025 (+716317us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:36.474085 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.474181 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:37.454936 (+980755us) raft_consensus.cc:1685] finished
0709 13:08:37.454958 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.455111 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5210,"raft.queue_time_us":5993,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":11023,"thread_start_us":715848,"threads_started":1,"wal-append.queue_time_us":1177207}
I20260709 13:08:37.449484 27172 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:37.584637 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:37.592628 27266 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.645s	user 0.004s	sys 0.005s
I20260709 13:08:37.593488 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:37.606704 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:37.596177 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.601004 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:37.607934 21035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.060s	sys 0.125s
I20260709 13:08:37.599300 21798 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
W20260709 13:08:37.608246 21035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.060s	sys 0.125s
W20260709 13:08:37.609202 21071 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.000s	sys 0.054s
I20260709 13:08:37.606859 21221 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
W20260709 13:08:37.612068 21071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.000s	sys 0.054s
I20260709 13:08:37.573726 26863 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:37.615252 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.578356 27260 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:37.618409 27073 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:37.618666 27257 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.619727 27257 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:37.620090 27257 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:37.620428 27257 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:37.623586 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.624260 21069 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:37.625903 27138 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:37.627255 27114 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:37.627985 27278 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.628535 27278 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:37.630666 27278 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:37.632382 27008 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.555s	user 0.004s	sys 0.007s
I20260709 13:08:37.633793 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3240) took 1911 ms. Trace:
I20260709 13:08:37.635519 27211 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:37.642226 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.636646 21144 rpcz_store.cc:276] 0709 13:08:35.722552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.722804 (+   252us) service_pool.cc:224] Handling call
0709 13:08:35.766231 (+ 43427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:36.486178 (+719947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:36.486225 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:36.492570 (+  6345us) log.cc:844] Serialized 52 byte log entry
0709 13:08:37.219989 (+727419us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:37.287251 (+ 67262us) log.cc:844] Serialized 14 byte log entry
0709 13:08:37.288949 (+  1698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.289025 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:37.486709 (+197684us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b1800360fa0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:37.633506 (+146797us) raft_consensus.cc:1685] finished
0709 13:08:37.633601 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.633734 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6241,"prepare.queue_time_us":67300,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2853,"raft.queue_time_us":49076,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":27260,"spinlock_wait_cycles":8234368,"thread_start_us":1429978,"threads_started":2,"wal-append.queue_time_us":1127067}
I20260709 13:08:37.642976 21068 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:37.634286 27209 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:37.642738 27228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.015s	sys 0.044s
I20260709 13:08:37.648036 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:37.648358 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.653563 21066 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:37.565191 27123 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:37.652434 21142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.009s	sys 0.045s
W20260709 13:08:37.655054 21142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.009s	sys 0.045s
I20260709 13:08:37.660336 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:37.662637 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3018) took 1220 ms. Trace:
I20260709 13:08:37.663087 21142 rpcz_store.cc:276] 0709 13:08:36.442438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.442580 (+   142us) service_pool.cc:224] Handling call
0709 13:08:36.442948 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:36.443189 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:37.127961 (+684772us) log.cc:844] Serialized 14 byte log entry
0709 13:08:37.662283 (+534322us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.662359 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:37.662412 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.662473 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.662582 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7011,"prepare.queue_time_us":69339,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"thread_start_us":1175800,"threads_started":2}
W20260709 13:08:37.657745 27228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.016s	sys 0.045s
I20260709 13:08:37.672554 21068 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:37.675364 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.677392 21146 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:08:37.684383 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.644743 27008 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:37.698318 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.674716 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2786) took 1082 ms. Trace:
W20260709 13:08:37.702982 21147 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.026s	sys 0.039s
W20260709 13:08:37.703346 21147 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.027s	sys 0.040s
I20260709 13:08:37.703507 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:37.676204 21216 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:37.706843 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:37.706000 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:37.645929 27211 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:37.701572 21063 rpcz_store.cc:276] 0709 13:08:36.592818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.936874 (+344056us) service_pool.cc:224] Handling call
0709 13:08:36.937167 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:37.094340 (+157173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:37.094379 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.094425 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:37.094463 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.674395 (+579932us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54000dc354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:37.674459 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.674646 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360975104}
I20260709 13:08:37.724936 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1845) took 1873 ms. Trace:
I20260709 13:08:37.731959 21070 rpcz_store.cc:276] 0709 13:08:35.851593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.851726 (+   133us) service_pool.cc:224] Handling call
0709 13:08:35.868193 (+ 16467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:35.885744 (+ 17551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:35.885791 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:35.906220 (+ 20429us) log.cc:844] Serialized 26 byte log entry
0709 13:08:36.915913 (+1009693us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:36.915953 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.916020 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:37.724619 (+808599us) raft_consensus.cc:1685] finished
0709 13:08:37.724651 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.724856 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":14038,"thread_start_us":994883,"threads_started":1,"wal-append.queue_time_us":1203420}
I20260709 13:08:37.748248 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3019) took 1305 ms. Trace:
I20260709 13:08:37.748757 21147 rpcz_store.cc:276] 0709 13:08:36.442761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.442890 (+   129us) service_pool.cc:224] Handling call
0709 13:08:36.470322 (+ 27432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:36.470532 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:08:37.121072 (+650540us) log.cc:844] Serialized 14 byte log entry
0709 13:08:37.747895 (+626823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.747951 (+    56us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:08:37.748009 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.748075 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.748183 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53561,"prepare.queue_time_us":42258,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"thread_start_us":1207606,"threads_started":2}
W20260709 13:08:37.751927 27271 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 13:08:37.752521 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46611.5us
I20260709 13:08:37.753569 27212 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:37.753866 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:37.754204 27212 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:37.751798 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45061.6us
I20260709 13:08:37.755868 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:37.757206 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.755144 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:37.763835 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 468526us
I20260709 13:08:37.661932 26863 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:37.658573 27260 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:37.772665 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:37.774564 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:37.777288 27063 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:37.787096 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:37.779327 21798 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:37.795727 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.742090 26956 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.768383 27141 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:37.902153 27141 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:37.902575 27141 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:37.891258 27260 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:37.909427 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:37.854758 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 74905.5us
I20260709 13:08:37.773720 27284 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:37.860535 27123 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:37.935551 27101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a0bb40f8e3428899935fdfa915dfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:37.864885 26863 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.867220 27227 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.869102 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.872352 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:37.873138 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:37.949486 26957 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.006s
I20260709 13:08:37.876062 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.877262 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.880066 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.882517 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:37.883703 27282 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:08:37.884027 27209 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:37.884925 27008 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.796904 27063 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:37.759562 27212 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:37.892347 27277 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:08:37.893123 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.893783 27211 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:37.895262 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:37.972348 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 77178.2us
W20260709 13:08:37.897826 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.900656 21063 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:37.901520 27207 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.763s	user 0.009s	sys 0.002s
I20260709 13:08:37.901731 27278 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:37.931982 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.933457 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.899068 27257 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:37.934304 27261 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:37.911437 27252 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.930s	user 0.005s	sys 0.006s
I20260709 13:08:37.959487 27080 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:37.936939 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:37.945127 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 93834.9us
W20260709 13:08:37.949810 27253 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.829s	user 0.012s	sys 0.001s
I20260709 13:08:37.969197 21216 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
W20260709 13:08:37.949693 27027 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.971s	user 0.010s	sys 0.002s
W20260709 13:08:37.950098 26889 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.659s	user 0.006s	sys 0.005s
I20260709 13:08:37.934504 21035 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:37.981371 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.981699 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.982228 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.988224 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:37.988569 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.023128 21213 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:08:37.988999 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.023931 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:37.989887 27141 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:37.990199 27172 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:38.025198 27172 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.025985 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3036) took 1322 ms. Trace:
I20260709 13:08:37.988998 27073 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:37.971342 27063 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:37.992455 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:38.029731 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 95303.7us
W20260709 13:08:38.030004 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 274908us
I20260709 13:08:38.030663 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:37.861810 27285 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:38.039249 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.976758 27050 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:38.020390 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:38.018951 20967 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.000s	sys 0.044s
W20260709 13:08:38.045142 20967 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.000s	sys 0.045s
I20260709 13:08:38.028606 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:37.952427 26957 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.036098 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:38.036339 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:37.976492 27278 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:37.977594 27247 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.046038 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.021118 27257 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.069427 27247 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.054191 27279 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:38.033563 26889 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.054392 27278 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:38.072933 21069 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:38.073772 26889 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.037724 27123 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:38.074540 27123 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:38.074939 27123 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:38.022406 27252 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.076632 26957 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.077206 27252 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.059542 27174 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:37.980214 27207 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.034080 27027 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.080631 27207 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:38.080912 27251 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.118s	user 0.009s	sys 0.003s
I20260709 13:08:38.081221 27027 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.081525 27251 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.083784 27261 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.081583 21068 raft_consensus.cc:2368] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15.
I20260709 13:08:38.036928 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:38.048478 27050 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:38.064945 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.033032 27253 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.071913 27279 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:38.079574 21140 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:38.032475 26899 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:38.026388 21145 rpcz_store.cc:276] 0709 13:08:36.703901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.600903 (+897002us) service_pool.cc:224] Handling call
0709 13:08:37.614089 (+ 13186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:37.614395 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:37.659099 (+ 44704us) log.cc:844] Serialized 14 byte log entry
0709 13:08:38.025613 (+366514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:38.025674 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:38.025726 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.025792 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:38.025915 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38697,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":71,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":366147,"threads_started":1}
I20260709 13:08:38.058017 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.044291 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3028) took 1477 ms. Trace:
I20260709 13:08:38.086624 21063 raft_consensus.cc:2368] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18.
I20260709 13:08:38.034390 27280 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:38.084877 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:38.089040 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.026688 27073 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:38.084751 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:38.091131 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:38.088531 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.093847 21139 rpcz_store.cc:276] 0709 13:08:36.566910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.940612 (+373702us) service_pool.cc:224] Handling call
0709 13:08:36.940859 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:36.976236 (+ 35377us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:36.976531 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:08:37.512445 (+535914us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b180035f380. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:08:37.953692 (+441247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.953741 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:37.953792 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.013234 (+ 59442us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b180035f380. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3fd2a6 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419
0709 13:08:38.043980 (+ 30746us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b5400279554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:38.044044 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:38.044214 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":130,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":58,"raft.queue_time_us":3600,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":15739,"spinlock_wait_cycles":89813504,"thread_start_us":437380,"threads_started":1,"wal-append.queue_time_us":1113573}
I20260709 13:08:38.097638 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.090711 27050 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:38.072384 27247 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.116303 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:38.110354 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.108963 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 716) took 1490 ms. Trace:
I20260709 13:08:38.088896 27207 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.125842 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.105168 21216 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:38.113704 21070 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:38.051438 27215 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:08:38.088472 27284 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.117731 27211 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:08:38.118840 27287 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:08:38.112901 21215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:38.135895 21215 raft_consensus.cc:606] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.136400 21215 raft_consensus.cc:614] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:38.093377 27279 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.121987 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:38.122641 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:38.123198 21145 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:38.123021 21794 rpcz_store.cc:276] 0709 13:08:36.618221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.618358 (+   137us) service_pool.cc:224] Handling call
0709 13:08:36.654257 (+ 35899us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:36.665032 (+ 10775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:36.665079 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:36.673171 (+  8092us) log.cc:844] Serialized 26 byte log entry
0709 13:08:37.351110 (+677939us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:37.351167 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.351267 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:38.108751 (+757484us) raft_consensus.cc:1685] finished
0709 13:08:38.108770 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:38.108895 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2697,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":23274,"thread_start_us":648456,"threads_started":1,"wal-append.queue_time_us":975977}
I20260709 13:08:38.124858 21765 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:38.125468 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.111778 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.130223 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.114835 27209 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:38.122627 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:38.127197 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 366324us
I20260709 13:08:38.146152 21146 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:38.146085 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:38.051757 27285 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.133723 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.105350 27172 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.145197 21142 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:38.151517 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.153432 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.154551 21064 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:38.127566 27027 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:38.158381 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.159870 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:38.165573 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.165757 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:38.148541 27218 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:38.168883 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.169883 27073 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:38.170636 27285 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.154659 27286 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:38.145071 27120 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.177577 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:38.155123 27247 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.161761 26863 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:38.142962 27207 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.190428 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.181383 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.192667 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.180778 27251 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:38.193065 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:38.193336 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 139780us
I20260709 13:08:38.191875 21147 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:38.164609 27261 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:38.196700 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.193423 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.174453 27252 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:38.202363 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.202764 21213 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:38.200317 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.200505 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.204576 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.206753 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.211046 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.212420 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.213044 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3264) took 1224 ms. Trace:
I20260709 13:08:38.213510 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.214121 21212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:38.214852 21212 raft_consensus.cc:606] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.216441 27008 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:38.218164 27299 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:08:38.213470 21144 rpcz_store.cc:276] 0709 13:08:36.988596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.662380 (+673784us) service_pool.cc:224] Handling call
0709 13:08:37.693333 (+ 30953us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:37.750719 (+ 57386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:37.750757 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:37.751249 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:08:38.017792 (+266543us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:38.017855 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.017952 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:38.212834 (+194882us) raft_consensus.cc:1685] finished
0709 13:08:38.212858 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:38.212986 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42059,"raft.queue_time_us":44657,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":266025,"threads_started":1,"wal-append.queue_time_us":375847}
I20260709 13:08:38.214339 21065 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:38.213063 21067 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:38.214260 27301 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:38.212373 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.225045 21066 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:38.219739 21212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:38.184719 26889 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:38.185702 27228 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:38.226121 21212 raft_consensus.cc:606] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.228804 21212 raft_consensus.cc:614] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:38.142623 27284 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.234040 21064 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:38.236239 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.197679 27141 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.248584 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2788) took 1644 ms. Trace:
I20260709 13:08:38.255939 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1212) took 1310 ms. Trace:
I20260709 13:08:38.203382 27261 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:38.202163 27252 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:38.256006 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.235008 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.270403 21111 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.271606 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.256006 21071 rpcz_store.cc:276] 0709 13:08:36.604499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.986116 (+381617us) service_pool.cc:224] Handling call
0709 13:08:36.986457 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:37.021785 (+ 35328us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:37.021819 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:37.030593 (+  8774us) log.cc:844] Serialized 26 byte log entry
0709 13:08:37.614377 (+583784us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:37.615655 (+  1278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.615755 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:38.248337 (+632582us) raft_consensus.cc:1685] finished
0709 13:08:38.248374 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:38.248522 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3602,"raft.queue_time_us":6034,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":20241,"spinlock_wait_cycles":1073920,"thread_start_us":543259,"threads_started":1,"wal-append.queue_time_us":731616}
I20260709 13:08:38.214869 27303 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:38.201519 27217 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:08:38.291492 21146 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:38.282747 27252 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.293129 21111 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.297796 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:38.298724 21799 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:38.256582 21801 rpcz_store.cc:276] 0709 13:08:36.945938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.956914 (+ 10976us) service_pool.cc:224] Handling call
0709 13:08:36.975316 (+ 18402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:36.980918 (+  5602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:36.980950 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:36.983519 (+  2569us) log.cc:844] Serialized 26 byte log entry
0709 13:08:37.692246 (+708727us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b180011b8a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:37.971607 (+279361us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:37.971649 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.971717 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:38.228032 (+256315us) raft_consensus.cc:1685] finished
0709 13:08:38.228051 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:38.255818 (+ 27767us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6571,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":32447,"spinlock_wait_cycles":4854528,"thread_start_us":278851,"threads_started":1,"wal-append.queue_time_us":1120993}
W20260709 13:08:38.303582 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 121740us
I20260709 13:08:38.305861 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:38.306808 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 114271us
I20260709 13:08:38.200884 27120 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.307317 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:38.274081 27261 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.308152 21147 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:38.307822 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.309381 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:38.309556 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:38.310050 21144 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:38.311673 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:38.312188 21139 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.313831 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.225207 27247 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:38.314945 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 126136us
I20260709 13:08:38.291702 27303 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:38.317834 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 119642us
W20260709 13:08:38.313735 27295 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.193s	user 0.001s	sys 0.001s
I20260709 13:08:38.324510 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.303488 27302 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:38.309474 27141 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:38.338022 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:38.338939 21143 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:38.313966 27253 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:38.352829 27297 log.cc:927] Time spent T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
W20260709 13:08:38.354866 27290 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 13:08:38.324110 27303 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.321106 27301 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:38.358904 21221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a473266451804da089da06e9c927e7ee"
dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:08:38.357134 27080 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.325476 27306 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.328156 27260 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:38.314186 27008 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:08:38.376802 21221 raft_consensus.cc:606] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:38.374055 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56296.3us
I20260709 13:08:38.383368 21221 raft_consensus.cc:614] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:38.381494 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 67792.4us
I20260709 13:08:38.342923 27217 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:38.384617 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.401136 27120 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.377430 27310 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:38.403867 27284 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.381361 27080 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:38.410040 21147 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:38.421628 27063 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.424256 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:38.423031 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:38.433327 21798 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:38.425181 21792 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:38.395553 27217 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:38.426365 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:38.430408 21212 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:38.430755 21066 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:38.431174 21139 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:38.431123 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.425784 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:38.403807 27310 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:38.435788 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 110189us
I20260709 13:08:38.437639 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:38.442260 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46062.6us
W20260709 13:08:38.443024 21110 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6beb666b5d504215a141add24a6d632f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:38.444275 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.444777 21142 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:38.444948 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 109657us
I20260709 13:08:38.451930 27050 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.453792 27261 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.455601 27063 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.456398 27217 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.457638 27217 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:38.383473 27306 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.462787 27306 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:38.440126 27310 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:38.465246 21799 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:38.466264 21797 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:38.437511 27284 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:08:38.467526 27212 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.469709 27212 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.472780 27080 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.477128 27080 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.480646 26899 raft_consensus.cc:993] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:38.481326 26899 raft_consensus.cc:1081] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:38.481992 21144 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:38.473114 27278 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.488499 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:38.488662 27278 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:38.492847 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.503693 21143 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
W20260709 13:08:38.510759 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 62691.9us
I20260709 13:08:38.512957 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.513499 21801 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
W20260709 13:08:38.513931 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 67937.9us
I20260709 13:08:38.494266 27050 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.517484 27252 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.518787 27312 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:38.540309 21066 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:38.541705 27252 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.549165 27123 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.532445 27261 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:38.527067 27050 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:38.557190 27310 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.557758 27247 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.559849 27279 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.568434 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:38.569583 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:38.572757 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:38.575441 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.576670 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.578024 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:38.581144 27317 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:38.581985 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:38.583820 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.584354 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:38.589709 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:38.596360 21072 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 22: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:38.592754 21071 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:38.559142 27207 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.604878 21065 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:38.608271 27281 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:08:38.617180 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 97914.1us
I20260709 13:08:38.606662 27215 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:38.604902 27251 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.619822 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:38.625378 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3263) took 1645 ms (client timeout 2077 ms). Trace:
I20260709 13:08:38.625437 21141 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:38.625773 21140 rpcz_store.cc:269] 0709 13:08:36.980011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.038562 (+ 58551us) service_pool.cc:224] Handling call
0709 13:08:38.625114 (+1586552us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54004254d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:38.625318 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2383232256}
I20260709 13:08:38.628561 27218 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:38.632447 27303 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.635418 27217 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:38.616047 27281 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:38.639425 27050 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:38.640074 27050 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:38.640168 27281 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.641295 27050 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.642182 27281 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:38.624886 27251 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.644793 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1050) took 1005 ms. Trace:
I20260709 13:08:38.645334 21069 rpcz_store.cc:276] 0709 13:08:37.640550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.646103 (+  5553us) service_pool.cc:224] Handling call
0709 13:08:37.659863 (+ 13760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:38.073452 (+413589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.644470 (+571018us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54003ca854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:38.644712 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949353728}
I20260709 13:08:38.646253 27251 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.648488 27123 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.650656 27279 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.652081 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:38.652595 27310 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:38.659427 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:38.661082 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.662091 27209 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:38.662636 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.664119 27209 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:08:38.669381 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67648.1us
I20260709 13:08:38.667196 27285 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:08:38.680114 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.685429 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:38.690040 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:38.691936 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:38.668648 27251 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:38.643143 27050 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.695376 21187 leader_election.cc:400] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15.
I20260709 13:08:38.695820 21187 leader_election.cc:403] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:38.671970 27303 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.688647 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:38.701776 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:38.702023 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:38.702633 21793 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:38.703343 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.674635 27209 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:38.687609 27215 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:38.707964 27253 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:38.710980 27215 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.695011 27251 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:38.684705 27286 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:38.696977 27318 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:38.697009 27123 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:38.722505 27318 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:38.723071 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 291994us
I20260709 13:08:38.709512 21187 leader_election.cc:400] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18.
I20260709 13:08:38.729820 21187 leader_election.cc:403] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:38.729907 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.707737 27228 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:38.706516 27209 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:38.735733 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.737936 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.738466 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.716640 27314 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.719759 27253 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:38.712901 27002 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:38.767784 27002 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:38.769212 27002 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.767882 27253 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.716027 27306 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:38.745203 27284 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:38.745445 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:38.771595 27284 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:38.746160 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.741091 27008 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:38.734515 27063 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:38.773062 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:38.747071 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 179644us
I20260709 13:08:38.775614 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.776988 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:38.747792 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.748098 27293 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:08:38.778152 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 97183.6us
W20260709 13:08:38.778975 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 264391us
I20260709 13:08:38.748621 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.748888 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.749049 27260 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:38.750801 27247 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.781147 27247 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.751518 27172 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:08:38.788484 21793 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:38.756135 27329 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:38.756506 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.757341 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.791702 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:38.792161 21069 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:38.789232 27063 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:38.758616 27218 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:38.759627 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.760268 27228 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:38.761116 27318 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.762383 21138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce55364f10c348819e7c63c08633b184"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:38.771708 27306 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:38.772672 27284 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.745873 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.750319 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3033) took 2047 ms. Trace:
I20260709 13:08:38.784708 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:38.800113 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 43690.6us
I20260709 13:08:38.754089 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:54634 (request call id 1120) took 6425 ms. Trace:
I20260709 13:08:38.803759 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.799743 21141 rpcz_store.cc:276] 0709 13:08:36.702919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.580836 (+877917us) service_pool.cc:224] Handling call
0709 13:08:38.625273 (+1044437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:38.625870 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.750148 (+124278us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54004254d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:38.750266 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2331490560}
I20260709 13:08:38.792428 21070 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:08:38.805677 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.802597 21035 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:38.808996 21214 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:08:38.806349 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.800998 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:38.783159 27326 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.180s	user 0.003s	sys 0.000s
I20260709 13:08:38.810324 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:38.815992 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:38.824096 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.824908 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:38.825381 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.817595 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.817574 27008 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:38.795555 27293 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:38.795063 27228 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.826023 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.841825 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.842374 21219 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:38.843175 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.844061 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.826825 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.832119 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:38.833061 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.836256 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.836719 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.849376 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.840883 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.850211 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.850525 21215 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:38.851559 21071 raft_consensus.cc:2368] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14.
I20260709 13:08:38.853566 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.851346 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.807621 27217 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:38.823714 21069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39498
I20260709 13:08:38.859339 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.863147 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.864490 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:38.865110 21069 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:38.866564 21069 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:38.859355 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.870491 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.870532 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:38.861014 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 416757us
W20260709 13:08:38.828378 27324 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:08:38.881117 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.825519 26863 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:38.833987 27073 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:38.852360 27063 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:38.860795 27141 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:38.873497 27328 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:38.831851 27008 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:38.837119 27293 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:38.828310 27302 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.856351 27284 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:38.887701 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:38.897648 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.887849 21145 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:38.863924 26956 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:38.835785 27247 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:38.905871 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.906996 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:38.887223 27312 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:38.908078 21113 leader_election.cc:400] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14.
I20260709 13:08:38.922081 21113 leader_election.cc:403] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:08:38.910706 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.886829 27303 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:38.897626 26863 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:38.911950 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 63260us
I20260709 13:08:38.882313 27123 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:38.915489 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:38.917796 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.921087 21070 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:38.935745 21140 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:38.935741 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.936244 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.930892 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:38.900701 27063 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.904487 27328 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:38.886086 27217 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.932152 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.944382 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:38.938086 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.832962 27306 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.894568 27257 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:08:38.916208 26956 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:38.856346 27251 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.951074 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.940748 27247 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:38.940862 27278 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:38.923369 27337 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:38.964944 27312 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:38.968230 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.968935 27328 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.970165 27328 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:38.955533 27008 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:38.976933 21212 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:38.962845 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:38.941515 27063 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:38.966945 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.946834 27212 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:38.964694 27310 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:38.935160 27302 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:38.992972 21140 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:39.001616 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69566.6us
I20260709 13:08:39.008832 21214 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:38.961004 26956 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:39.016913 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:38.962355 27141 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:38.975900 27257 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:39.025660 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:39.026067 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 313827us
I20260709 13:08:38.971236 27314 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15.
I20260709 13:08:39.036942 21145 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:39.037772 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.038062 21140 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 17. Ops: []
I20260709 13:08:38.808142 27101 rpcz_store.cc:276] 0709 13:08:32.371427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.438883 (+ 67456us) service_pool.cc:224] Handling call
0709 13:08:32.782899 (+344016us) ts_tablet_manager.cc:934] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
0709 13:08:37.907972 (+5125073us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:08:38.616392 (+708420us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:38.624162 (+  7770us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:08:38.753263 (+129101us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:08:38.753576 (+   313us) tablet_replica.cc:206] Creating consensus instance
0709 13:08:38.753900 (+   324us) ts_tablet_manager.cc:989] Replica registered.
0709 13:08:38.754007 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":90715,"tablet-copy.queue_time_us":235569,"thread_start_us":272764,"threads_started":2}
I20260709 13:08:39.038291 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:39.041533 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:39.016140 27247 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:39.027221 27318 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:39.055943 21140 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 12. Current term is 14. Ops: []
I20260709 13:08:39.036624 27257 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:39.050339 27174 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:39.065362 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:38.975476 27293 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.053474 27315 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:38.973124 26863 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.085842 27217 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:39.042385 27343 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:08:39.087532 27343 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:39.087962 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.088042 27343 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:39.089300 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.089885 21142 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:39.089383 27343 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.090654 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:39.091557 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.092203 21143 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:39.093041 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.042028 27123 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.094548 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.095945 21139 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:39.068037 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.073516 27302 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:39.075875 27338 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:08:39.103976 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:39.093885 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.099491 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:39.109601 21185 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:39.112336 27217 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:39.115574 27257 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.099891 21219 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:39.063097 27318 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.068893 27174 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:39.136207 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 179137us
I20260709 13:08:39.091152 27343 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:39.063372 27101 tablet_copy_client.cc:806] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:39.167390 27174 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:39.070014 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.168150 27101 tablet_copy_client.cc:670] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
W20260709 13:08:39.168497 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 98566us
I20260709 13:08:39.169435 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.170394 27252 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:39.173769 27227 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:39.127779 21069 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.104920 27172 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18.
I20260709 13:08:39.181851 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.183106 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:39.184789 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.178421 21069 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:39.071703 27256 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:08:39.167697 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:39.191401 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 63955.4us
I20260709 13:08:39.117544 27257 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:39.197471 21218 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:39.205521 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.205754 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:39.206260 27281 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.006s
I20260709 13:08:39.206817 21140 raft_consensus.cc:2368] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:39.209170 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:39.210237 21140 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:39.179062 27332 raft_consensus.cc:2764] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:08:39.133401 27217 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:39.215432 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:39.243649 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 58919.3us
I20260709 13:08:39.166025 27285 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:39.191174 27227 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:39.190501 27333 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:39.244421 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:39.179253 27247 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:39.070803 27335 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.128s
I20260709 13:08:39.196157 27310 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:39.256420 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.264744 21185 leader_election.cc:400] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:39.190606 27174 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.274912 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:39.265972 27300 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:08:39.248920 27318 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:39.252274 26863 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:39.276096 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:39.252494 27217 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:39.284075 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:39.290475 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.317507 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:39.301746 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.318725 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.273355 27281 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:39.292770 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:39.198896 27321 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:08:39.296097 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.285928 27247 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:39.285501 27172 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:39.299198 27328 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:39.302300 21216 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:39.320263 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.328840 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:39.345721 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.378192 27215 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:39.383592 21138 raft_consensus.cc:606] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:39.383912 27174 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:39.384122 21138 raft_consensus.cc:614] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:39.385936 27350 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:08:39.387949 27347 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:39.390772 27174 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:39.391422 27174 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:39.393023 27174 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.394461 27174 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:39.400000 21143 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:08:39.291642 26863 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:39.270602 27285 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.301038 27252 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:39.435442 27218 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:39.435952 27218 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:39.436328 27218 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:39.446426 27346 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:08:39.289927 27310 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.448292 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.449457 27310 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:39.449731 21146 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:39.424639 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 95893.6us
I20260709 13:08:39.319473 27217 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:39.457782 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:39.318632 27261 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:39.323710 27333 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:39.479239 21146 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:39.442941 27256 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:39.452400 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:39.484426 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:39.325513 27228 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:39.325688 27335 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:39.324770 27227 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:39.451680 26863 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:39.491499 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:39.497965 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:39.491504 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:39.329757 27063 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.493755 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:39.313905 26956 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.500461 21792 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:39.500072 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 179854us
I20260709 13:08:39.479514 26884 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:39.501420 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.502529 21794 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:39.506438 21145 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:39.507737 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.489485 27328 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:39.515960 21145 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:39.489298 27215 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:08:39.503759 27209 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.714s	user 0.012s	sys 0.000s
I20260709 13:08:39.538448 21144 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:08:39.538693 21145 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:39.518739 27252 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:39.541276 27335 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:39.542181 27278 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.002s
I20260709 13:08:39.542829 27278 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14.
I20260709 13:08:39.520730 27215 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:39.543803 27215 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:39.544138 27329 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:39.544621 27329 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:39.545524 27329 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.546631 27329 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:39.547849 27356 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:39.518726 26884 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.548636 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:39.548650 21140 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:39.550009 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:39.572216 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:39.575881 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:39.588364 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:39.596939 21147 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:39.573275 27310 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:39.599874 21141 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:39.574929 27281 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:39.610885 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:39.615938 21219 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:39.609436 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:39.617058 27228 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:39.618283 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:39.618916 21217 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:39.591248 27328 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:39.615813 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:39.619890 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:39.619203 21217 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:39.623433 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:39.624845 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:39.625258 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:39.626919 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.551687 27209 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:39.637374 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.603067 27310 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:39.640645 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 64830.3us
I20260709 13:08:39.588290 27218 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.613765 27332 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:39.648803 21213 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:39.645852 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.647547 21217 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.650689 21217 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:39.656025 27362 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:08:39.664718 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:39.670456 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.643857 27310 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:39.688083 21142 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:39.662421 27328 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:39.663184 27358 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:08:39.691345 21145 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:39.691690 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:39.698007 21213 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:39.695483 27321 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:39.662055 27285 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:39.702420 27321 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.704613 27209 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:39.693607 21217 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:39.719791 26956 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:39.699237 27328 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:39.710016 27247 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
W20260709 13:08:39.764253 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 102317us
I20260709 13:08:39.748281 27063 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:39.768764 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:39.784485 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:39.785125 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 148395us
I20260709 13:08:39.786265 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:39.789523 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.763702 27365 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:39.766772 27336 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:39.803223 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.812748 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:39.813194 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:39.814864 21792 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:39.818586 21213 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:39.762270 27328 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.819336 21068 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:39.780381 27247 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:39.822016 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.824882 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:39.821313 27256 raft_consensus.cc:2764] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:08:39.825842 27218 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:39.839735 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.840178 27218 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:08:39.840553 27367 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:08:39.840804 27218 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.841485 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:39.825781 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:39.844043 21217 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:39.827139 27378 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:39.846284 27218 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:39.844723 27301 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:39.851779 27256 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:39.852618 27256 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.855590 27301 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:39.856693 27301 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.860215 27256 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:39.861197 27365 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.842972 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:39.805662 27369 ts_tablet_manager.cc:933] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:39.867077 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:39.792367 27063 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:39.869076 21219 raft_consensus.cc:2368] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:08:39.821657 27227 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:39.877791 21145 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:39.879671 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:39.879067 21070 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:39.881906 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:39.898218 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:39.896049 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:39.831095 26863 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.902215 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:39.819229 27336 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:39.889253 27063 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.910212 21145 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:39.920742 27321 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:39.920619 27215 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:39.930353 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:39.932952 21065 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:39.986387 27215 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:39.988329 27380 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:08:39.893901 27217 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:39.903142 27310 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.041250 21065 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:08:40.072419 27217 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:39.929872 27063 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.067039 27329 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:40.082243 27369 tablet_copy_client.cc:323] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.194:39351
I20260709 13:08:40.087049 27328 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.081282 27301 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.088083 27301 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.089021 21157 tablet_copy_service.cc:140] P 5e5e65640c8b4f5ca7eadc6a14312c55: Received BeginTabletCopySession request for tablet 6beb666b5d504215a141add24a6d632f from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:50996)
I20260709 13:08:40.089640 21157 tablet_copy_service.cc:161] P 5e5e65640c8b4f5ca7eadc6a14312c55: Beginning new tablet copy session on tablet 6beb666b5d504215a141add24a6d632f from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:50996: session id = b70fc04de37e4239a8336534f0c9c0fb-6beb666b5d504215a141add24a6d632f
I20260709 13:08:40.090325 27301 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:40.090926 27373 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:40.091507 27373 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.098205 27310 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.099548 26863 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.100438 27365 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.100910 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.101009 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:40.111256 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 828) took 1025 ms. Trace:
I20260709 13:08:40.111682 21142 rpcz_store.cc:276] 0709 13:08:39.085990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.090440 (+  4450us) service_pool.cc:224] Handling call
0709 13:08:40.111044 (+1020604us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b540027a454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:40.111167 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118803840}
I20260709 13:08:40.112371 27336 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.112851 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:40.111706 21141 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:40.085687 27329 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.115929 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.116405 27329 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.118853 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:40.110960 27063 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.120817 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:40.121295 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.121270 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:40.121784 27373 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.122138 21065 raft_consensus.cc:2368] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:08:40.122570 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:40.120067 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:40.123440 21070 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:40.123497 21142 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:40.121455 21071 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:40.128317 21113 leader_election.cc:400] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:08:40.128728 21113 leader_election.cc:403] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:08:40.130703 21071 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:40.130801 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.132138 21142 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:08:40.136379 27251 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:40.132551 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.138854 26957 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:40.140764 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.141269 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:40.134027 21065 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:40.134778 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.148733 27141 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:40.149261 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:40.150110 27141 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:40.150484 27141 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:40.150839 27141 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:40.152841 27359 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.775s	user 0.015s	sys 0.033s
I20260709 13:08:40.156006 26884 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:40.160948 27302 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:40.162525 27063 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:40.162982 27063 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:40.164745 27333 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:40.165295 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.166028 21218 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:08:40.170403 27063 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.171597 27063 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:40.176396 27253 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:40.176947 27253 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:40.177966 27253 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.180054 26957 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:08:40.168639 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.185012 21800 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:40.186394 21794 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:40.177153 27388 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:08:40.187966 27253 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:40.190297 21797 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:08:40.192735 21216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 3112) took 1303 ms (client timeout 1727 ms). Trace:
W20260709 13:08:40.193161 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.198493 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:40.199505 21142 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:40.200557 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32968 (request call id 30) took 1435 ms. Trace:
I20260709 13:08:40.200810 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:40.200984 21138 rpcz_store.cc:276] 0709 13:08:38.761732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.762219 (+   487us) service_pool.cc:224] Handling call
0709 13:08:40.196729 (+1434510us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54002b3ed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42516853 00007f7f474e460a 00007f7f40a6a7d4 00007f7f40a6a762 00007f7f40a6a691 00007f7f40a6a60c 00007f7f40a698c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:40.196835 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1352368000}
I20260709 13:08:40.201555 21142 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:40.201925 27301 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:40.202670 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:40.204202 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:40.198259 21216 rpcz_store.cc:269] 0709 13:08:38.889250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.889389 (+   139us) service_pool.cc:224] Handling call
0709 13:08:40.192527 (+1303138us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54002eead4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:40.192643 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673965952}
I20260709 13:08:40.205987 27101 tablet_copy_client.cc:538] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:40.206745 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:40.207557 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:40.208686 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.206740 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:40.209676 21138 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:40.208254 21794 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 11.
I20260709 13:08:40.210896 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.212224 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:40.213660 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.210994 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:40.197353 26957 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:40.218492 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:40.210528 27370 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:40.228273 27370 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:40.216408 26899 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:40.228691 27370 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:40.228956 26899 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.217322 27374 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.216017 27389 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:40.250767 27389 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:40.251222 27309 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:40.252791 27174 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:40.223253 21138 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:08:40.226608 21111 leader_election.cc:400] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:08:40.193826 27141 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.227072 27301 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:40.262043 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3059) took 1066 ms. Trace:
I20260709 13:08:40.262509 21146 rpcz_store.cc:276] 0709 13:08:39.195189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.450457 (+255268us) service_pool.cc:224] Handling call
0709 13:08:39.450714 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.483301 (+ 32587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.261801 (+778500us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b540030b954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:40.261970 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65092864}
I20260709 13:08:40.257221 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.229719 27332 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:08:40.231879 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 825) took 1146 ms. Trace:
I20260709 13:08:40.233242 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3056) took 1038 ms. Trace:
I20260709 13:08:40.235215 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 826) took 1149 ms. Trace:
I20260709 13:08:40.235628 26899 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:40.268173 27301 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:40.262679 27325 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:08:40.271997 26899 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:40.272253 27325 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:40.272742 27325 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.273058 26899 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:40.273515 26899 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.274454 26899 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.251742 27389 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.236049 21145 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:40.238366 27315 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:40.240945 21794 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:40.241370 27217 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:40.277256 27315 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:40.277648 27217 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:40.278074 27217 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.278437 26899 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:40.242019 27172 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:40.242491 21793 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:40.242666 27375 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:40.279819 27318 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:40.280198 27318 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.280719 27375 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:40.281124 27375 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.281312 27318 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.281961 27375 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.282536 27318 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:40.246531 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 1952) took 1432 ms. Trace:
I20260709 13:08:40.283306 27375 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:40.279004 27217 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.284677 27217 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:40.283552 21220 rpcz_store.cc:276] 0709 13:08:38.813844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.814004 (+   160us) service_pool.cc:224] Handling call
0709 13:08:38.833871 (+ 19867us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:38.965308 (+131437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.965355 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:39.011530 (+ 46175us) log.cc:844] Serialized 513 byte log entry
0709 13:08:39.202141 (+190611us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:39.202180 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.152164 (+949984us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:40.246199 (+ 94035us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400101dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:40.246258 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:40.246306 (+    48us) raft_consensus.cc:1685] finished
0709 13:08:40.246319 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.246470 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321959936,"thread_start_us":149316,"threads_started":1,"wal-append.queue_time_us":245476}
I20260709 13:08:40.287761 27215 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.288407 27215 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.289685 21145 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:40.237345 27278 raft_consensus.cc:2955] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:08:40.292112 21146 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:40.242211 21792 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:40.222330 27251 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:40.293726 27251 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:40.294158 27251 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:40.248131 27374 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.257004 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:40.257185 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.256258 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:40.254937 27174 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.264616 21035 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:40.268639 21147 rpcz_store.cc:276] 0709 13:08:39.085015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.103725 (+ 18710us) service_pool.cc:224] Handling call
0709 13:08:40.231697 (+1127972us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b540030af54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:40.231813 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1088500608}
I20260709 13:08:40.269114 21143 rpcz_store.cc:276] 0709 13:08:39.085372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.092810 (+  7438us) service_pool.cc:224] Handling call
0709 13:08:40.235055 (+1142245us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54002b3ed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:40.235159 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704633088}
I20260709 13:08:40.268750 21140 rpcz_store.cc:276] 0709 13:08:39.194544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.211719 (+ 17175us) service_pool.cc:224] Handling call
0709 13:08:39.212244 (+   525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.552687 (+340443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.232986 (+680299us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b540030b454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:40.233180 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749025152}
I20260709 13:08:40.302178 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.236052 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:40.274024 27325 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.255239 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:40.302866 21800 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:08:40.307078 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71097.8us
I20260709 13:08:40.304287 27374 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.322638 21142 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:08:40.309856 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:40.323113 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 56427.5us
W20260709 13:08:40.323309 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 58775.8us
I20260709 13:08:40.310962 21801 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:40.328886 21140 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:40.331429 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:40.332089 21065 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:40.334401 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.337591 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:40.338501 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.339229 21213 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:40.301050 21068 raft_consensus.cc:2955] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:08:40.272008 27141 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.335155 21066 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:40.277658 27315 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.345209 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:40.345695 21142 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:40.308530 27281 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:40.322268 27329 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.324613 27212 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:40.335705 27209 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:40.323946 27285 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:08:40.311271 27172 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:08:40.268816 27370 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.365437 27141 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:40.373648 27101 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:08:40.379488 27215 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:40.381769 27251 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.355744 27228 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:08:40.323725 27321 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:08:40.364136 27209 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:40.365947 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:40.402488 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:40.402647 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 75156.9us
I20260709 13:08:40.426596 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.358404 27174 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.412495 21066 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:40.398448 21065 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
W20260709 13:08:40.425015 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 59129.8us
I20260709 13:08:40.352890 27388 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:08:40.437270 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:40.443630 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.403671 27141 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:40.425918 21800 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:40.363780 27329 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.433889 27209 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:40.401260 27281 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.433657 27215 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:40.385947 27309 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:40.358139 27364 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:08:40.455730 21797 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:40.448536 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:40.400161 27306 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:40.453672 27321 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:40.465099 27141 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:40.462709 27212 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:08:40.469323 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.451377 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.445106 27251 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.437951 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:40.452873 27377 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:08:40.446609 27123 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:08:40.478535 27306 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.423919 27389 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:08:40.485941 27306 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:40.486600 27329 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:40.487391 27329 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:40.489746 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.490403 21066 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:40.491614 21767 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:40.490849 21214 raft_consensus.cc:2955] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:08:40.494117 27218 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:40.494521 27218 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:40.494879 27218 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:40.496237 27370 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.471616 27378 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.497788 27378 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.502386 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:40.503136 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51880us
I20260709 13:08:40.507386 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:40.508296 21801 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:40.472662 27174 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:40.461272 27328 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:08:40.461042 27366 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:08:40.530368 27329 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:40.534176 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:40.567939 21065 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:08:40.571044 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 133287us
I20260709 13:08:40.460384 27315 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.538443 21141 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:40.549687 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:40.572988 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.574023 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:40.529780 27261 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:40.473410 27281 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.578392 27215 raft_consensus.cc:2764] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:08:40.589107 27281 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:40.583168 27328 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:40.566468 27301 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.592702 27301 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.477275 27309 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.594810 27301 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.595510 27309 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:40.599823 27396 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:40.550782 27399 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:08:40.600543 27396 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.527344 27321 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:40.601445 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51830.3us
I20260709 13:08:40.527031 27212 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:40.626681 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 52758.5us
I20260709 13:08:40.609553 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:40.537885 27374 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.632442 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:40.635270 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.646572 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.550028 27318 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:08:40.656478 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:40.657212 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.657876 21213 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:40.659745 27212 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.601682 27309 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:40.647704 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.602133 27378 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.628597 27120 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:40.646147 27375 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:40.605194 27343 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:08:40.672433 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.674212 27120 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.680644 27308 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:40.681124 27308 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.681289 27120 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:40.682178 27308 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.679494 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:40.687155 26899 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:40.687657 26899 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.688264 27317 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:40.688745 27317 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.688498 26899 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.689971 26899 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:40.689642 27317 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.691260 27309 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:40.648037 27374 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.692225 27309 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.693987 27374 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:40.573191 27329 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.694612 27141 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:40.700112 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.700587 21220 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:40.699579 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:40.669214 27396 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:40.707981 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:40.710095 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:40.710807 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 69882.2us
I20260709 13:08:40.712020 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.714968 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:40.718178 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:40.721630 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:40.721935 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:40.722988 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:40.672812 27375 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:40.724954 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.724951 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:40.722563 21801 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:40.726158 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.723989 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.729139 21765 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.730768 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:40.732023 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.678714 27278 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:60072 (request call id 1) took 4899 ms. Trace:
I20260709 13:08:40.701117 27399 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:40.725764 21139 raft_consensus.cc:2376] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14. Re-sending same reply.
I20260709 13:08:40.736686 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.734026 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:40.697037 27317 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:40.719480 21221 raft_consensus.cc:2376] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18. Re-sending same reply.
I20260709 13:08:40.710950 27321 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:40.734220 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:40.741742 21213 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:40.737591 21140 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:40.742709 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.744336 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:40.746169 21218 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:40.728972 27375 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:40.748693 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:40.749185 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:40.698127 27209 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.903728 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.737381 27374 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:40.738775 27261 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:40.914018 21065 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
W20260709 13:08:40.920809 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 285735us
W20260709 13:08:40.950615 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.957927 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:40.958199 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.960268 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.962185 21065 raft_consensus.cc:2393] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:40.921944 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:40.923853 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.924497 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.716106 27218 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:40.925593 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:40.982151 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60306.9us
I20260709 13:08:40.983000 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:40.916498 27080 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:40.981106 21066 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:40.965448 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.909165 27404 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:40.995246 27407 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:40.995443 27328 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:41.010488 27218 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:41.011366 21800 raft_consensus.cc:2368] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18.
I20260709 13:08:41.019569 21217 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
W20260709 13:08:41.023756 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.038237 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:41.042126 27413 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.913480 27374 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.062887 27374 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:40.985082 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:40.929721 27406 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:40.958609 27390 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.067317 27390 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.069877 27390 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:40.994606 27261 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.075778 27261 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.078042 27261 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:40.735838 27278 rpcz_store.cc:276] 0709 13:08:35.785562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.852555 (+ 66993us) service_pool.cc:224] Handling call
0709 13:08:36.131938 (+279383us) log.cc:844] Serialized 513 byte log entry
0709 13:08:37.483720 (+1351782us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b2c00010828. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e24a0a 00007f7f40e24693 00007f7f40e24631 00007f7f42558b52 00007f7f42558cf5 00007f7f42495691 00007f7f42475eaa 00007f7f4247cd53 00007f7f4247ccd9 00007f7f4247cc69
0709 13:08:37.591519 (+107799us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:37.737113 (+145594us) spinlock_profiling.cc:243] Waited 9.74 ms on lock 0x7b540006f020. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42495142 00007f7f42475eaa 00007f7f4247cd53 00007f7f4247ccd9 00007f7f4247cc69 00007f7f4247cc31 00007f7f4247bdfd 000000000038f354 000000000038f199 00007f7f3f42f475
0709 13:08:37.776890 (+ 39777us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b4c00071500. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f407e8a20 00007f7f407f9a29 00007f7f407fa13f 00007f7f407be2ab 00007f7f407ebf3c 00007f7f407ec3fe 00007f7f40a7950b 00007f7f42478381 00007f7f4247626b 00007f7f4247cd53 00007f7f4247ccd9
0709 13:08:40.672278 (+2895388us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20678,"raft.queue_time_us":345554,"raft.run_cpu_time_us":29363,"raft.run_wall_time_us":1867703,"spinlock_wait_cycles":128545920,"thread_start_us":545850,"threads_started":1,"wal-append.queue_time_us":636704,"wal-append.run_cpu_time_us":12915,"wal-append.run_wall_time_us":1892456}
I20260709 13:08:41.051066 27141 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:41.045938 27174 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.000s
I20260709 13:08:41.086175 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:41.074432 21215 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:40.992580 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:40.996760 27227 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:40.992378 27378 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:41.094681 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 102194us
I20260709 13:08:41.112576 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:41.082329 27321 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:41.131317 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:41.136855 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:41.105602 27218 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:41.104905 27209 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:41.133411 27328 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:41.082878 27413 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.141173 27407 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:41.134068 27378 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:41.174259 21112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.029s	sys 0.064s
W20260709 13:08:41.174680 21112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.029s	sys 0.064s
I20260709 13:08:41.179143 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:41.166456 27218 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:41.201792 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:41.202709 21071 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:41.205665 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:41.206429 21068 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:41.208257 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 282758us
I20260709 13:08:41.210285 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:41.209267 27212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.009s	sys 0.016s
I20260709 13:08:41.199995 27218 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:41.229507 27329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.010s	sys 0.022s
W20260709 13:08:41.227319 27212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.009s	sys 0.016s
W20260709 13:08:41.237645 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 100830us
I20260709 13:08:41.230547 27321 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.241245 21068 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:41.245805 21767 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:41.242200 21143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.017s	sys 0.024s
W20260709 13:08:41.240406 27329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.010s	sys 0.023s
W20260709 13:08:41.248054 21143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.017s	sys 0.024s
I20260709 13:08:41.250821 27413 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.250723 27212 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:41.259403 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49210.7us
W20260709 13:08:41.282039 21018 auto_rebalancer.cc:629] Failed to schedule move for tablet c20ca56847f64504a85694d10f47558e: Illegal state: Replica 5e5e65640c8b4f5ca7eadc6a14312c55 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:41.295485 27008 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.296206 27008 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.298578 27008 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:41.299428 27424 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.300032 27424 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.301460 27374 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.302184 27424 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.275213 27329 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:41.303256 27329 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:41.304239 27329 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.304157 27375 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.248836 27321 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.280656 27228 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:08:41.296217 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:41.277086 27212 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:41.374810 21018 auto_rebalancer.cc:629] Failed to schedule move for tablet 5196cb9fb3c1488180a6421083b18ff9: Illegal state: Replica 5e5e65640c8b4f5ca7eadc6a14312c55 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:41.412972 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:41.382325 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:41.383852 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:41.387250 21071 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:08:41.309491 27317 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.004s
I20260709 13:08:41.259807 27209 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:41.280019 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:41.390514 27227 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.379454 27218 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.312203 27406 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:41.404111 27325 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:41.307233 27375 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:41.316195 27321 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:41.454653 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.472525 27406 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.467783 27120 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.474997 27120 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.477995 27120 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:41.478644 27120 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.479125 27120 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.485685 27120 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.459510 21065 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
W20260709 13:08:41.464326 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 188866us
I20260709 13:08:41.546686 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:41.468331 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1250) took 1235 ms. Trace:
I20260709 13:08:41.491525 27375 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.547434 21797 rpcz_store.cc:276] 0709 13:08:40.233218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.233365 (+   147us) service_pool.cc:224] Handling call
0709 13:08:40.233691 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.475397 (+241706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.468080 (+992683us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54005d5f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:41.468267 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498753792}
I20260709 13:08:41.460345 27002 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.493253 27374 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:41.495347 27212 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.496304 27315 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:41.420073 21068 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:41.540253 27321 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.540488 27378 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.466065 27384 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.469244 27424 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:41.461674 27417 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.550916 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:41.553829 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:41.555462 21144 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:08:41.551714 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:41.574994 21145 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:08:41.560442 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:41.575217 21146 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:41.564073 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3308) took 1006 ms. Trace:
I20260709 13:08:41.575834 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:41.544548 27406 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:41.562152 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:41.570441 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:41.576396 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:41.469882 27227 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.584642 27227 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.585502 27396 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:41.585965 27396 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.586298 27396 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:41.587401 27228 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:41.591470 27424 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:41.574121 27384 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.595733 27384 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.564761 27252 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:41.579145 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:41.596964 27396 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.599525 27209 raft_consensus.cc:2764] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:08:41.599658 27396 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:41.600914 27209 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:41.580300 21801 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:41.570020 27321 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.606920 27321 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:41.574290 27172 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.582098 27329 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:41.607874 27172 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.582185 27417 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.566917 27002 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.610041 27172 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:41.563690 27375 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.610276 27002 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.617509 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:41.620198 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:41.620774 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.622090 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.622711 21221 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:41.622992 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:41.623196 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:41.623576 21144 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:41.623896 21146 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:41.624904 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:41.625381 21146 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:41.601722 27209 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:41.629747 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:41.630344 21146 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:41.631305 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.631518 21185 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:41.632572 21218 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:41.633601 27227 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.579063 27378 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.636366 21217 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:41.641455 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:41.641780 21145 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:41.637302 21801 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:41.643666 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:41.610169 27417 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.646045 27396 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:41.645421 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:41.634153 27227 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.629010 27375 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:41.629444 27388 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.655107 21144 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:41.662789 21157 tablet_copy_source_session.cc:215] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:41.663025 27388 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.665853 27388 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:41.632906 27209 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:41.647527 27261 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:41.644588 27373 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.660244 27396 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.675173 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:41.659420 27302 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:41.678805 21069 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:41.674734 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:41.685400 21071 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:41.670873 27396 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:41.692371 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:41.693071 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:41.645148 27378 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.693920 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:41.667655 21157 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:50996 (request call id 838) took 1579 ms. Trace:
I20260709 13:08:41.669021 27002 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.668373 27397 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:41.671695 27261 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:41.698848 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 57461.6us
I20260709 13:08:41.700001 21071 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:41.703508 27261 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:41.704255 27378 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.700954 27397 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.707818 27174 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:41.678761 27373 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.737848 27101 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:41.738041 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:41.738760 27373 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.739321 27101 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:08:41.739506 27174 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:41.739827 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 844) took 1017 ms. Trace:
I20260709 13:08:41.691224 27407 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.712144 27227 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:41.712649 27209 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:41.750233 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:41.750494 21794 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:41.714253 27308 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:41.697490 27329 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:41.716187 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3311) took 1083 ms. Trace:
I20260709 13:08:41.717000 27002 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.241703 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:41.726850 27396 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.731025 21216 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:41.740195 21138 rpcz_store.cc:276] 0709 13:08:40.722840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.729327 (+  6487us) service_pool.cc:224] Handling call
0709 13:08:40.737948 (+  8621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:40.738386 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.738426 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:40.738963 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:08:41.147940 (+408977us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:41.147982 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.148050 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.285487 (+137437us) raft_consensus.cc:1685] finished
0709 13:08:41.285511 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.736016 (+450505us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b540027a450. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:41.739757 (+  3741us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10720,"raft.queue_time_us":213487,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":7650,"spinlock_wait_cycles":18956032,"thread_start_us":217680,"threads_started":1,"wal-append.queue_time_us":505734}
I20260709 13:08:41.769659 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:41.748714 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:41.697083 27302 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:41.776430 21793 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:08:41.740247 27101 ts_tablet_manager.cc:1403] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 1.367s	user 0.044s	sys 0.013s
I20260709 13:08:41.772439 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.773891 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.736892 27397 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:41.736232 27228 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:41.808636 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:41.811465 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:41.812100 21214 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:41.805820 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.806195 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:41.764757 21141 rpcz_store.cc:276] 0709 13:08:40.632952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.633177 (+   225us) service_pool.cc:224] Handling call
0709 13:08:40.633628 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:40.695813 (+ 62185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.695843 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:40.696294 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:08:40.750125 (+ 53831us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:40.750164 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.715774 (+965610us) spinlock_profiling.cc:243] Waited 61.8 ms on lock 0x7b540032c8d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:41.715826 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.715878 (+    52us) raft_consensus.cc:1685] finished
0709 13:08:41.715892 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.716124 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9467,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":13061,"spinlock_wait_cycles":142175488,"thread_start_us":53447,"threads_started":1,"wal-append.queue_time_us":339498}
I20260709 13:08:41.806586 21215 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:41.814210 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.777629 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:41.820446 21072 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:41.830806 21217 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:08:41.831243 21217 pending_rounds.cc:85] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 12
I20260709 13:08:41.831529 21217 pending_rounds.cc:107] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:08:41.831761 21217 raft_consensus.cc:2889] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:41.696630 21157 rpcz_store.cc:276] 0709 13:08:40.088300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.088753 (+   453us) service_pool.cc:224] Handling call
0709 13:08:41.661316 (+1572563us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002968d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425263f7 00007f7f474bc036 00007f7f474c5a3e 00007f7f3ed8927b 00007f7f474c59ba 00007f7f474c0acc 00007f7f474bbc7b 00007f7f474b08d1 00007f7f465644b4 00007f7f46564442
0709 13:08:41.667524 (+  6208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2372881792}
I20260709 13:08:41.837823 21215 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:41.829104 27443 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.840075 27443 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.811551 27375 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:41.840989 27375 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.842238 27443 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:41.843096 27002 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:41.846879 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 76318us
I20260709 13:08:41.854720 21142 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:08:41.854839 21214 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:41.836519 27432 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:41.861317 27432 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.861752 27432 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:41.828114 27209 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:41.863230 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.864588 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.777956 27332 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:41.865557 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.866568 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:41.812817 27416 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.779516 27310 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:41.740499 27174 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.777712 27211 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:08:41.777452 27329 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.871558 21221 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:41.769040 27407 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.843096 27375 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:41.882222 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:41.884809 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 115215us
I20260709 13:08:41.905220 21217 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:41.890406 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:41.901883 27302 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.871364 27364 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:41.907625 21071 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:41.910715 21220 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:41.920042 21797 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:41.922150 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:41.924070 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:41.886795 27321 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:41.930212 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 161588us
I20260709 13:08:41.891794 27002 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:41.927102 27375 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:41.927440 27317 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:41.884272 27416 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:41.887612 27310 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:41.947571 27424 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:08:41.934597 27002 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.939850 27328 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:41.946520 27452 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.954643 27396 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:41.973502 27416 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:41.932384 27364 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:41.986871 21219 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:08:41.959729 27002 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:41.994037 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49717.8us
I20260709 13:08:42.026703 27451 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:42.027438 27451 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.030206 27451 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.043129 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:42.043608 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.043994 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.044555 21065 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:42.044878 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.045302 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.045430 21070 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:42.045849 21065 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:42.047686 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.048810 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.050355 21067 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:41.963542 27452 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.051903 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.053252 27257 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:41.995781 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:42.104936 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.106277 21138 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:42.108368 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:41.986593 27416 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:41.986931 27317 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:42.133541 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.155442 21138 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:42.053790 27257 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.128432 27443 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:42.139447 27389 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.141733 27449 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:08:42.141824 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 146190us
W20260709 13:08:42.024497 27420 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:08:42.158215 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:41.963747 27328 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:42.159343 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 234054us
I20260709 13:08:41.999140 27407 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.168030 21184 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:42.053838 27452 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.173799 27396 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:42.174422 27396 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:42.175062 27257 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:42.161432 27174 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:42.175398 27396 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.176055 27174 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.178624 27174 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:42.166222 27449 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:42.179699 27449 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:42.112257 27378 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:42.155154 27008 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:08:42.247160 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 293621us
I20260709 13:08:42.247983 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.248764 21142 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:08:42.248994 21213 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:42.254030 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.190763 27433 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:42.258147 21142 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:42.259243 21069 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:42.162003 27317 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.130199 27101 raft_consensus.cc:359] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:42.167704 27443 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.159578 27416 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.266048 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.279304 21142 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:42.170473 27389 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.268745 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.269297 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.282752 21141 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:42.282991 21071 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 12: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:42.270934 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.280563 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:42.245572 27452 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:42.256031 27329 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.251533 27325 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:42.244638 27008 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.270177 27101 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:08:42.233760 27378 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.289314 26956 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:42.291894 27008 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.294229 27002 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.296489 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:42.307358 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:42.299077 27328 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.300612 21071 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:42.308229 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:42.314738 27461 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:42.315619 27461 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.316083 27461 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:42.265033 27433 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.285455 27451 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:08:42.314651 27101 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:42.344599 27101 ts_tablet_manager.cc:1434] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.525s	user 0.011s	sys 0.000s
W20260709 13:08:42.344980 27101 ts_tablet_manager.cc:1467] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Tablet startup took 529ms
I20260709 13:08:42.316484 27423 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:42.323869 27452 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.299952 27325 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.341394 27328 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:42.313495 26956 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:42.346511 27397 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.519s	user 0.005s	sys 0.005s
W20260709 13:08:42.372359 27419 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.167s	user 0.005s	sys 0.005s
I20260709 13:08:42.340411 27285 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:42.383596 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:42.346846 27384 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:42.354909 27423 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:42.395330 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:42.376469 27002 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.376518 27228 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:42.413441 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.414352 27253 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:42.429648 27253 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:08:42.383877 27285 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:42.387825 27141 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:42.388307 27407 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.388512 27384 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.414080 27228 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:42.384948 27374 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 13:08:42.345422 27101 ts_tablet_manager.cc:1469] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Trace:
0709 13:08:32.371427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.438883 (+ 67456us) service_pool.cc:224] Handling call
0709 13:08:32.782899 (+344016us) ts_tablet_manager.cc:934] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
0709 13:08:37.907972 (+5125073us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:08:38.616392 (+708420us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:38.624162 (+  7770us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:08:38.753263 (+129101us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:08:38.753576 (+   313us) tablet_replica.cc:206] Creating consensus instance
0709 13:08:38.753900 (+   324us) ts_tablet_manager.cc:989] Replica registered.
0709 13:08:38.754007 (+   107us) inbound_call.cc:177] Queueing success response
0709 13:08:40.373084 (+1619077us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:08:41.827440 (+1454356us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:08:41.838158 (+ 10718us) tablet_replica.cc:261] Starting instrumentation
0709 13:08:41.838831 (+   673us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":90715,"mutex_wait_us":9831,"raft.queue_time_us":4543,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2136,"tablet-copy.queue_time_us":235569,"tablet-download-pool.queue_time_us":185,"tablet-download-pool.run_cpu_time_us":1219,"tablet-download-pool.run_wall_time_us":1037025,"thread_start_us":272764,"threads_started":2}
I20260709 13:08:42.414289 26884 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.423918 27314 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.385010 27397 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.398783 27423 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:42.468546 27432 consensus_meta.cc:287] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.606s	user 0.011s	sys 0.000s
I20260709 13:08:42.469018 27432 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.472817 27397 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:42.459442 27317 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:42.475620 21214 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:42.478788 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.479334 27423 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.485464 27228 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.486824 27228 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:42.481261 27432 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.487164 27423 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.495220 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:42.497542 21220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 775) took 1788 ms (client timeout 1946 ms). Trace:
W20260709 13:08:42.498027 21220 rpcz_store.cc:269] 0709 13:08:40.709663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.717858 (+  8195us) service_pool.cc:224] Handling call
0709 13:08:42.497360 (+1779502us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540034c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:42.497479 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2467588352}
I20260709 13:08:42.503911 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1926) took 1505 ms. Trace:
I20260709 13:08:42.504491 21066 rpcz_store.cc:276] 0709 13:08:40.998216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.998358 (+   142us) service_pool.cc:224] Handling call
0709 13:08:40.998676 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:41.138724 (+140048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:41.138772 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:41.138832 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:41.138886 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.503563 (+1364677us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54000f9e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:42.503633 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:42.503822 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321466240}
W20260709 13:08:42.506562 21216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 1987) took 1849 ms (client timeout 2136 ms). Trace:
W20260709 13:08:42.507102 21216 rpcz_store.cc:269] 0709 13:08:40.657088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.657196 (+   108us) service_pool.cc:224] Handling call
0709 13:08:42.506341 (+1849145us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400102554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:42.506491 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2427152640}
W20260709 13:08:42.511600 21800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 746) took 1790 ms (client timeout 2086 ms). Trace:
W20260709 13:08:42.512140 21800 rpcz_store.cc:269] 0709 13:08:40.721245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.721375 (+   130us) service_pool.cc:224] Handling call
0709 13:08:42.511338 (+1789963us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54006c11d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:42.511501 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664549760}
I20260709 13:08:42.457818 27407 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.517998 27407 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.455605 27384 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:42.461720 27469 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:42.523911 27469 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.526561 27469 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:42.463835 26884 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.530707 26884 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:42.464102 27314 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.532989 27310 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.533648 27314 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:42.457823 27321 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:42.535365 27321 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.537569 27321 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:42.539760 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.540885 27449 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.542100 27364 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.542934 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:42.543092 27172 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.543581 21797 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:42.543926 26884 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:42.544505 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:42.544423 26884 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.545111 21794 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:42.545938 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:42.546589 27209 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.671s	user 0.007s	sys 0.000s
I20260709 13:08:42.546671 26884 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:42.547557 27398 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:42.548033 27378 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:42.548676 27416 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:42.549520 21799 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:42.549829 27452 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:42.551632 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 72018.4us
I20260709 13:08:42.548079 27398 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.554590 27398 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:42.555397 27443 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.555932 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.556318 21799 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:08:42.557442 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:42.466627 26956 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.563654 27285 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.569820 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:42.568976 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.571136 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.572603 21215 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:42.573712 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:42.574245 21215 raft_consensus.cc:2376] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13. Re-sending same reply.
I20260709 13:08:42.570406 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.576597 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.578634 27433 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.579550 27325 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:42.580922 27261 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.847s	user 0.011s	sys 0.000s
I20260709 13:08:42.581372 27261 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.543582 27172 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.575802 21213 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:42.592228 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.593098 21072 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:42.592228 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.596098 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.599995 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.602175 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:42.605115 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:42.605597 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.606302 21217 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:42.575178 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.611162 21110 leader_election.cc:400] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18.
I20260709 13:08:42.611837 21110 leader_election.cc:403] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:42.612167 21066 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:42.612311 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:42.613111 21799 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 12: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:42.612960 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:42.613806 21795 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:42.615142 21213 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:42.619737 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:42.623376 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.623965 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:42.624521 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.622609 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.625941 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:42.626427 21221 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:42.626587 21145 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:42.627668 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.628391 21144 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:42.629385 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.629560 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.630177 21146 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
W20260709 13:08:42.618743 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 205117us
I20260709 13:08:42.632117 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:42.632673 21067 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:08:42.637105 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:42.638175 21066 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:42.639001 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:42.654759 21146 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:42.639254 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.639015 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:42.640831 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:42.659176 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:42.660247 21064 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:08:42.662678 21799 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:42.543186 27449 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:42.643854 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.668712 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:42.670161 21800 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:42.670469 27390 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:42.671142 27369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beb666b5d504215a141add24a6d632f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:42.620268 27398 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:42.633915 27378 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:42.633486 27325 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:42.645772 27002 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:42.647665 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.649852 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:42.651051 27476 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:08:42.650334 27462 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.653272 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.664687 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.634146 27416 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:42.699926 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:42.673810 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:42.695652 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:42.701328 21800 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:42.674468 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.704754 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.706446 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.717388 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.702162 27462 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.677863 27398 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:42.730355 27398 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:42.704977 27416 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.689193 27461 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.681980 27325 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.748281 27325 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:42.697099 27378 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:42.787771 27141 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:42.788044 27461 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.793445 27285 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:42.793859 26957 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:42.795449 27008 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.796499 27063 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.679337 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:42.680222 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:42.671473 27209 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:42.640583 27310 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18.
I20260709 13:08:42.682850 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:42.799261 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 118132us
I20260709 13:08:42.799825 27389 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.801131 27374 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:42.801703 27374 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.802145 27374 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:42.802076 27310 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:42.804441 27310 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:42.804858 27172 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:42.805580 27310 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:42.810657 27310 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:42.811174 27378 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:08:42.687500 27002 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.604290 27399 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:08:42.811735 27310 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.794679 26957 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:42.813293 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:42.814296 21142 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:41.576143 21143 rpcz_store.cc:276] 0709 13:08:40.557921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.558110 (+   189us) service_pool.cc:224] Handling call
0709 13:08:40.561883 (+  3773us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:40.570975 (+  9092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.571005 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:40.571640 (+   635us) log.cc:844] Serialized 52 byte log entry
0709 13:08:41.256702 (+685062us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:41.544181 (+287479us) log.cc:844] Serialized 14 byte log entry
0709 13:08:41.563778 (+ 19597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.563852 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.563892 (+    40us) raft_consensus.cc:1685] finished
0709 13:08:41.563903 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.564024 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26462,"raft.queue_time_us":30772,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":44667,"thread_start_us":855064,"threads_started":2,"wal-append.queue_time_us":667246}
I20260709 13:08:42.816196 27002 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.817438 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.817725 27462 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.818199 21143 raft_consensus.cc:2368] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:42.799299 21139 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:42.819947 27416 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:42.820521 27261 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:42.822067 21140 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:08:42.823014 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:42.824481 21142 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:42.827421 27310 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:42.829080 27457 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.572s	user 0.002s	sys 0.001s
I20260709 13:08:42.829078 21142 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 13:08:42.852818 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 173552us
I20260709 13:08:42.853811 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:42.854806 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:42.855517 27328 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:42.856442 27328 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:42.856886 27328 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.857299 27328 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:42.862077 21219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 777) took 2133 ms (client timeout 2144 ms). Trace:
W20260709 13:08:42.862577 21219 rpcz_store.cc:269] 0709 13:08:40.731792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.732197 (+   405us) service_pool.cc:224] Handling call
0709 13:08:42.861856 (+2129659us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54004439d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:42.861980 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817447168}
I20260709 13:08:42.698859 21216 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:42.866595 27218 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:42.867061 27218 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:42.796103 27008 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.867458 27218 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:42.797063 27063 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:42.788546 27141 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:42.870726 27315 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.873291 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:42.676442 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.878624 21213 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:42.686054 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.886654 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:42.907948 21064 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:42.914062 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:43.041755 21139 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:42.964125 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:43.042886 21142 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:43.045126 27008 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.046362 27063 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:42.969024 21066 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:42.963577 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b7a0bb40f8e3428899935fdfa915dfd0 received from {username='slave'} at 127.0.0.1:54620
I20260709 13:08:42.899957 27433 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:42.989985 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:42.990840 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.991537 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.992425 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:43.052094 21146 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:42.993363 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.994029 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.997385 21145 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
W20260709 13:08:42.998129 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 165791us
I20260709 13:08:43.057547 21218 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 13: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:08:42.986539 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 108403us
I20260709 13:08:42.974821 27469 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.908901 27315 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.105371 21139 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:43.105701 27315 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:43.108016 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:43.109668 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:43.021396 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 122472us
I20260709 13:08:43.113337 21036 leader_election.cc:400] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
I20260709 13:08:43.113734 21036 leader_election.cc:403] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:43.115124 21217 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:43.056138 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b7a0bb40f8e3428899935fdfa915dfd0 on tablet b7a0bb40f8e3428899935fdfa915dfd0 with peer b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:43.117624 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 59851.4us
I20260709 13:08:43.119880 21142 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 13:08:43.024955 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 233983us
I20260709 13:08:43.024735 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:42.976112 27467 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:08:42.976459 27384 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.102313 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.103235 27469 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:43.129150 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:43.142879 21072 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:08:43.144042 21064 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:08:43.144610 21064 pending_rounds.cc:85] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 12
I20260709 13:08:43.144855 21064 pending_rounds.cc:107] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:08:43.139500 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:43.145061 21064 raft_consensus.cc:2889] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:43.133455 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.147102 27285 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:43.147701 27285 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:43.150331 27285 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:43.150736 27469 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:43.050853 27443 raft_consensus.cc:2764] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:08:43.155722 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.180105 21147 raft_consensus.cc:2368] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19.
W20260709 13:08:43.191416 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 89172.7us
I20260709 13:08:43.198283 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.191938 21068 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:43.220562 21072 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:43.156533 21217 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:43.157809 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:43.226704 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 68976.8us
I20260709 13:08:43.268355 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.138038 27211 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:43.271050 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:43.180240 27473 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:08:43.168884 27425 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16.
W20260709 13:08:43.298628 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 100451us
I20260709 13:08:43.224049 27417 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:43.301663 21141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.113s	sys 0.191s
I20260709 13:08:43.282150 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:43.306877 21141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.114s	sys 0.192s
I20260709 13:08:43.180912 27384 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.225090 27467 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:43.303741 21072 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:08:43.310703 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:43.276257 26956 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:43.171991 26884 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 13:08:43.159561 27325 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:43.312481 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.175771 27398 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:43.159030 27315 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.143s
I20260709 13:08:43.318665 21072 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:43.275931 27483 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:43.318616 27080 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:43.325687 27377 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:08:43.326264 27080 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:43.332715 21215 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:43.272073 27218 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:43.313975 27309 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:43.313927 26956 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:43.281908 27211 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.302897 27063 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:08:43.313849 27469 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:43.343400 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:43.352721 27211 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.348836 26956 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:43.322027 27384 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:43.313468 27467 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:43.333374 27389 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:43.340907 27257 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:43.297787 27312 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:08:43.319738 27399 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:08:43.355482 27218 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.382704 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.384457 21072 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:43.376758 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.378466 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.378501 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.379904 27494 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:43.374758 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:43.533550 21068 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:43.535262 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:43.536646 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 162007us
I20260709 13:08:43.538151 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:43.412684 27374 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.005s
I20260709 13:08:43.417306 27467 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.420953 21219 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:08:43.513391 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.429308 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.413314 27399 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:43.402426 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.544498 21063 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:43.543848 21220 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
W20260709 13:08:43.520975 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 123774us
W20260709 13:08:43.412143 27488 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:08:43.415390 27314 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:43.432206 27257 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:43.415836 27309 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:43.383049 27389 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:43.416546 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:43.382378 27328 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.525s	user 0.006s	sys 0.006s
I20260709 13:08:43.521576 21187 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:43.430560 27489 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:08:43.367672 27469 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.404407 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.418716 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:43.406131 27482 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.362s	user 0.001s	sys 0.000s
I20260709 13:08:43.417994 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.576754 21214 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:43.381340 27211 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:43.428373 27325 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:43.371644 27398 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:43.521600 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 189437us
I20260709 13:08:43.522555 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.460613 27080 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:43.471169 27310 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.002s
I20260709 13:08:43.540652 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.583899 27469 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:43.576099 27328 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:43.585306 27312 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:43.540949 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:43.531556 27285 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:43.545817 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:43.534633 27321 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:43.546160 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 231228us
I20260709 13:08:43.552366 21063 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:08:43.541438 27456 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:43.567991 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.542131 27404 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:43.571005 21065 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:43.579135 21066 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:43.545321 27374 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:43.588043 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:43.594074 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 53186us
I20260709 13:08:43.543061 27494 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.594310 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.594463 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.570863 27492 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:43.603633 27492 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:43.603977 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 82505.8us
I20260709 13:08:43.605581 26957 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:43.605772 27492 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:43.606343 27492 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:43.587468 27211 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.591471 27309 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:43.587592 27328 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.608258 21072 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:08:43.607106 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:43.609959 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.611244 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:43.612576 27493 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:43.612780 27211 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.613086 27493 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:43.613588 27493 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:43.615718 21218 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:43.612344 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:43.602674 21063 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 12.
I20260709 13:08:43.608117 27309 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.599077 27467 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:43.618177 21072 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:43.618103 21767 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:43.620693 27389 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:43.613037 27312 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:43.621513 27312 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:43.621879 27312 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:43.595186 27285 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.624634 27285 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.617195 27496 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:43.627789 27496 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:43.628185 27496 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:43.618526 26957 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:43.630425 26957 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:43.616743 27492 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:43.619966 27467 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:43.636736 27467 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:43.567205 27314 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.611930 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.557336 27399 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.619917 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.641281 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.641796 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.616976 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:43.602363 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:43.644749 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.646085 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.642807 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:43.650913 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:43.651036 21185 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.687s (SENT) [suppressed 20 similar messages]
I20260709 13:08:43.643558 27261 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:43.655885 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.656740 21216 raft_consensus.cc:2368] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:43.656610 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.676476 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:43.675577 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:43.661967 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.660440 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.680414 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:43.680900 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 78630.4us
I20260709 13:08:43.681243 21143 raft_consensus.cc:2368] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:43.682423 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.677462 21067 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:43.684224 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.686439 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.690508 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:43.695124 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:43.639648 27389 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:43.699306 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.703965 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:43.704564 21214 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:08:43.700223 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.706220 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.660346 27008 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:43.711730 26957 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.712471 27008 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:43.712884 27008 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:43.692003 27302 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:43.709218 27080 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:43.677459 26956 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.715875 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:43.716195 26956 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.702704 27384 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:43.711530 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 792) took 1066 ms. Trace:
I20260709 13:08:43.717190 21215 rpcz_store.cc:276] 0709 13:08:42.652738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.652916 (+   178us) service_pool.cc:224] Handling call
0709 13:08:43.711332 (+1058416us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54003fa7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:43.711475 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1459990784}
I20260709 13:08:43.718941 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:43.712069 21219 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:43.676930 27310 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:43.720520 27310 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:43.716495 21220 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:43.720909 27310 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:43.691823 27461 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:43.651748 27404 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:43.683920 27328 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:43.771263 27502 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:43.771803 27502 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:43.772253 27502 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:43.772667 27374 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:43.775394 26884 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:43.775857 26884 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:43.708400 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:43.676883 27497 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:43.705590 27413 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:43.797250 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:43.798048 21070 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:43.619069 27257 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.768492 27456 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:43.801340 27369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:54660 (request call id 739) took 4292 ms. Trace:
I20260709 13:08:43.801776 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:43.809003 21216 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:43.776784 26884 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:43.810213 26884 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:43.818759 27456 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:43.821758 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:43.821918 21070 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:43.823043 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:43.824313 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:43.826296 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:43.756227 27389 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.829358 21071 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:08:43.831238 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:43.839627 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:43.845328 21067 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:43.766631 27404 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:43.688437 27406 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 13:08:43.793431 21214 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:43.804848 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:43.845510 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:43.848657 27413 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:43.807651 27314 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:43.853296 27312 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.808560 27369 rpcz_store.cc:276] 0709 13:08:39.509463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.509685 (+   222us) service_pool.cc:224] Handling call
0709 13:08:39.872987 (+363302us) ts_tablet_manager.cc:934] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
0709 13:08:42.604472 (+2731485us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:08:43.187056 (+582584us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:43.187172 (+   116us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:08:43.740529 (+553357us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:08:43.797327 (+ 56798us) tablet_replica.cc:206] Creating consensus instance
0709 13:08:43.801210 (+  3883us) ts_tablet_manager.cc:989] Replica registered.
0709 13:08:43.801276 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":5115,"tablet-copy.queue_time_us":222442,"thread_start_us":261992,"threads_started":2}
I20260709 13:08:43.860047 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 3103) took 1235 ms. Trace:
I20260709 13:08:43.928669 21147 rpcz_store.cc:276] 0709 13:08:42.624217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.624301 (+    84us) service_pool.cc:224] Handling call
0709 13:08:43.859789 (+1235488us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54002b52d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:43.859966 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276023552}
I20260709 13:08:43.869149 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:43.930428 21219 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:43.802263 27496 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.883248 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:43.886785 21216 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
W20260709 13:08:43.943707 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60562.2us
I20260709 13:08:43.802039 21220 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:08:43.920651 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 269620us
I20260709 13:08:43.926645 21214 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
W20260709 13:08:43.956207 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.950834 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:43.961077 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:43.917322 27404 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:43.977432 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:43.981688 21185 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:43.988142 21219 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:44.004105 21141 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:44.008046 27496 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.979591 27369 tablet_copy_client.cc:806] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:44.009426 27369 tablet_copy_client.cc:670] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:44.012431 27312 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:43.920480 27493 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:43.967070 27497 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:44.019985 21213 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:43.967480 27314 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:44.022038 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 339634us
I20260709 13:08:44.029978 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:44.001739 27309 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:44.017524 27417 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:44.016636 27500 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:44.047986 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:44.049505 21792 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:44.021601 27399 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.047343 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:44.026546 27389 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.055735 21795 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:44.058403 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:44.034762 27497 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:44.035995 27218 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:44.037760 27493 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.067749 27456 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.040056 27310 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:44.090740 27456 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.092697 21142 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:44.092991 27456 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.093739 27310 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.051234 27502 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.095897 27502 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.096417 27502 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:44.096927 27502 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.098266 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 783) took 2857 ms. Trace:
I20260709 13:08:44.102953 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:44.103780 21141 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:44.103082 27458 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:44.104627 27458 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:44.105120 27494 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.105482 27458 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.106541 27458 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:44.068925 27452 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:44.111722 27141 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.111984 27452 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.114205 27452 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.057753 27417 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:44.098649 21212 rpcz_store.cc:276] 0709 13:08:41.240824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.241108 (+   284us) service_pool.cc:224] Handling call
0709 13:08:42.863701 (+1622593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:42.977828 (+114127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:42.977871 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:42.977924 (+    53us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:08:42.977962 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.978042 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.098052 (+1120010us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54004439d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:44.098180 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118107,"raft.queue_time_us":114534,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5265,"spinlock_wait_cycles":3719504128}
I20260709 13:08:44.099001 27502 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.035341 27467 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.117178 27417 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.119413 27467 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.120234 21801 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:44.033578 27314 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:44.136411 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 209040us
I20260709 13:08:44.139004 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:44.063611 27500 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:44.160794 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:44.162482 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:44.162925 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:44.163724 21219 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:44.165774 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:44.166306 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:44.167033 21219 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:44.174355 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:44.175246 21212 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:44.177085 21221 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:44.179306 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:44.162490 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 142875us
I20260709 13:08:44.126696 27008 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.140285 27328 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:44.194027 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:44.213536 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:44.214303 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:44.217474 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:44.159410 27469 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:44.222343 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:44.090997 27497 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.226581 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.231267 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.228472 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:44.236732 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:44.235708 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:44.176396 27493 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:44.194284 27325 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:44.253010 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:44.251071 21138 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:44.253957 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.221282 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:44.255971 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:44.254624 21141 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:44.266537 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:44.194759 27499 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:08:44.271301 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:44.232014 27261 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:44.231805 27172 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:44.221465 27494 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:44.231635 27433 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:44.244946 27469 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.272733 27312 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:08:44.284906 27314 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.284972 27080 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:44.270187 26957 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:44.270468 27413 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:44.289958 27008 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.200182 27398 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:08:44.234956 27389 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:44.290402 27325 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:44.250573 27458 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:44.269172 27309 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:44.288084 27321 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:44.291131 27172 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:44.293112 27494 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:44.312860 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2011) took 1323 ms. Trace:
I20260709 13:08:44.317422 21217 rpcz_store.cc:276] 0709 13:08:42.989322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.989722 (+   400us) service_pool.cc:224] Handling call
0709 13:08:44.312610 (+1322888us) spinlock_profiling.cc:243] Waited 64.1 ms on lock 0x7b540034b7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:44.312766 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147398912}
I20260709 13:08:44.305114 27080 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:44.330766 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 279308us
I20260709 13:08:44.318188 27172 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:44.275332 27500 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.318392 27494 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:44.344388 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 73431.7us
I20260709 13:08:44.344946 27328 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:44.313735 27389 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.352393 21792 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:44.316823 27458 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:44.341642 27509 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:08:44.353605 21793 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:44.350069 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:44.351876 21794 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:44.354262 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:44.347388 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:44.340847 27505 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:44.373289 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:44.349524 27469 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.347638 27527 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:44.355406 27413 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:44.355515 27467 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:44.376219 27506 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:08:44.366564 27452 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.364668 27521 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:44.357985 27384 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:08:44.349720 27328 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.445001 21070 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
W20260709 13:08:44.456188 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 107925us
I20260709 13:08:44.503542 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:44.361289 27309 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.504484 21794 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:44.509426 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 226489us
I20260709 13:08:44.538380 27389 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.544128 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.545560 21143 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:44.546268 27328 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.550650 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.557261 21139 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:44.512161 26957 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:08:44.516312 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.349705 27080 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.525033 27314 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:44.572038 27314 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.567891 21138 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:44.576056 27314 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.578225 27314 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:44.570712 26957 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.383731 27505 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:44.409857 27515 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:08:44.400556 27521 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.394835 27527 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.416359 27534 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:44.396307 27467 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.397220 27452 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.592829 26957 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:44.626471 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52331.3us
I20260709 13:08:44.646448 27495 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:08:44.649276 21036 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.988s	user 0.022s	sys 0.011s
W20260709 13:08:44.651927 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 297640us
W20260709 13:08:44.652832 21036 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.023s	sys 0.012s
W20260709 13:08:44.653613 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 992948us
I20260709 13:08:44.600788 27458 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.659191 27520 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:44.659766 27520 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:44.661792 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:44.663872 27540 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:44.665544 27495 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:44.666123 27495 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:44.668200 27531 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:44.396445 27413 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.671438 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:44.673061 27524 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:44.674108 27413 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.684768 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:44.689316 27389 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:44.689173 27172 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:44.690250 27389 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.693431 27389 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.694870 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:44.695902 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:44.697926 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:44.700452 21216 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:08:44.700701 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:44.706159 21217 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:08:44.717833 27505 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.719161 21219 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:08:44.720814 21215 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:08:44.720597 27520 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.724259 27389 raft_consensus.cc:2764] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:08:44.726845 27520 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:44.727468 21217 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:44.728560 27404 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.012s
I20260709 13:08:44.729072 27404 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:44.731563 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 73947.7us
W20260709 13:08:44.719187 27218 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.632s	user 0.014s	sys 0.009s
I20260709 13:08:44.737989 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.742534 27218 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:44.747684 26884 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:44.752108 21141 raft_consensus.cc:2410] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:08:44.750946 26884 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:44.772555 21800 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:44.774698 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:44.734755 27404 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.774487 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:44.775404 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3169) took 1104 ms. Trace:
I20260709 13:08:44.776328 27460 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:44.769101 26884 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:44.781719 27172 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.812674 27378 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:08:44.763949 27389 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.813814 27389 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.814459 27377 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.815136 27377 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.811018 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:44.812903 21216 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:44.818408 27377 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:44.821271 27532 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.822134 27080 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:44.822891 27080 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:44.824542 21221 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.647s	user 0.005s	sys 0.019s
I20260709 13:08:44.825044 21221 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
W20260709 13:08:44.826057 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 93590.5us
I20260709 13:08:44.826293 27467 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.827093 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:44.832824 27527 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.833706 27460 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.836872 27460 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:44.839820 27529 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.450s	user 0.001s	sys 0.004s
I20260709 13:08:44.822129 27532 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.843225 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:44.843565 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:44.809731 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:44.845402 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 61837.3us
W20260709 13:08:44.846263 27538 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 13:08:44.846807 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:44.848170 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:44.849984 27404 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:44.851923 27452 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.808911 21218 rpcz_store.cc:276] 0709 13:08:43.642987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.643136 (+   149us) service_pool.cc:224] Handling call
0709 13:08:43.644045 (+   909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:43.644248 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:43.685163 (+ 40915us) log.cc:844] Serialized 14 byte log entry
0709 13:08:44.669017 (+983854us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b180007d3e0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a 00007f7f42526a2a
0709 13:08:44.746726 (+ 77709us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:44.746830 (+   104us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:44.746895 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:44.747027 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.747216 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40763,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2517,"raft.queue_time_us":352,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":174,"spinlock_wait_cycles":8918912,"thread_start_us":46569,"threads_started":1,"wal-append.queue_time_us":1089727}
I20260709 13:08:44.855600 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:44.856642 21215 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
W20260709 13:08:44.858098 27494 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.002s
I20260709 13:08:44.858669 27494 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.861213 21212 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:44.864368 21217 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:44.817004 27389 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.865942 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:44.866984 26957 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:08:44.870122 27546 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:44.870919 27546 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.871335 27533 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:08:44.873337 27541 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.186s	user 0.000s	sys 0.002s
I20260709 13:08:44.873757 27546 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:44.874150 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:44.874518 27546 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:44.875174 27172 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:44.876688 27494 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.868229 27460 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:44.878455 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:44.880790 27416 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:44.883981 27460 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:44.881498 27389 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:44.885392 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.886531 27312 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:44.889304 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:44.890038 21141 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:44.890990 27080 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.875061 27546 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:44.903959 27141 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:08:44.906244 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2020) took 1160 ms. Trace:
I20260709 13:08:44.906709 21213 rpcz_store.cc:276] 0709 13:08:43.699183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.705976 (+  6793us) service_pool.cc:224] Handling call
0709 13:08:44.850318 (+1144342us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54003b60d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:44.859619 (+  9301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711663488}
I20260709 13:08:44.907510 21212 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:44.909564 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:44.910398 21801 raft_consensus.cc:2368] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13.
I20260709 13:08:44.911270 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.920269 27449 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:44.921698 27532 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.923579 27449 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.923981 27080 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:44.929402 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:44.930096 27389 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.944344 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.945691 21143 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:44.930930 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:44.947142 27257 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:44.933459 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.933847 27312 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.897377 27546 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:44.940493 21141 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:44.942653 21147 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:44.943408 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:44.943773 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.944298 21184 leader_election.cc:400] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13.
I20260709 13:08:44.954708 21184 leader_election.cc:403] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:08:44.949071 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:44.950611 21140 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:44.947213 27389 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.947674 27257 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:44.953833 27312 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.995718 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:44.996942 27406 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:44.997277 27312 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:44.998723 27495 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:44.992323 26957 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:45.001360 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:45.004252 21111 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:45.001362 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:44.984516 27520 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13.
I20260709 13:08:45.011454 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:45.014086 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65114.1us
I20260709 13:08:45.014883 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:44.989253 27389 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.978603 27449 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.023917 21801 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
W20260709 13:08:45.030278 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 203338us
I20260709 13:08:45.024709 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.040607 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.042153 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.023692 27406 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:45.024811 27141 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:45.010370 26884 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:45.057089 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 113993us
I20260709 13:08:45.055871 27406 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:45.048220 27389 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.087134 27532 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:45.094413 27532 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.094970 27389 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:45.097712 27532 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:45.099308 27417 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.004s	sys 0.020s
W20260709 13:08:45.099882 27417 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.004s	sys 0.021s
I20260709 13:08:45.100299 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.101032 21067 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
W20260709 13:08:45.101986 27547 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:45.105414 27257 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.119717 27460 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:45.121182 21797 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:45.123008 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.123973 21068 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:45.127579 26884 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:45.099728 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.097440 20963 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 11 to 14, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 14 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:45.141603 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.083794 27406 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:45.076865 27141 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:45.153857 27544 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:08:45.155718 27449 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:45.157235 27141 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:45.161634 27553 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:08:45.161892 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.162556 21071 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:45.155793 21064 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:45.164781 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.163811 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:45.183666 27543 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:08:45.184880 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.186415 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.187570 21065 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:08:45.188123 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 104018us
I20260709 13:08:45.189558 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:45.192643 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 61551.6us
I20260709 13:08:45.193627 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:45.194828 27398 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:08:45.203636 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 172281us
I20260709 13:08:45.176612 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.233531 27560 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:45.234319 27560 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.237092 27560 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:45.237887 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:45.242295 21070 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:08:45.174780 27531 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:45.246122 21065 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:45.248507 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.253247 21216 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:08:45.254346 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.242766 21069 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:45.260257 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.260947 21070 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:45.265112 27495 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:45.268375 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.269148 21065 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:45.265570 27546 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.271136 27551 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:45.271387 21765 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:45.271809 27551 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.273067 27449 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:45.270153 21065 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:45.273712 21065 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:45.270004 27497 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:08:45.283802 27261 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:45.284003 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67623.7us
I20260709 13:08:45.285472 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:45.286255 21214 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:45.286659 27261 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:45.287508 27369 tablet_copy_client.cc:538] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:45.288028 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:45.288956 21792 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:45.273454 27449 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:45.281358 21065 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.285600 21069 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:08:45.287448 27384 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:45.270891 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:45.277294 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:45.286814 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:45.250686 27533 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.296959 27449 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:45.318215 21065 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:45.321341 27521 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:45.328363 21069 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:08:45.331622 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44885.4us
I20260709 13:08:45.297767 27531 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.333792 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:45.245991 27398 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:45.345980 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 170015us
W20260709 13:08:45.347389 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 70179.9us
I20260709 13:08:45.287170 27499 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:45.352900 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:45.360038 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 251681us
I20260709 13:08:45.360512 21794 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:45.369704 21069 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:08:45.372162 27218 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:45.372439 27546 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:45.293629 27416 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:08:45.378531 27398 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.379452 27384 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:45.379956 27384 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.380314 27384 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:45.380542 26884 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:45.381390 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:45.381553 27533 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.382071 21800 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:45.383458 27499 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.383960 27533 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:45.342235 27458 raft_consensus.cc:2764] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:08:45.358897 21070 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:45.380617 27257 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:45.388777 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:45.385946 27561 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.118s	user 0.003s	sys 0.000s
I20260709 13:08:45.393822 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:45.407222 21065 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
W20260709 13:08:45.408775 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:45.402799 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49972.1us
I20260709 13:08:45.425873 21765 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:45.418170 21065 pending_rounds.cc:85] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 10
I20260709 13:08:45.385954 27499 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:45.436812 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:45.441413 21069 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:45.427551 21065 pending_rounds.cc:107] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:08:45.415805 27218 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:45.394464 27551 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:45.452903 21065 raft_consensus.cc:2889] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:45.453975 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60251.1us
I20260709 13:08:45.454823 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:45.445160 27449 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:45.458295 27566 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:08:45.458899 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:45.461761 27416 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:45.462755 27416 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:45.436929 27312 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:45.463760 27312 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.466092 27312 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:45.467730 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:45.468515 27413 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:45.468832 21139 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:45.472258 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.469326 27413 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.390178 27524 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:08:45.436527 27456 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:45.476308 21068 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:45.479349 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:45.473256 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 47399us
I20260709 13:08:45.437934 27565 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:45.488689 21145 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:08:45.491921 27452 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:45.492825 27374 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:45.493523 27374 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.495404 27549 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:45.393301 27460 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:08:45.453281 27218 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:45.497326 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:45.497941 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:45.481938 27499 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:45.524295 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:45.528034 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.532184 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:45.542569 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 142906us
W20260709 13:08:45.543401 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 247441us
I20260709 13:08:45.543854 20963 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 12 to 16, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 16 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:45.545164 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.548691 27548 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:45.550076 26956 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:45.523649 27218 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:45.501065 27531 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:45.561687 27369 tablet_bootstrap.cc:492] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
I20260709 13:08:45.576112 27499 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:45.526399 27389 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:45.578711 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 77096.6us
W20260709 13:08:45.579710 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.500072 27549 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.583938 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:45.586338 27449 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:45.587399 27314 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.591158 27374 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:45.592036 27141 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:45.593058 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.596974 21796 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:45.691069 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 107359us
I20260709 13:08:45.692518 21069 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:45.597944 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:45.598815 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:45.599680 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:45.603586 27080 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.531527 21142 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:45.638895 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:45.556454 27406 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.650846 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:45.652562 27456 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:45.653388 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 153675us
I20260709 13:08:45.641199 26956 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:45.639413 27548 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:45.666051 27413 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:45.669380 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.671752 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:45.684170 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.687937 21145 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:08:45.690589 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.638463 27384 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.693996 27467 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:45.694111 21800 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 12.
I20260709 13:08:45.706334 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:45.694355 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:45.694491 27314 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.694898 21799 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:45.695591 27389 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.697700 27549 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:45.697906 27564 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:45.701908 27374 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:08:45.709658 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 70857.1us
I20260709 13:08:45.708870 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:45.717412 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:45.718103 21801 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:45.718751 27406 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:45.719473 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:45.719638 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 68940us
I20260709 13:08:45.720352 27314 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:45.720782 27546 raft_consensus.cc:2764] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:08:45.720939 27524 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:45.739614 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:45.721295 21147 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:45.718752 21140 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:45.714744 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:45.722275 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:45.742476 21221 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:45.732879 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:45.726573 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:45.733881 21035 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:45.746122 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.746899 21066 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:45.723026 27467 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:45.751492 21064 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:45.753036 21767 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:45.732117 27449 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:45.702724 27141 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:08:45.757941 21146 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:45.762439 21141 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:45.765302 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:45.744479 27524 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:45.789443 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 83180.3us
I20260709 13:08:45.807694 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:45.790555 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:45.795003 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 84602us
I20260709 13:08:45.737696 26884 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:08:45.750555 27384 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:45.754474 27568 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:45.812371 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:45.756232 27467 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:45.739898 27495 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:08:45.822455 27456 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:45.781728 27458 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:45.832465 27218 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.835471 27325 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:45.837276 27389 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:45.838867 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:45.702675 27499 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.846531 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:45.763082 27314 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:45.763301 27406 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:45.732378 27080 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:45.839066 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:45.849167 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83941.5us
I20260709 13:08:45.820901 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:45.811707 27524 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:45.767410 27533 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:45.847740 27384 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:45.823239 27568 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.861830 21147 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:45.864118 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:45.870580 21145 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:45.870375 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:45.874809 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:45.875000 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:45.853253 27452 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:45.887547 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:45.890384 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 43971.4us
I20260709 13:08:45.858582 27413 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:45.892568 21217 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:08:45.852035 27218 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:45.849181 27257 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:08:45.873256 27521 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:45.872206 27568 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:45.901938 21063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.156s	sys 0.277s
W20260709 13:08:45.907650 21063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.157s	sys 0.277s
I20260709 13:08:45.908160 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.909132 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:45.908440 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:45.914410 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:45.916162 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 125792us
I20260709 13:08:45.890276 27452 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.918491 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:45.848901 27548 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.929677 21147 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:45.902928 27314 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:45.953511 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:45.954198 27533 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:08:45.959550 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 92514.6us
I20260709 13:08:45.966073 27407 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:45.967005 27407 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:45.967363 27407 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:45.968192 27407 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:45.969417 27407 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:45.970101 27551 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:45.970541 27551 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:45.923950 27452 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:45.921319 27521 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:46.015265 27413 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:45.995225 27494 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:46.006067 27172 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:45.997577 27551 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:46.031929 27399 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.032542 27399 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.035190 27399 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:46.035737 27080 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:46.043877 27467 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.047683 27467 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:45.928342 27565 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 13:08:46.017023 27449 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:46.049770 27314 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:46.016566 27218 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:08:46.031006 21070 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.016s	sys 0.027s
W20260709 13:08:46.090119 21070 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.017s	sys 0.027s
W20260709 13:08:46.030694 21067 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.021s	sys 0.021s
W20260709 13:08:46.090884 21067 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.022s	sys 0.022s
I20260709 13:08:46.028188 27172 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:46.109208 21069 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:46.120326 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:46.123437 27008 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:08:46.125916 27568 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.127599 21066 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 13: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:46.128461 27568 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:46.129213 21072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.003s	sys 0.012s
W20260709 13:08:46.129567 21072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.003s	sys 0.012s
I20260709 13:08:46.130347 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2894) took 1090 ms. Trace:
I20260709 13:08:46.130697 21072 rpcz_store.cc:276] 0709 13:08:45.040300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.049746 (+  9446us) service_pool.cc:224] Handling call
0709 13:08:45.059593 (+  9847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:45.250354 (+190761us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:45.251657 (+  1303us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:45.391578 (+139921us) log.cc:844] Serialized 26 byte log entry
0709 13:08:46.129942 (+738364us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:46.129994 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.130067 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:46.130119 (+    52us) raft_consensus.cc:1685] finished
0709 13:08:46.130131 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:46.130266 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39244,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"thread_start_us":801148,"threads_started":2,"wal-append.queue_time_us":453392}
I20260709 13:08:46.092651 27548 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:46.141083 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:46.142076 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:46.142470 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:46.146993 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:46.148980 21065 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:08:46.138684 21147 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:46.150869 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:46.149762 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 240686us
I20260709 13:08:46.111408 27172 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:46.152777 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 239764us
I20260709 13:08:46.187180 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:46.189271 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:46.099992 27524 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:46.100512 27449 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.190457 21795 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:46.185745 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:46.189309 21792 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:46.185796 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:46.192212 21799 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:46.105257 27218 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.195709 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:46.200312 21069 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:46.217005 21065 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
W20260709 13:08:46.219111 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67289.8us
I20260709 13:08:46.223125 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:46.193912 27499 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.225397 21221 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:46.148190 27521 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:46.227991 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 274575us
I20260709 13:08:46.230720 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:46.231873 27560 raft_consensus.cc:2764] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:08:46.232748 21069 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:46.233300 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:46.234097 27560 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:46.236255 21036 leader_election.cc:400] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:46.236593 21036 leader_election.cc:403] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:46.237622 27560 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:46.235066 21069 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:08:46.240799 21069 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:46.242704 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:46.243480 27499 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:46.245558 27499 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:46.247018 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:46.247154 21795 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:46.247786 21217 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:46.248874 21035 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:46.249936 27560 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:46.250638 27560 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:46.251226 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:46.254577 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:46.255237 21800 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:08:46.251886 27560 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.236436 27407 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:46.287405 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1980) took 1173 ms. Trace:
W20260709 13:08:46.305737 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 116591us
I20260709 13:08:46.256711 27080 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:46.353402 21217 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
W20260709 13:08:46.358177 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:46.306528 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:46.313943 21145 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:46.289270 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 1997) took 1133 ms. Trace:
I20260709 13:08:46.373886 21765 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:46.221510 27586 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.083s
I20260709 13:08:46.302961 27564 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:46.378374 21070 rpcz_store.cc:276] 0709 13:08:45.155803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.421476 (+265673us) service_pool.cc:224] Handling call
0709 13:08:45.453283 (+ 31807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:45.492081 (+ 38798us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:46.092825 (+600744us) log.cc:844] Serialized 14 byte log entry
0709 13:08:46.284828 (+192003us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:46.285185 (+   357us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:46.285731 (+   546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.286574 (+   843us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:46.288484 (+  1910us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":397,"prepare.queue_time_us":83891,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":43,"raft.queue_time_us":18209,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":725926,"threads_started":2}
W20260709 13:08:46.378412 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71976us
I20260709 13:08:46.384951 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:46.300890 21067 rpcz_store.cc:276] 0709 13:08:45.114223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.275063 (+160840us) service_pool.cc:224] Handling call
0709 13:08:45.317454 (+ 42391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:45.435735 (+118281us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:46.091407 (+655672us) log.cc:844] Serialized 14 byte log entry
0709 13:08:46.286019 (+194612us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:46.286294 (+   275us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:46.286581 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.286844 (+   263us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:46.287249 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1348,"prepare.queue_time_us":111960,"prepare.run_cpu_time_us":199,"prepare.run_wall_time_us":201,"raft.queue_time_us":2039,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"thread_start_us":752002,"threads_started":2,"wal-append.queue_time_us":192188}
I20260709 13:08:46.386119 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:46.320523 27458 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:46.390465 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:46.272423 27499 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 13:08:46.368307 27497 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:46.424001 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:46.399885 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 107973us
I20260709 13:08:46.430285 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:46.432577 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:46.433435 21143 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:08:46.434376 21138 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:08:46.436067 27008 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.436757 27008 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:46.451862 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78053.7us
I20260709 13:08:46.453909 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:46.395145 27325 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:46.394441 27458 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:46.440364 27008 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:46.379084 27080 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:46.387737 27581 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 13:08:46.457415 27325 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:46.441326 27497 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:46.474751 27458 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:46.487717 21140 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:46.456207 27599 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:46.492559 21138 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:46.474745 27407 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:46.495146 27407 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.496970 27607 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:46.497552 27607 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:46.499401 27564 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:46.499970 27564 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:46.499722 27607 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.504344 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:46.505177 21147 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:46.506284 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:46.508594 27594 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:46.509166 27594 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:46.510028 27594 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.510651 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:46.511631 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:46.511978 27579 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:08:46.511749 26956 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.481743 27584 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.483314 27325 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:46.523634 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:46.528965 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.001s	sys 0.005s
W20260709 13:08:46.529876 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.001s	sys 0.005s
W20260709 13:08:46.530404 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 525734us
I20260709 13:08:46.491093 27604 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:46.535859 27585 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 13:08:46.557780 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 106363us
I20260709 13:08:46.524484 27599 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:46.529275 27141 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:46.491791 27452 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.576586 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:46.539067 27524 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:46.576319 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:46.586411 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1100) took 1161 ms. Trace:
I20260709 13:08:46.575186 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:46.575593 27413 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:46.569383 27589 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
W20260709 13:08:46.581933 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58417.4us
I20260709 13:08:46.647949 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:46.563834 27449 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:46.584226 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:46.586798 21068 rpcz_store.cc:276] 0709 13:08:45.424763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.557887 (+133124us) service_pool.cc:224] Handling call
0709 13:08:45.558231 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:45.744498 (+186267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:45.744530 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:45.744574 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:45.754751 (+ 10177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.586242 (+831491us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400215054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:46.586277 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:46.586365 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420444928}
I20260709 13:08:46.624426 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:46.667589 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 96295us
I20260709 13:08:46.672302 21798 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:46.528654 27584 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:46.656239 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:46.645242 21036 connection.cc:256] RPC call timeout handler was delayed by -1.6701228169999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
I20260709 13:08:46.691972 21214 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:08:46.701812 21036 connection.cc:256] RPC call timeout handler was delayed by -1.572298069999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 13:08:46.702319 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2842980700000908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
W20260709 13:08:46.702636 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2095980700000837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
I20260709 13:08:46.699993 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:46.704874 21220 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:08:46.706477 27614 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:46.710072 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:46.711558 27549 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:46.712208 27549 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:46.713430 27549 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.692575 27611 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:46.678625 27141 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:46.714202 27592 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 13:08:46.699981 27599 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:46.715078 27549 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:46.729753 27551 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.673s	user 0.006s	sys 0.006s
I20260709 13:08:46.730283 27551 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:46.732537 27551 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:46.733592 27141 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:46.734694 27601 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 13:08:46.735723 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:46.739584 27531 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.683133 27524 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:08:46.745389 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 97536.1us
I20260709 13:08:46.746361 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:46.563452 27604 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:46.744374 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:46.747551 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:46.758899 27548 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.611s	user 0.008s	sys 0.003s
I20260709 13:08:46.759652 27141 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:46.759735 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:46.760398 27141 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:46.760941 27257 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:46.761345 27257 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.761736 27257 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:46.761502 27141 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.763835 27614 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:46.765295 27614 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:46.769810 27257 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:46.779561 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 229165us
I20260709 13:08:46.743993 27524 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.784921 21215 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:46.786861 27449 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:46.805500 21797 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:08:46.793819 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:46.790071 27524 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:46.826535 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64387.9us
W20260709 13:08:46.822676 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 294034us
I20260709 13:08:46.748379 27531 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.788991 27568 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:46.799542 27483 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:08:46.828702 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 89641.7us
I20260709 13:08:46.828969 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:46.800956 27080 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:46.826129 27494 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.803s	user 0.008s	sys 0.004s
I20260709 13:08:46.821055 27607 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:46.830721 27172 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.001s
I20260709 13:08:46.806504 27449 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.929365 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:46.916848 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:46.930379 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 101853us
I20260709 13:08:46.932755 21063 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:46.917333 27497 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:46.918167 27483 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.937723 21797 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:46.921738 27494 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:46.944860 27257 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:46.927853 27605 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:46.929281 27616 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:46.832826 27564 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:46.952230 27621 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.952417 27452 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:46.952879 27621 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:46.953954 27564 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:46.954846 27605 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:46.955852 27456 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.129s	user 0.008s	sys 0.005s
I20260709 13:08:46.956278 27456 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:46.956594 27449 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:46.957340 21793 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:46.927027 27172 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:46.959995 21212 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:46.960278 27172 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:46.956460 27616 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:46.960979 27172 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:46.936280 27532 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:46.962899 27616 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:46.964358 27595 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.670s	user 0.001s	sys 0.007s
I20260709 13:08:46.966303 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2909) took 1154 ms. Trace:
I20260709 13:08:46.966673 21064 rpcz_store.cc:276] 0709 13:08:45.816749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.962283 (+145534us) service_pool.cc:224] Handling call
0709 13:08:45.998654 (+ 36371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:46.025711 (+ 27057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:46.025760 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:46.029677 (+  3917us) log.cc:844] Serialized 26 byte log entry
0709 13:08:46.285974 (+256297us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:46.286237 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.286622 (+   385us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:46.963680 (+677058us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:46.965997 (+  2317us) raft_consensus.cc:1685] finished
0709 13:08:46.966023 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:46.966243 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7083,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3907,"spinlock_wait_cycles":20838272,"thread_start_us":175756,"threads_started":1,"wal-append.queue_time_us":206344}
I20260709 13:08:46.941799 27483 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:46.960525 27257 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:46.969050 27257 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:46.969529 27257 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:46.969151 21036 leader_election.cc:400] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15.
I20260709 13:08:46.971225 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:46.973701 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:46.939210 27499 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:08:46.974184 27548 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:46.939973 27568 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:46.980172 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:46.968530 27598 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:46.930959 27607 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.000779 27531 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.002282 27607 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:47.021425 27608 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:08:47.022853 27584 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.028710 27532 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:47.032480 27597 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.467s	user 0.002s	sys 0.000s
I20260709 13:08:47.030133 27257 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.030622 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.033267 27586 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:47.033713 27458 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.503s	user 0.003s	sys 0.007s
W20260709 13:08:47.034228 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 54171.9us
I20260709 13:08:47.034188 27458 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.036494 27458 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.031668 27604 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:46.992138 27598 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:47.031337 27483 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.037168 27564 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:08:47.071563 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 237826us
I20260709 13:08:47.083535 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2063) took 1335 ms. Trace:
I20260709 13:08:47.084022 21221 rpcz_store.cc:276] 0709 13:08:45.748464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.748634 (+   170us) service_pool.cc:224] Handling call
0709 13:08:45.754294 (+  5660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:46.225960 (+471666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.083244 (+857284us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b5400442ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:47.083457 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082993408}
I20260709 13:08:47.089051 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:47.092255 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.093500 21796 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:47.096125 21798 raft_consensus.cc:2410] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:08:47.100008 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:47.095070 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:47.103154 21063 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:47.106799 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:47.107976 21072 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:08:47.085922 27607 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.119020 27598 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.120677 27598 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:47.121336 27596 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.827s	user 0.007s	sys 0.006s
I20260709 13:08:47.124320 26956 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:47.127988 27389 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:47.128486 27389 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:47.129416 27389 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.099433 27413 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:47.135632 27627 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:47.136096 27413 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.136264 27627 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.138677 27627 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:47.140888 27325 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.000s
I20260709 13:08:47.139364 27389 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:47.141373 27325 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.107496 27551 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:08:47.104220 27497 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:46.228434 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:47.149142 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2009) took 1044 ms. Trace:
W20260709 13:08:47.147118 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 318225us
I20260709 13:08:47.149646 21065 rpcz_store.cc:276] 0709 13:08:46.105024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.253920 (+148896us) service_pool.cc:224] Handling call
0709 13:08:46.279494 (+ 25574us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:46.316076 (+ 36582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:46.316111 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:46.386283 (+ 70172us) log.cc:844] Serialized 26 byte log entry
0709 13:08:46.808570 (+422287us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:47.108430 (+299860us) log.cc:844] Serialized 14 byte log entry
0709 13:08:47.147527 (+ 39097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.147615 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.147670 (+    55us) raft_consensus.cc:1685] finished
0709 13:08:47.147689 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.149071 (+  1382us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5673,"prepare.queue_time_us":228047,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":100455,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":11777,"thread_start_us":620403,"threads_started":2,"wal-append.queue_time_us":555286}
W20260709 13:08:47.143517 27587 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.849s	user 0.012s	sys 0.004s
I20260709 13:08:47.155365 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.155682 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:47.168784 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.172714 21147 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:47.144618 27257 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:47.177860 21794 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:47.177146 21145 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:08:47.178707 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.170071 27568 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.219826 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:47.221302 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:47.144104 27607 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.154047 20963 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:47.241611 27568 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.147852 27497 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.245591 27621 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.233412 27588 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:47.246706 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:47.251416 27622 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:08:47.252724 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 875) took 1397 ms. Trace:
W20260709 13:08:47.327200 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:47.262035 27607 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:47.269130 27605 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.271059 27546 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:47.290104 27406 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.178216 27604 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.113s
I20260709 13:08:47.293295 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 816) took 1047 ms. Trace:
W20260709 13:08:47.304387 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:47.322110 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:47.338680 21145 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:47.323599 21072 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:47.326148 21143 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:47.326674 21146 rpcz_store.cc:276] 0709 13:08:45.855486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.855602 (+   116us) service_pool.cc:224] Handling call
0709 13:08:45.856021 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:45.868201 (+ 12180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:45.868251 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:45.894136 (+ 25885us) log.cc:844] Serialized 26 byte log entry
0709 13:08:46.168666 (+274530us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:46.168732 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.168837 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.092437 (+923600us) spinlock_profiling.cc:243] Waited 8.47 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:47.252287 (+159850us) raft_consensus.cc:1685] finished
0709 13:08:47.252310 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.252536 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8853,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":13505,"spinlock_wait_cycles":19490560,"thread_start_us":239172,"threads_started":1,"wal-append.queue_time_us":382535}
I20260709 13:08:47.325829 27588 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.358669 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.338218 21212 rpcz_store.cc:276] 0709 13:08:46.245493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.245658 (+   165us) service_pool.cc:224] Handling call
0709 13:08:46.246114 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:46.960355 (+714241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.293072 (+332717us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b540034c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:47.293234 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1641314176}
W20260709 13:08:47.363404 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 181604us
I20260709 13:08:47.363778 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.372373 21138 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:47.340605 27398 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.334748 27080 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:47.335178 27546 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:47.381767 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.358121 27325 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.368570 27588 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:47.390698 21145 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:47.387931 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.392830 21801 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEARNER]: Advancing to term 19
I20260709 13:08:47.393632 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:47.394558 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.387086 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.395148 21145 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:47.375742 27605 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:47.446799 27524 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
I20260709 13:08:47.447600 27524 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.471480 27605 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.381505 27080 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.367307 27406 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.476498 27080 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.376127 27008 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:08:47.387233 27584 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:47.478220 27584 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:47.455976 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67010.8us
I20260709 13:08:47.480337 27584 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.383031 27398 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.482421 27398 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:47.456506 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 63039.1us
I20260709 13:08:47.456904 27640 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:47.392730 27314 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:47.484114 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:47.483975 27314 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:47.457813 27631 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:08:47.461194 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3178) took 1032 ms. Trace:
I20260709 13:08:47.461732 27533 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.463367 27629 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:47.388530 27399 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:47.509907 27399 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.511665 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.512243 27399 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.463539 27497 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.512749 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.394028 27581 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:47.513936 27581 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.516203 27581 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.509801 27629 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.517736 27218 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:47.467095 27642 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:08:47.396252 27627 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:47.391711 27460 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:47.468505 27646 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:08:47.502082 21142 rpcz_store.cc:276] 0709 13:08:46.428741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.430014 (+  1273us) service_pool.cc:224] Handling call
0709 13:08:46.430364 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:46.523631 (+ 93267us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:46.821451 (+297820us) log.cc:844] Serialized 14 byte log entry
0709 13:08:47.002976 (+181525us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:47.003028 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:47.003073 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.460889 (+457816us) spinlock_profiling.cc:243] Waited 93 ms on lock 0x7b54004231d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:47.460952 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.461121 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8563,"prepare.queue_time_us":280724,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":71248,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":16880,"spinlock_wait_cycles":219819136,"thread_start_us":418180,"threads_started":2,"wal-append.queue_time_us":283794}
W20260709 13:08:47.470883 27591 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.177s	user 0.000s	sys 0.013s
I20260709 13:08:47.446674 27565 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:47.379546 27531 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:47.396124 27632 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:08:47.501137 27605 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.519212 27629 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.530303 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.532245 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.534559 27314 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.520262 27627 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.536888 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:47.537560 27627 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:47.520361 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:47.539216 27406 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:47.520499 27460 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.524916 27565 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:47.527715 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.541273 27460 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.519830 27497 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.527195 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.541708 27565 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.542003 27497 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:47.543426 27565 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:47.543753 27638 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:08:47.544687 27497 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:47.534590 27609 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.706s	user 0.000s	sys 0.003s
I20260709 13:08:47.506003 27633 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:47.505889 27533 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.546701 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2914) took 1395 ms. Trace:
I20260709 13:08:47.525175 27531 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.548002 27483 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:47.547082 21070 rpcz_store.cc:276] 0709 13:08:46.150929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.425549 (+274620us) service_pool.cc:224] Handling call
0709 13:08:46.451372 (+ 25823us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:46.456906 (+  5534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:46.456960 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:46.486617 (+ 29657us) log.cc:844] Serialized 26 byte log entry
0709 13:08:46.629472 (+142855us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:46.629545 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.629647 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.534064 (+904417us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b240013a5e8. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f2b59c4 00007f7f3f2b3acb 00007f7f3f46edc9 00007f7f3f46ea55 00007f7f3f46e09d 00007f7f420a010e 00007f7f420b788c 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221
0709 13:08:47.546392 (+ 12328us) raft_consensus.cc:1685] finished
0709 13:08:47.546416 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.546635 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3510,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":17645,"spinlock_wait_cycles":4167424,"thread_start_us":87172,"threads_started":1,"wal-append.queue_time_us":284961}
I20260709 13:08:47.549544 27531 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:47.549557 21147 raft_consensus.cc:2410] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:08:47.507498 27257 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:47.509143 27413 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.542207 21138 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
W20260709 13:08:47.553686 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 50767us
I20260709 13:08:47.504675 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.578059 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:47.586321 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:47.586323 27483 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:47.587499 27483 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:47.588323 27483 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.589567 27483 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:47.590701 21797 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:47.592379 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.568611 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:47.593472 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.588737 27533 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.596997 21801 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:47.604514 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.605270 21796 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:47.598443 27413 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:47.606945 27413 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:47.607438 27413 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:47.609892 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:47.610625 27551 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.610757 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.611173 27551 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:47.612317 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.613879 20963 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:47.614727 21140 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:47.616230 21140 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:47.611903 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.613464 27551 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:47.617102 20967 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 243) took 43942 ms (client timeout 15000 ms). Trace:
I20260709 13:08:47.618477 21143 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:47.619005 27549 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:47.619746 21147 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:47.608345 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:47.625994 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.634284 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:47.637749 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:47.638142 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.638893 21215 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:47.643698 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:47.644095 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.633921 27551 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:08:47.647104 21215 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:47.622470 27524 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.653206 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:47.653831 21217 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:47.652521 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:47.656133 27598 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.001s
I20260709 13:08:47.656243 27548 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.657042 27548 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.657832 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:47.657899 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.661078 21145 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:47.661156 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:47.662320 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.664306 21072 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:47.664772 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:47.666571 21072 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:47.668433 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:47.669126 21072 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:47.669793 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.668613 21217 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:47.665396 21069 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:47.672297 21068 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:47.627592 27257 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:47.673166 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.677891 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.681326 21147 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:47.684077 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:47.676856 21797 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:47.686334 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:47.698920 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:47.687708 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.686856 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.672084 27548 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.689374 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.690704 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.619479 20963 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 12 to 15, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 15 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:47.691407 27257 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.713413 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:47.691833 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:47.715050 21072 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:47.692075 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:47.692610 27378 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:47.645594 27633 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:47.696270 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.700008 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3180) took 1270 ms. Trace:
I20260709 13:08:47.700464 21147 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:47.627660 27549 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.704521 21142 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:47.672323 27407 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.705217 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.711972 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.714860 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.695802 21146 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:47.715615 21800 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:47.720217 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:47.725025 20963 catalog_manager.cc:5697] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:47.726078 27615 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:47.701891 27568 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.727160 21071 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:47.729177 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.729801 21220 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:47.723996 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:47.717839 21139 rpcz_store.cc:276] 0709 13:08:46.430074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.431831 (+  1757us) service_pool.cc:224] Handling call
0709 13:08:47.248731 (+816900us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:08:47.248933 (+   202us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:47.249337 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:47.249364 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:47.249831 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:08:47.369595 (+119764us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:47.369623 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.369668 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.693434 (+323766us) raft_consensus.cc:1685] finished
0709 13:08:47.693464 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.699937 (+  6473us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3872,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":7698688,"thread_start_us":119464,"threads_started":1,"wal-append.queue_time_us":345870}
I20260709 13:08:47.733695 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.734244 21220 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:47.641484 27546 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.736521 21145 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:08:47.701865 27613 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:47.726536 27615 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:47.763967 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:47.771847 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:47.772505 21068 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:47.717370 27583 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.722286 27565 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:47.776311 21068 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:47.781517 27649 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
W20260709 13:08:47.617509 20967 rpcz_store.cc:269] 0709 13:08:03.674626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.674935 (+   309us) service_pool.cc:224] Handling call
0709 13:08:47.616867 (+43941932us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:38.021173 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:38.217530 (+196357us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:38.217589 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:08:38.217611 (+    22us) tablet.cc:662] Decoding operations
0709 13:08:38.435602 (+217991us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:38.446365 (+ 10763us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:38.460299 (+ 13934us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:08:38.462062 (+  1763us) tablet.cc:701] Row locks acquired
0709 13:08:38.462113 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:08:38.462289 (+   176us) write_op.cc:270] Start()
0709 13:08:38.462448 (+   159us) write_op.cc:276] Timestamp: P: 1783602518462216 usec, L: 0
0709 13:08:38.462478 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:08:38.463228 (+   750us) log.cc:844] Serialized 8747 byte log entry
0709 13:08:39.091494 (+628266us) op_driver.cc:464] REPLICATION: finished
0709 13:08:39.504564 (+413070us) write_op.cc:301] APPLY: starting
0709 13:08:39.510246 (+  5682us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:40.139860 (+629614us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:40.140003 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:40.289564 (+149561us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:40.289955 (+   391us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:08:40.290010 (+    55us) write_op.cc:312] APPLY: finished
0709 13:08:40.330548 (+ 40538us) log.cc:844] Serialized 196 byte log entry
0709 13:08:40.488681 (+158133us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:40.499513 (+ 10832us) write_op.cc:454] Released schema lock
0709 13:08:40.499612 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:40.500061 (+   449us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:40.500089 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":31170,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":366396,"apply.run_cpu_time_us":47747,"apply.run_wall_time_us":1010383,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96131,"num_ops":22,"prepare.queue_time_us":689100,"prepare.run_cpu_time_us":26613,"prepare.run_wall_time_us":653126,"raft.queue_time_us":215284,"raft.run_cpu_time_us":103857,"raft.run_wall_time_us":213659,"replication_time_us":606098,"thread_start_us":1110856,"threads_started":5,"wal-append.queue_time_us":354170,"wal-append.run_cpu_time_us":29267,"wal-append.run_wall_time_us":2378198}]]}
I20260709 13:08:47.726689 20963 catalog_manager.cc:5697] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:47.796074 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:47.782888 27560 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:47.783352 27378 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.721191 27605 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:47.718210 27627 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:08:47.815227 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:47.778398 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:47.791435 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78094.1us
I20260709 13:08:47.726662 27549 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:47.732427 27580 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:47.805733 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:47.837457 27369 tablet_bootstrap.cc:492] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:47.838353 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:47.735513 27633 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:47.782995 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:47.721570 27524 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:08:47.774932 27615 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:47.822425 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:47.732118 27620 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:08:47.829926 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.830741 21068 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:47.737543 27407 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.833251 27497 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:47.833343 20963 catalog_manager.cc:5697] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 2 to 9, term changed from 12 to 14, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:47.834978 27598 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:47.838574 27369 tablet_bootstrap.cc:492] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:08:47.771020 27568 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:47.840690 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 57757.7us
I20260709 13:08:47.844213 21214 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:47.887797 21800 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:08:47.831797 21145 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:08:47.847299 21767 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:47.847914 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:47.840224 27633 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.807991 27560 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:47.891724 27560 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:47.893994 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:47.849540 27325 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:08:47.791381 27613 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.792140 27583 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.849596 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:47.868939 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:47.828697 27378 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:47.880506 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.841050 27626 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:47.891017 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.885912 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:47.909410 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.912007 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:47.920030 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 70531.2us
W20260709 13:08:47.921872 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 74698.3us
I20260709 13:08:47.909987 21796 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:47.865809 27643 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:47.866103 27584 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:47.941059 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.921327 21185 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:47.942883 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:47.865386 27497 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:47.866031 27565 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:47.908983 27218 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:47.915834 27626 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.965579 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.966401 21221 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:47.942286 27611 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:47.967878 27611 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.968297 27611 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:47.940241 27524 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:47.939000 27656 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:47.884852 27588 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.840375 27605 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:47.979404 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 141105us
I20260709 13:08:47.954246 27497 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:47.965382 27369 ts_tablet_manager.cc:1403] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 2.404s	user 0.040s	sys 0.023s
I20260709 13:08:47.965317 27626 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:47.960963 27599 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:47.987105 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:47.961383 27261 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:08:47.988296 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 207381us
I20260709 13:08:47.961776 27584 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:47.951749 27607 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
W20260709 13:08:47.992863 21036 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.544s (SENT)
W20260709 13:08:47.993700 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 126810us
I20260709 13:08:47.996842 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:47.990054 27406 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:48.005012 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.006016 21221 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:47.980490 27565 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:47.993772 27257 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:48.014596 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 334730us
I20260709 13:08:48.029258 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:48.043977 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 120780us
I20260709 13:08:47.996018 27261 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:48.004460 27548 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:48.048606 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:48.045331 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 135752us
I20260709 13:08:48.006546 27008 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:48.006410 27613 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:48.049240 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:48.043537 27549 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:48.053682 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:47.992971 27599 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.049559 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.060206 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.061045 21212 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:48.044201 27614 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:48.065044 27378 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:48.066337 27660 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:08:48.066185 27669 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:48.066998 27261 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.079833 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.081067 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:48.081655 21219 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:48.081758 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:48.072196 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79412.9us
I20260709 13:08:48.067348 20963 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 12 to 17, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 17 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:48.083518 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:48.084246 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:48.084450 27551 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:48.075513 27588 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:48.072935 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 43756.1us
I20260709 13:08:48.068079 27008 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:48.088011 21217 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:48.081152 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:48.089125 21213 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:48.090701 21215 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:48.091822 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:48.072273 27653 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:08:48.052452 27548 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.095458 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.097401 27673 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:48.102298 21212 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:48.087921 27483 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:48.047752 27584 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.093322 27549 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:48.068795 27413 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.077790 27607 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:08:48.116348 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.115034 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.117082 21216 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:48.073859 27614 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.119597 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.111368 27483 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.125187 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.129989 21216 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:48.117353 27549 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.133328 21138 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:48.132552 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.136838 21139 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:08:48.139485 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:48.143862 27661 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:08:48.143947 27568 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:48.144874 27565 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.120694 27616 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:08:48.139792 27605 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.169121 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:48.170058 21035 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:48.084280 20963 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 13 to 15, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 15 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:48.164014 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.173004 21220 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:48.164316 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.163515 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.180800 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.183524 21138 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:48.134183 27483 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:48.143747 27549 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:48.197688 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:48.167083 27652 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 13:08:48.146486 27664 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:48.150269 27407 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.204169 21138 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:48.145249 27568 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:48.145424 27565 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:48.205965 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.213112 26884 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:48.217237 21219 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:48.218945 27615 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.224725 21213 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:48.148846 27560 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.147092 27583 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.148295 27548 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.160684 27616 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.164736 27389 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:48.119832 27497 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.160492 27406 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:48.264694 21221 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:48.199370 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 151061us
I20260709 13:08:48.230618 21214 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:48.234366 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 184877us
I20260709 13:08:48.266613 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.250576 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:48.267617 21144 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:48.247251 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:48.165571 27611 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.248852 27633 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.231673 27257 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.265322 21068 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:48.265964 21767 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:48.279425 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:48.280107 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 140738us
I20260709 13:08:48.281193 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:48.281117 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.238657 27664 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.232177 27369 raft_consensus.cc:359] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:48.297206 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.253189 26884 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.289049 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:48.264439 27548 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:48.268093 27680 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:48.269048 27583 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:48.269388 27406 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:48.297910 21219 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:48.259562 27389 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:48.299996 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.300557 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.274688 27604 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:08:48.318514 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:48.330960 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.282284 27672 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:48.261951 27616 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:48.338899 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 168901us
I20260709 13:08:48.339535 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.274660 20963 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 12 to 19, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 19 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:48.342820 27681 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:08:48.342118 20963 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:48.275084 27605 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:08:48.344086 20963 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 12 to 17, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 17 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:48.345868 27605 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:48.346335 27605 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.346715 27605 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:48.315995 27548 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.285003 27598 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:08:48.302593 27369 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:08:48.315442 27667 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:48.320741 27680 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:48.345492 27384 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:48.286027 27633 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.286770 27257 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.295496 27639 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:08:48.347167 20963 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:48.355548 27551 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:48.382205 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 116354us
I20260709 13:08:48.356668 27604 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:48.363090 27548 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:48.369489 27384 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:48.369472 27684 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:48.389330 27398 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:48.391328 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.391084 27565 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:48.411315 27604 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.415508 27615 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:48.417906 27626 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:48.394429 27686 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:48.421586 27080 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:48.407941 27398 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:48.422299 27384 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.424430 27549 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:48.425578 27483 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.430815 27407 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:48.432233 27611 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.399104 27677 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:08:48.420075 20963 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 11, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 11 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:48.442870 21144 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
I20260709 13:08:48.463934 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.444311 27676 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:48.444996 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 53784us
I20260709 13:08:48.443449 27560 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:48.445307 27689 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:08:48.450896 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 111502us
I20260709 13:08:48.427616 26884 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:48.390637 27672 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:08:48.457849 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:48.468828 27384 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:48.461179 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.462505 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.388991 27389 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.385697 27369 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:48.466531 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.467585 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.474922 21069 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:48.468375 21071 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:48.446796 27080 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:48.469549 21063 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: []
W20260709 13:08:48.469431 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 150272us
I20260709 13:08:48.473306 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.482613 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:48.471406 21070 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:48.473837 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:48.480041 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.486562 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.469168 27694 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:48.492926 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.469231 27689 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:48.501041 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.450553 27684 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.473059 27616 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:48.503940 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.429991 27680 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.489494 27560 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:48.511668 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.494073 27583 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:48.540166 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:48.541311 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54934.6us
I20260709 13:08:48.542050 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.547320 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.547712 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.548846 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.496199 27639 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:48.501272 27690 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:08:48.549469 21071 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:48.553113 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.556560 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:48.498494 27664 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:48.505892 27689 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:48.470413 27676 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.546195 27616 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:48.551684 27680 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:48.566642 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.571496 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:48.539439 27560 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:48.540511 27695 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:48.579049 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:48.581754 27497 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:48.551997 27565 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:48.566043 27686 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:48.589543 21142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ca4319420e54bc19f621da03c811035"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:48.590446 21142 raft_consensus.cc:606] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.590873 21142 raft_consensus.cc:614] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:48.595028 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:48.597065 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 129542us
I20260709 13:08:48.536972 27694 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:48.604817 27693 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:08:48.605232 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:48.605753 27691 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:08:48.606055 21066 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:48.607385 27546 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:48.605646 27694 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.615756 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:48.617128 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.617874 21069 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:48.621809 21066 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:48.623973 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.625231 27532 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:48.625658 27532 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:48.626492 27532 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.627381 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:48.627614 27532 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:48.628162 27677 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:48.628583 27677 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.628916 27677 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:48.633556 27549 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.634555 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:48.635897 27684 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:48.578992 27560 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:48.577761 27551 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:48.590353 27685 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.204s	user 0.001s	sys 0.001s
I20260709 13:08:48.618887 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.582464 27583 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:48.646153 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.646011 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:48.647410 21068 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:48.642171 21794 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:08:48.649994 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55032.1us
I20260709 13:08:48.650983 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:48.661517 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:48.672190 21798 raft_consensus.cc:2368] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:08:48.537317 27398 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.637596 27686 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:48.597991 27700 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:48.564216 27639 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.640920 27218 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.672734 27684 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:48.663056 27583 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:48.674666 27565 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:48.575979 27080 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:48.681602 27626 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.685747 27686 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.685362 27647 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:48.724817 21145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:48.725536 21145 raft_consensus.cc:606] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.725940 21145 raft_consensus.cc:614] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:48.718895 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 102988us
I20260709 13:08:48.728885 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.729583 21214 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:48.729772 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.730176 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.730422 21221 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:48.730789 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.730849 21219 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:48.730432 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:48.729514 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.732164 27616 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:48.732510 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.732776 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:48.732654 27616 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.733319 21221 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:48.734025 20963 catalog_manager.cc:5697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:48.734820 21145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:48.735481 21145 raft_consensus.cc:606] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.710304 27218 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.740854 21221 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:48.709970 27583 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:48.708344 27483 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:48.691438 27546 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:08:48.712316 27398 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:48.759918 27406 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.762305 27406 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.764204 21069 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:48.715999 27565 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:48.686569 27551 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.647490 27695 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.775791 27599 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:48.777475 27695 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.724952 27686 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:48.782641 21145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:48.783377 21145 raft_consensus.cc:606] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.783743 21145 raft_consensus.cc:614] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:48.784307 21071 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:08:48.758484 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:48.756451 21068 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:48.741252 27218 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:48.746167 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.768092 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.769935 27080 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:48.712800 27684 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:48.775403 27676 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.764850 27694 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:08:48.802768 27684 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.803593 27694 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:48.804073 27694 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:48.708947 27677 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.805069 27694 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.806150 21071 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
I20260709 13:08:48.806829 27677 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.808955 21145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:48.809617 21145 raft_consensus.cc:606] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.809983 21145 raft_consensus.cc:614] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:48.815711 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.815371 27565 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.817083 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:48.823367 27695 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:48.823891 27695 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.824031 27639 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:48.824304 27695 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:48.826083 27551 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.756453 27626 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:48.828442 27626 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.769606 27614 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.794422 27546 consensus_queue.cc:1243] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:48.830463 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:08:48.832592 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.833009 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:08:48.833555 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.833869 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:48.834241 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.835063 27584 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:48.835609 27584 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:48.836438 27584 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.840864 21765 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:48.844216 21826 heartbeater.cc:646] Failed to heartbeat to 127.19.127.254:37743 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.127.254:37743 timed out after 15.000s (SENT)
I20260709 13:08:48.851714 27633 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.849493 27689 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.854399 27413 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.854691 27689 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.855821 27583 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:48.856950 21069 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:48.860589 21071 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:48.734933 27616 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.864260 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:48.866110 27703 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:48.866374 27583 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.865689 27369 ts_tablet_manager.cc:1434] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.875s	user 0.046s	sys 0.090s
I20260709 13:08:48.851722 21217 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:48.866961 27369 ts_tablet_manager.cc:1467] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Tablet startup took 877ms
W20260709 13:08:48.868255 27605 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
W20260709 13:08:48.864405 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 118403us
I20260709 13:08:48.890239 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.895145 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.903617 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:48.907006 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.908591 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.904868 27626 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:48.806629 27694 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:48.913311 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 145305us
I20260709 13:08:48.920199 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:48.923215 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 82480.1us
I20260709 13:08:48.925989 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:48.927069 21221 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:48.928351 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:48.930537 21140 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:48.884567 27614 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:48.933415 21138 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:48.939318 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:48.922690 27548 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.511s	user 0.005s	sys 0.007s
I20260709 13:08:48.923604 27677 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:48.949728 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.910775 27703 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:48.968447 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.950724 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:48.972348 21800 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:48.930495 27689 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:48.951493 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.922209 27639 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:48.744489 27647 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:48.952131 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.921728 27599 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:48.954330 21797 raft_consensus.cc:1217] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEARNER]: Deduplicated request from leader. Original: 18.10->[18.11-18.11]   Dedup: 18.11->[]
I20260709 13:08:48.955755 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:48.956569 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:48.999580 21141 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:48.956645 27611 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:48.957312 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:48.957327 21110 leader_election.cc:400] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:08:49.000860 21110 leader_election.cc:403] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:48.960979 27257 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:48.944221 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.962805 21142 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:08:48.910009 27706 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:48.951354 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.976974 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:49.011346 21800 raft_consensus.cc:2368] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:48.978061 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:48.946596 27532 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:48.906010 27605 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:48.981923 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:49.034444 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:48.983814 21147 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:48.984735 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:49.036176 21801 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:48.922757 27580 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
W20260709 13:08:48.989986 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:49.037078 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.929330 27674 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:08:49.019922 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:49.020929 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 65587.8us
I20260709 13:08:49.021190 21068 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:08:49.022213 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 78078.2us
I20260709 13:08:49.044340 21068 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:49.044873 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:48.928201 27673 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:08:49.045513 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:49.011855 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:49.018823 21214 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:48.998250 27639 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.955165 27548 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:49.043880 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:49.056270 21065 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:49.056907 21797 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:49.010599 27611 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:48.964762 27614 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:49.033100 27689 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.038144 27613 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:49.038763 27643 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:08:49.091879 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:49.071601 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:49.102610 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:49.076850 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:49.095822 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:49.047832 27682 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.104808 21065 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:49.142570 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:49.059685 27639 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:49.096751 21068 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:49.149576 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:49.170878 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 93675.6us
I20260709 13:08:49.173339 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:49.153461 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 81936.1us
I20260709 13:08:49.147313 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:49.090219 27524 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:49.210610 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:49.180959 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:49.171494 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:49.107563 27607 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:08:49.218147 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 70970.3us
I20260709 13:08:49.174283 21139 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:49.173494 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:49.223345 21113 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:49.215472 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:49.173620 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:49.213306 21068 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:49.105743 27483 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.216274 21138 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:49.218500 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 115677us
W20260709 13:08:49.171154 27659 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 13:08:49.299217 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 53735.2us
I20260709 13:08:49.299712 27599 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:49.300002 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:49.296994 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:49.301973 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:49.212395 27389 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:49.229172 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56787.2us
I20260709 13:08:49.305864 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:49.109773 27689 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:49.305820 27378 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:49.317399 27689 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:49.303382 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:49.090675 27532 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.125s
I20260709 13:08:49.319007 21139 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:49.304191 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:49.071558 27665 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:08:49.297223 27715 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:08:49.316629 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:49.298424 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 85107.5us
I20260709 13:08:49.317962 27378 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:49.227819 27607 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.175519 27413 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:49.176554 27588 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:49.176350 27080 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:49.297580 27714 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.258s	user 0.000s	sys 0.000s
I20260709 13:08:49.335254 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:49.345112 21140 raft_consensus.cc:2368] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25.
I20260709 13:08:49.326012 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:49.319181 27398 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.001s
I20260709 13:08:49.071310 27706 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.237s
I20260709 13:08:49.292297 27669 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:49.244886 27640 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:49.324816 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 81965.9us
I20260709 13:08:49.148638 27682 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:49.149005 27686 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:49.368134 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 291325us
I20260709 13:08:49.408643 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:49.325785 27665 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:49.405452 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 54091us
I20260709 13:08:49.244345 27483 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:49.409652 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 74495.6us
I20260709 13:08:49.415740 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:49.409577 21220 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:49.361202 27607 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:49.414378 27665 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:48.884673 27369 ts_tablet_manager.cc:1469] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Trace:
0709 13:08:39.509463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.509685 (+   222us) service_pool.cc:224] Handling call
0709 13:08:39.872987 (+363302us) ts_tablet_manager.cc:934] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
0709 13:08:42.604472 (+2731485us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:08:43.187056 (+582584us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:43.187172 (+   116us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:08:43.740529 (+553357us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:08:43.797327 (+ 56798us) tablet_replica.cc:206] Creating consensus instance
0709 13:08:43.801210 (+  3883us) ts_tablet_manager.cc:989] Replica registered.
0709 13:08:43.801276 (+    66us) inbound_call.cc:177] Queueing success response
0709 13:08:45.561117 (+1759841us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:08:47.994055 (+2432938us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:08:48.125396 (+131341us) tablet_replica.cc:261] Starting instrumentation
0709 13:08:48.126090 (+   694us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":5115,"mutex_wait_us":32397,"raft.queue_time_us":83067,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":10769,"tablet-copy.queue_time_us":222442,"tablet-download-pool.queue_time_us":9994,"tablet-download-pool.run_cpu_time_us":5648,"tablet-download-pool.run_wall_time_us":1257757,"thread_start_us":482685,"threads_started":3}
I20260709 13:08:49.423357 21065 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:49.426584 21071 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:08:49.430438 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:49.437415 27378 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:49.438819 21184 leader_election.cc:400] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:49.439172 21184 leader_election.cc:403] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:49.366948 27616 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:08:49.441635 27613 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:49.443797 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:49.432652 21064 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:49.433655 21070 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:49.450160 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:49.451447 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:49.456014 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:49.414367 27580 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:49.428479 27691 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:08:49.458161 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:49.460647 27614 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:49.460848 21796 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:49.457762 27598 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:49.483461 27588 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:49.484570 27532 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:49.484963 27532 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:49.487077 21139 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:49.493657 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:49.497640 21068 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:49.497835 21070 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:49.503491 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:49.454144 27647 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:08:49.510375 27218 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:49.477324 27584 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:08:49.511106 27218 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:49.512220 27682 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:49.511075 27723 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:08:49.485829 27532 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:49.474279 27599 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:49.456284 27684 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:49.457120 27261 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:49.477042 27524 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:49.516064 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:49.513618 27560 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.829s	user 0.012s	sys 0.000s
I20260709 13:08:49.512365 27218 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:49.506907 27586 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:08:49.506904 27613 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:49.528491 27524 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:49.526894 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:49.513229 27605 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:49.525476 27261 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:49.513424 27598 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:08:49.534377 27532 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:49.551811 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:49.539573 27684 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:49.560482 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:49.538755 27560 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:49.565567 21072 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:49.569991 27560 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:49.573619 27665 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:49.578123 27695 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.753s	user 0.006s	sys 0.003s
I20260709 13:08:49.583863 27684 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:49.563714 27261 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:49.589352 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:49.600234 21068 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:49.591042 27605 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:49.555773 27524 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:49.603325 21066 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
W20260709 13:08:49.597515 27677 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.002s
I20260709 13:08:49.712739 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:49.719220 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:49.885025 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:49.885437 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:49.892211 27706 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:49.892851 27706 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:49.885102 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:49.896139 27325 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:49.896741 27325 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:49.897086 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:49.897791 21214 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:49.900328 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:49.903131 27706 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:49.909650 27669 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:49.910405 27669 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:49.911525 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 502285us
I20260709 13:08:49.913308 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1132) took 1141 ms. Trace:
I20260709 13:08:49.913712 21071 rpcz_store.cc:276] 0709 13:08:48.771561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.861426 (+ 89865us) service_pool.cc:224] Handling call
0709 13:08:48.861684 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:49.427061 (+565377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.913024 (+485963us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400214654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:49.913234 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1298553088}
I20260709 13:08:49.914505 27620 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:08:49.915318 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 506686us
W20260709 13:08:49.916374 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 467920us
I20260709 13:08:49.916903 21071 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
W20260709 13:08:49.922047 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:49.926678 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:49.915481 27640 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:49.932117 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:49.932754 21215 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:49.890841 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:49.934504 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 43786.5us
I20260709 13:08:49.935464 21214 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:49.610140 27676 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:49.933279 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:49.957581 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:49.942600 21216 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:49.943535 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:49.609587 27686 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:49.710427 27677 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:49.600340 27695 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:49.709811 27673 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:49.972553 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:49.709474 27389 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:49.708472 27483 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:49.604028 27605 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:49.942644 27689 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:49.972608 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:49.605029 27607 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:49.933086 27584 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.587060 27611 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:49.993835 21795 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:49.954260 27680 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.998703 21216 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: []
W20260709 13:08:49.979969 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.002140 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:50.006147 27680 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.002214 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:49.953780 27676 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:50.011904 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:49.978581 27682 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:49.982478 27684 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:49.977509 27616 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:50.034351 21139 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:50.017350 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:50.039673 21145 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:08:50.044548 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:50.054680 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:50.056942 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:50.017423 21213 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:50.013644 27686 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:50.052037 27560 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:08:50.028748 27639 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.000s
I20260709 13:08:50.010347 27730 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.063863 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:50.024392 27640 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:50.011400 27583 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:50.030836 27684 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:50.034283 27257 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:50.046509 27680 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:50.025910 27682 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:50.069260 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 91383.7us
I20260709 13:08:50.041817 27611 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:50.029405 27734 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:50.001238 27389 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.077623 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:50.064519 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:50.027932 27584 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:49.996950 27673 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.109660 21147 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:50.023731 27378 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
W20260709 13:08:50.098361 21187 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.678s (SENT) [suppressed 3 similar messages]
I20260709 13:08:50.065500 27686 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:50.085186 27257 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:50.087062 27583 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.116034 27257 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:50.115819 27686 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:50.117679 27583 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:50.057432 27613 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:08:50.088369 27684 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:50.047034 27689 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:08:50.067096 27733 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:50.108623 27682 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:50.131356 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 68307.4us
I20260709 13:08:50.070113 21157 tablet_copy_service.cc:342] P 5e5e65640c8b4f5ca7eadc6a14312c55: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-6beb666b5d504215a141add24a6d632f received from {username='slave'} at 127.0.0.1:50996
I20260709 13:08:50.169428 21794 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:08:50.142072 21140 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:08:50.144191 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.145444 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2044) took 1536 ms. Trace:
I20260709 13:08:50.171458 21072 rpcz_store.cc:276] 0709 13:08:48.608650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.726875 (+118225us) service_pool.cc:224] Handling call
0709 13:08:48.727215 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.566011 (+838796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.145181 (+579170us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b540011b7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:50.145387 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927603584}
I20260709 13:08:50.172978 21072 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:50.148065 21220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 3203) took 1418 ms (client timeout 1578 ms). Trace:
W20260709 13:08:50.173933 21220 rpcz_store.cc:269] 0709 13:08:48.729161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.729282 (+   121us) service_pool.cc:224] Handling call
0709 13:08:50.147869 (+1418587us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54002ee5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:50.147999 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485519232}
I20260709 13:08:50.070503 27730 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.072099 27676 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:50.146955 21064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2930) took 1683 ms (client timeout 1678 ms). Trace:
W20260709 13:08:50.175590 21064 rpcz_store.cc:269] 0709 13:08:48.463916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.464350 (+   434us) service_pool.cc:224] Handling call
0709 13:08:50.146709 (+1682359us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b540001aed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:50.146851 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063835520}
I20260709 13:08:50.176745 21215 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:50.161139 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:50.177973 21064 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:08:50.163414 27389 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:50.178035 21142 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:08:50.164237 27735 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.455s	user 0.001s	sys 0.000s
W20260709 13:08:50.159143 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.148043 21069 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:08:50.070272 27639 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.170111 21218 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:50.183954 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:50.068439 27616 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.196787 21036 leader_election.cc:400] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25.
I20260709 13:08:50.197220 21036 leader_election.cc:403] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:50.166172 27583 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:50.242466 21796 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:50.245842 21215 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
W20260709 13:08:50.239948 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 102519us
I20260709 13:08:50.169164 27647 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:50.251797 21219 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 13:08:50.256214 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 157872us
W20260709 13:08:50.258041 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 61387.7us
I20260709 13:08:50.239053 27080 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.199275 27680 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:50.270177 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:50.282550 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:50.250233 27533 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.272985 21220 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:50.238266 21157 tablet_copy_service.cc:434] P 5e5e65640c8b4f5ca7eadc6a14312c55: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-6beb666b5d504215a141add24a6d632f on tablet 6beb666b5d504215a141add24a6d632f with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:50.254946 27483 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:50.257624 27694 consensus_meta.cc:287] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.333s	user 0.002s	sys 0.010s
I20260709 13:08:50.270977 21215 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:50.309070 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 54713.7us
I20260709 13:08:50.241133 27733 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:50.252027 27703 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.240152 27389 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:50.257880 27689 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:08:50.315923 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 45830.2us
I20260709 13:08:50.282404 27695 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:50.341213 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:50.323266 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:50.266021 27560 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25.
I20260709 13:08:50.342926 21063 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
W20260709 13:08:50.344798 27742 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:08:50.319345 27080 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:50.319126 27680 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:50.266050 27647 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:08:50.316864 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 159844us
I20260709 13:08:50.346896 27586 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:50.358446 27565 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:50.366904 21220 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:50.393589 27565 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.394371 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:50.395685 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:50.388669 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:50.325594 27406 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:50.348748 27733 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.318778 27758 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:50.377636 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:50.349041 27689 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:50.348778 27607 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:50.315065 27734 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:08:50.370582 27531 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.372001 27605 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:50.410274 27746 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:08:50.409974 27689 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.411175 27734 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:50.317194 27740 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.113s
I20260709 13:08:50.393632 27586 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.377127 27583 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.323120 27483 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:50.413568 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 90502.9us
I20260709 13:08:50.383560 27695 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:50.336038 27533 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:50.494230 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 310384us
W20260709 13:08:50.493420 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 88806.6us
W20260709 13:08:50.479444 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.426101 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:50.424909 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:50.377138 27389 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.578894 21217 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:50.346035 27703 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:50.500887 27717 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:08:50.598397 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 275057us
I20260709 13:08:50.593189 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:50.414790 27749 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:50.420284 27758 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:50.649423 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56335us
I20260709 13:08:50.649974 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:50.650332 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:50.594022 27580 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:50.653221 27261 consensus_meta.cc:287] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.049s	user 0.008s	sys 0.003s
I20260709 13:08:50.653456 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:50.617314 27749 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.654915 27733 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:50.416764 27406 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:50.656515 27565 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:50.658393 27483 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:50.415834 27748 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.239s
W20260709 13:08:50.663190 21065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 2057) took 1674 ms (client timeout 1750 ms). Trace:
W20260709 13:08:50.693655 21065 rpcz_store.cc:269] 0709 13:08:48.988475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.180618 (+192143us) service_pool.cc:224] Handling call
0709 13:08:50.663023 (+1482405us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540026e654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:50.663130 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474616576}
I20260709 13:08:50.663319 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:50.694933 21141 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:50.664580 27758 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.665814 27677 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:50.589915 27731 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:50.667145 27725 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:08:50.669319 27686 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.552s	user 0.000s	sys 0.007s
I20260709 13:08:50.670694 27703 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:50.525817 27583 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.692099 27699 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:50.671556 27588 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:50.671571 27728 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 1.112s	user 0.003s	sys 0.000s
I20260709 13:08:50.601817 27734 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.673017 27730 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:50.673431 27560 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:50.673483 27643 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:50.622197 27673 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:50.677083 27389 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:50.680558 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:50.418111 27762 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.271s
I20260709 13:08:50.654870 27533 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:50.415690 27531 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.702159 27694 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:50.588853 27605 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.090s
W20260709 13:08:50.694411 27257 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.574s	user 0.017s	sys 0.002s
I20260709 13:08:50.599138 27695 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.672653 27616 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:50.689227 27749 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:50.699811 27565 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.700899 27725 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:50.712817 27771 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:50.691005 27261 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:50.728575 27760 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:08:50.704392 27677 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:50.709179 27731 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.728542 27588 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.702543 27640 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:08:50.690533 27580 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.705798 27686 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.735409 27560 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:50.737723 27584 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:50.745169 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 64708.5us
W20260709 13:08:50.755223 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.867275 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 832) took 1154 ms. Trace:
I20260709 13:08:50.817363 21141 raft_consensus.cc:2410] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:50.849341 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:50.881422 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:50.928721 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 261313us
I20260709 13:08:50.882587 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1145) took 1434 ms. Trace:
I20260709 13:08:51.005897 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.006450 21141 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
W20260709 13:08:51.006594 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.880561 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2954) took 1373 ms. Trace:
I20260709 13:08:50.929478 21147 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:51.007614 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:50.844838 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:50.769200 21220 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:08:50.842586 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2952) took 2222 ms. Trace:
W20260709 13:08:50.862035 21066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 2055) took 2247 ms (client timeout 1900 ms). Trace:
I20260709 13:08:50.989595 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:50.994717 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:50.801796 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1135) took 1841 ms. Trace:
I20260709 13:08:50.989816 21219 rpcz_store.cc:276] 0709 13:08:49.713243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.713521 (+   278us) service_pool.cc:224] Handling call
0709 13:08:50.141422 (+427901us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:08:50.141627 (+   205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:50.264072 (+122445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.866988 (+602916us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400102f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:50.867211 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292786816}
I20260709 13:08:51.017923 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:51.006178 21068 rpcz_store.cc:276] 0709 13:08:49.448137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.065836 (+617699us) service_pool.cc:224] Handling call
0709 13:08:50.091116 (+ 25280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.189180 (+ 98064us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:50.377123 (+187943us) log.cc:844] Serialized 14 byte log entry
0709 13:08:50.590880 (+213757us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:50.590944 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:50.590995 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.882366 (+291371us) spinlock_profiling.cc:243] Waited 97.7 ms on lock 0x7b540026e154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:50.882416 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:50.882536 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":181069,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":295865,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":80,"spinlock_wait_cycles":224677632,"thread_start_us":354767,"threads_started":2}
I20260709 13:08:51.094901 21139 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:51.144747 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 155245us
I20260709 13:08:50.826222 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.012871 21069 rpcz_store.cc:276] 0709 13:08:48.620195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.859845 (+239650us) service_pool.cc:224] Handling call
0709 13:08:50.147669 (+1287824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.182002 (+ 34333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.842373 (+660371us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540001aed0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:50.842523 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2960980224}
I20260709 13:08:51.052733 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.097669 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.161859 21064 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:51.159262 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:51.119441 21035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.003s	sys 0.010s
W20260709 13:08:51.164212 21035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.004s	sys 0.011s
I20260709 13:08:51.007647 21072 rpcz_store.cc:276] 0709 13:08:49.507017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.174042 (+667025us) service_pool.cc:224] Handling call
0709 13:08:50.174345 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.174538 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:50.256919 (+ 82381us) log.cc:844] Serialized 14 byte log entry
0709 13:08:50.880104 (+623185us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b1800457f60. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287
0709 13:08:50.880208 (+   104us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:50.880257 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:50.880304 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.880391 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:50.880507 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8879,"prepare.queue_time_us":171841,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":233865,"raft.run_cpu_time_us":4161,"raft.run_wall_time_us":53762,"spinlock_wait_cycles":4058496,"thread_start_us":242636,"threads_started":3,"wal-append.queue_time_us":156658}
I20260709 13:08:50.875644 27413 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:08:51.014307 21066 rpcz_store.cc:269] 0709 13:08:48.614238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.623702 (+  9464us) service_pool.cc:224] Handling call
0709 13:08:50.847273 (+2223571us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b54003ca854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:50.861935 (+ 14662us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2250680192}
I20260709 13:08:51.091141 21067 rpcz_store.cc:276] 0709 13:08:48.960270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.960433 (+   163us) service_pool.cc:224] Handling call
0709 13:08:48.960733 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:48.961057 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:48.961085 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:48.961564 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:08:49.427089 (+465525us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:49.427149 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.427317 (+   168us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:50.670914 (+1243597us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:50.767529 (+ 96615us) raft_consensus.cc:1685] finished
0709 13:08:50.767562 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:50.801699 (+ 34137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5045,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":10386,"spinlock_wait_cycles":79032192,"thread_start_us":465084,"threads_started":1,"wal-append.queue_time_us":570484}
I20260709 13:08:50.829978 27080 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:50.820983 27611 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.000s
I20260709 13:08:50.861443 27531 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:51.175122 21144 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:50.880330 27774 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:50.837385 27680 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:51.169167 21221 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
W20260709 13:08:50.831336 27684 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.661s	user 0.011s	sys 0.000s
I20260709 13:08:51.172312 21066 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:50.862960 27706 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.322s
I20260709 13:08:50.860036 27740 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.326s
I20260709 13:08:51.176951 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:51.181558 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.788184 27565 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.173624 27413 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:51.222613 21212 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
W20260709 13:08:50.869344 27524 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.153s	user 0.004s	sys 0.006s
I20260709 13:08:50.880146 27218 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.835421 27695 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:51.097209 27682 consensus_meta.cc:287] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.915s	user 0.010s	sys 0.001s
I20260709 13:08:50.883968 27605 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:50.838639 27665 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:50.885929 27770 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:51.188771 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:50.860792 27583 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:50.829563 27586 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.192142 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.097013 27626 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:51.190656 21797 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
W20260709 13:08:50.902098 27669 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.991s	user 0.011s	sys 0.004s
I20260709 13:08:51.174706 27761 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:51.211580 27765 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:08:51.192600 27560 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.754400 27694 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:50.752223 27406 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.206991 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.173880 27614 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.754420 27257 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.827006 27734 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.168972 27767 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:51.183383 27733 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.191936 27548 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.193457 27686 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.237772 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:50.866454 27647 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:50.887670 27325 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:50.845507 27749 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.245821 27533 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:51.265709 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 805) took 1316 ms. Trace:
W20260709 13:08:51.308667 21063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 2953) took 1802 ms (client timeout 1553 ms). Trace:
I20260709 13:08:51.102594 27771 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.211117 27684 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:51.269073 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 432087us
I20260709 13:08:51.102880 27699 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:51.279985 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 98778.7us
I20260709 13:08:51.251507 27378 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:51.260231 27580 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:51.202301 27774 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:51.299050 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 122176us
I20260709 13:08:51.228858 27677 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.249706 27218 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.314126 21071 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:51.307935 21795 rpcz_store.cc:276] 0709 13:08:49.937118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.937264 (+   146us) service_pool.cc:224] Handling call
0709 13:08:49.943034 (+  5770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.005810 (+ 62776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.253253 (+1247443us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b54003bb854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:51.253500 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116370688}
I20260709 13:08:51.314849 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.339303 21139 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:51.273866 27626 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.254084 27682 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:51.309299 21063 rpcz_store.cc:269] 0709 13:08:49.506454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.551499 (+ 45045us) service_pool.cc:224] Handling call
0709 13:08:51.308419 (+1756920us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400214b54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:51.308569 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1750013696}
I20260709 13:08:51.267273 27605 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.328014 21140 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:51.332891 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:51.333952 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:51.335562 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:51.244656 27524 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.305151 27560 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:51.259851 27413 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:51.336473 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 169787us
I20260709 13:08:51.316815 21070 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:08:51.225461 27725 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.344946 21069 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:51.275636 27669 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.355700 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:51.344136 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:51.375026 21799 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:51.367008 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:51.375653 21070 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:51.320286 27767 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:51.372987 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.319159 27531 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:08:51.310881 27614 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.381345 21063 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:08:51.341053 27261 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:51.385321 21213 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:51.313550 27733 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:51.385727 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:51.387621 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:51.388679 21072 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:51.328378 27533 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:51.391045 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.395040 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.337981 27378 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.395831 21145 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:51.337805 27695 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:08:51.396313 21142 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:51.373750 27588 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.403314 27779 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:51.403888 27779 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:51.403965 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:51.397173 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 63298.1us
I20260709 13:08:51.405120 27739 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.405514 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:51.384097 27688 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:51.406306 27781 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.404402 27779 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:51.406973 27781 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.409104 27781 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.409624 27781 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:51.409930 27781 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:51.406378 27688 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:51.410269 27781 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:51.411307 27776 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:51.404341 27790 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:08:51.411906 27776 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.414165 27776 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.414924 27586 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:51.415675 27586 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:51.416769 27780 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.417088 27758 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:51.417311 27780 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.425402 27257 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.429292 27690 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:51.431262 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3217) took 1155 ms. Trace:
I20260709 13:08:51.431814 21220 rpcz_store.cc:276] 0709 13:08:50.275773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.396552 (+120779us) service_pool.cc:224] Handling call
0709 13:08:50.396844 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.011625 (+614781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.430943 (+419318us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54004349d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:51.431180 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848122112}
I20260709 13:08:51.436067 27633 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.436697 27633 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.437419 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.438244 21799 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:51.439363 27524 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:51.439604 27633 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.441533 21144 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:51.442938 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 838) took 1580 ms. Trace:
I20260709 13:08:51.443655 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:51.444592 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 841) took 1556 ms. Trace:
I20260709 13:08:51.444896 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.445561 21794 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:51.431332 21070 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:51.443346 21218 rpcz_store.cc:276] 0709 13:08:49.862855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.900036 (+ 37181us) service_pool.cc:224] Handling call
0709 13:08:51.442795 (+1542759us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540015d454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:51.442887 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618676992}
I20260709 13:08:51.386250 27765 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.388175 27665 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.405650 27739 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:51.454188 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.389374 27413 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:51.465435 21139 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:08:51.396202 27620 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:51.471113 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 107096us
I20260709 13:08:51.444931 21216 rpcz_store.cc:276] 0709 13:08:49.887969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.007340 (+119371us) service_pool.cc:224] Handling call
0709 13:08:50.007555 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.007766 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:50.143219 (+135453us) log.cc:844] Serialized 14 byte log entry
0709 13:08:51.444116 (+1300897us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b18003726a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc23a
0709 13:08:51.444286 (+   170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:51.444330 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:51.444381 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.444438 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:51.444541 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":244876,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":1542,"spinlock_wait_cycles":91124608,"thread_start_us":299213,"threads_started":2,"wal-append.queue_time_us":274708}
I20260709 13:08:51.487062 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.397810 27731 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.492111 27779 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.448395 27740 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:08:51.515981 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 160346us
I20260709 13:08:51.519085 21066 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:51.518420 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.515460 27626 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.382781 27767 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.525048 27780 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.532126 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.533658 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.537580 27639 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.539778 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.541365 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.543893 27605 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:51.545241 27378 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:51.545464 21212 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 833) took 1831 ms (client timeout 1515 ms). Trace:
W20260709 13:08:51.545837 21212 rpcz_store.cc:269] 0709 13:08:49.713547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.718175 (+  4628us) service_pool.cc:224] Handling call
0709 13:08:50.849111 (+1130936us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:08:51.545252 (+696141us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54003fa7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:51.545352 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415561984}
I20260709 13:08:51.548085 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2128) took 1297 ms. Trace:
I20260709 13:08:51.548669 21213 rpcz_store.cc:276] 0709 13:08:50.242192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.248443 (+  6251us) service_pool.cc:224] Handling call
0709 13:08:50.248724 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.385807 (+1137083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.538950 (+153143us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540034c6d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:51.539211 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2613496576}
I20260709 13:08:51.534911 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.543203 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.552929 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.553829 21140 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:51.555349 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.474996 27665 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.552075 21143 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:51.561885 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.564972 21065 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:08:51.569820 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.570739 21143 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:51.572315 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.576630 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.519232 27740 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:51.584774 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.581418 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.579627 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:51.556239 27780 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:51.472004 27765 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.559120 27378 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:51.561096 27497 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:51.464300 27699 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.578843 21143 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
W20260709 13:08:51.598909 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 151360us
I20260709 13:08:51.599262 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:51.597147 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.605489 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:51.608675 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:51.586315 27779 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.612017 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.614230 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:51.618775 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.623596 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.624270 21218 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:51.624361 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.625059 21220 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:51.626448 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.626981 21220 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:51.629848 21220 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:51.629794 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.604291 27605 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:51.605413 27682 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:51.638707 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62430.6us
I20260709 13:08:51.639699 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:51.640193 21071 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:08:51.643893 27218 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.646061 21142 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:51.631531 27611 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:51.632153 27782 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:08:51.615717 27726 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.619071 27706 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:51.648262 27726 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.648658 27706 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:51.649142 27706 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:51.616452 27406 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:51.650800 27406 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:51.651819 27565 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.652706 27406 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.647619 27611 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:51.654826 27611 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:51.606876 27497 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.644006 27533 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.630995 21218 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:08:51.668519 27793 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:08:51.668747 27732 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 2.084s	user 0.054s	sys 0.048s
I20260709 13:08:51.669759 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.597756 27780 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.671773 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:51.633191 27669 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.671712 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:51.606104 27378 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.673449 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.672748 21071 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:51.688364 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:51.690522 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:51.711328 21067 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:51.700788 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:51.711690 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:51.712760 27779 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:51.701737 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:51.701861 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.717267 27733 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.718775 27605 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:51.719416 27605 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:51.720338 27605 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.721765 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:51.721542 27781 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.723424 21140 raft_consensus.cc:2368] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:08:51.724126 21068 raft_consensus.cc:2376] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19. Re-sending same reply.
I20260709 13:08:51.725217 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.725723 27694 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.726294 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.726418 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:51.727223 21071 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:51.727582 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.728753 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.728917 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.729624 21139 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:51.729849 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:51.730194 27781 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.731870 27736 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:51.732007 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.732815 21072 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:51.733924 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:51.685185 27789 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:08:51.734946 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.666656 27792 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:51.748260 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.749816 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.730535 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.732447 27736 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.739216 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.739756 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:51.757817 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.758798 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:51.760159 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:51.742352 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.761373 21072 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:51.664248 27614 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.762058 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.762075 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:51.740844 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.762797 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:51.732663 27497 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.763131 21072 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:51.745675 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.747558 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.754043 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.742900 21071 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:51.709898 27770 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:51.755205 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.768472 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:51.769217 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:51.769217 21064 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:51.769683 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.756304 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.730638 21068 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 14: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:51.757113 21067 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:51.743736 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.765623 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.686715 27774 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:51.766098 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.767155 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:51.770112 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.776198 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:51.776831 21214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2117) took 1856 ms (client timeout 2214 ms). Trace:
I20260709 13:08:51.764796 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.771761 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.777773 21072 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:51.777887 21067 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:51.778569 21064 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:51.739836 27218 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:51.779959 21072 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
W20260709 13:08:51.777112 21214 rpcz_store.cc:269] 0709 13:08:49.920905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.953173 (+ 32268us) service_pool.cc:224] Handling call
0709 13:08:51.769960 (+1816787us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:08:51.776781 (+  6821us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18566912}
I20260709 13:08:51.755618 27524 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.758024 27667 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:51.765810 27733 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:51.716586 27682 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:08:51.716398 27782 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:08:51.783043 21801 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:51.786322 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:51.787149 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:51.787704 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.788033 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:51.789058 27770 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:51.790504 27667 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:51.850016 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.850956 21068 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:51.780575 21071 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
W20260709 13:08:51.793417 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.790112 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:51.769388 27565 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:51.818681 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:51.853543 27713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.134s	user 0.002s	sys 0.019s
W20260709 13:08:51.854130 27713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.272s	user 0.002s	sys 0.020s
I20260709 13:08:51.857518 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:51.858004 27809 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
I20260709 13:08:51.858520 21147 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:51.866703 27780 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.868274 27765 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:51.870990 27736 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:51.871531 27772 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.188s	user 0.003s	sys 0.001s
I20260709 13:08:51.873349 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3243) took 1850 ms. Trace:
I20260709 13:08:51.875137 21146 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:51.818708 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.748667 27781 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:08:51.771745 27767 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.775615 27779 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:51.825699 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:51.833034 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:51.789779 27702 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:51.760164 27792 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.835780 27802 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:51.836900 27805 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:08:51.876863 27667 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:51.839977 21218 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:51.900699 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:51.901535 21218 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:51.840718 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:51.840651 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:51.841202 21216 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:51.843645 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.902763 21064 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:08:51.903683 21065 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:08:51.847592 27770 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:51.876012 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 85972.9us
I20260709 13:08:51.876343 21219 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
W20260709 13:08:51.876356 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 118117us
I20260709 13:08:51.796082 27524 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.790699 27218 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.887254 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:51.888652 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.888908 21139 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:51.891639 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:51.873769 21138 rpcz_store.cc:276] 0709 13:08:50.022958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.026622 (+  3664us) service_pool.cc:224] Handling call
0709 13:08:50.027063 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:50.284177 (+257114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:50.314626 (+ 30449us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:50.316157 (+  1531us) log.cc:844] Serialized 26 byte log entry
0709 13:08:50.395076 (+ 78919us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:50.395110 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.395175 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:51.836650 (+1441475us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:51.873012 (+ 36362us) raft_consensus.cc:1685] finished
0709 13:08:51.873040 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:51.873274 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18906,"raft.queue_time_us":49317,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":17066,"spinlock_wait_cycles":50834688,"thread_start_us":218482,"threads_started":2,"wal-append.queue_time_us":254278}
I20260709 13:08:51.880250 27694 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:51.757545 27604 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:51.840489 27733 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:51.906209 27770 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:51.921195 21212 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:51.908071 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:51.922979 21799 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:51.886181 27583 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:51.896374 27792 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.896855 27802 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.897349 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:51.874809 27740 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.895898 27689 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:51.924566 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:51.926203 21216 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:51.888208 27779 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:51.929540 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:51.930748 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:51.943950 21219 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:51.950548 21138 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:51.933023 21213 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:51.893707 27702 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:51.953501 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 66344.3us
I20260709 13:08:51.954412 27682 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:51.955075 27734 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:51.955890 27802 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:51.956736 21215 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:51.958040 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:51.924235 27733 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:51.966857 21064 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:51.929240 27583 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:51.920442 27524 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:51.959833 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:51.972934 21221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2118) took 2051 ms (client timeout 2119 ms). Trace:
W20260709 13:08:51.976404 21221 rpcz_store.cc:269] 0709 13:08:49.921205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.951128 (+ 29923us) service_pool.cc:224] Handling call
0709 13:08:50.826061 (+874933us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b58001e4298. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:08:51.972748 (+1146687us) spinlock_profiling.cc:243] Waited 9.88 ms on lock 0x7b5400442ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:51.972863 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50606720}
W20260709 13:08:51.974548 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 68094.9us
I20260709 13:08:51.979728 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:51.952225 27779 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:51.920840 27531 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:08:51.920804 27639 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
W20260709 13:08:51.986994 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.930135 27604 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:51.931668 27565 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:08:51.973464 27767 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:51.968582 27802 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.973781 27740 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:51.929342 27694 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:51.976706 27583 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:51.995814 21217 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:51.990108 27580 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:51.993028 27740 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.014640 27684 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.075124 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:52.016144 27706 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.018520 27665 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:51.995644 27565 consensus_queue.cc:1243] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:51.994302 27736 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:52.023241 27533 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.023428 27795 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:52.023825 27776 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:52.104918 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 120444us
W20260709 13:08:52.025691 27560 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.000s
I20260709 13:08:52.026468 27790 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:52.027038 27218 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:52.027570 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:52.026415 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 185843us
I20260709 13:08:52.031960 27676 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:52.033839 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 803) took 2097 ms. Trace:
I20260709 13:08:52.033911 27611 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:52.045307 27751 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.429s	user 0.001s	sys 0.001s
I20260709 13:08:52.048866 27694 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:52.049598 27531 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:52.050554 27604 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.052453 27378 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:52.007766 27580 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:52.056193 27740 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:52.056872 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.057159 27725 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:52.064666 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:52.068434 21218 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:52.071872 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.078203 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.038968 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:52.099095 21217 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:52.008873 27812 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:52.102743 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.025521 27702 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.110280 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.110778 21220 raft_consensus.cc:2368] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18.
I20260709 13:08:51.997080 27802 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.108566 27533 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.119724 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.120048 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:52.120605 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:52.123145 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.134164 21145 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:52.114372 21798 rpcz_store.cc:276] 0709 13:08:49.936404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.936556 (+   152us) service_pool.cc:224] Handling call
0709 13:08:49.936948 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:50.648056 (+711108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:50.648099 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:50.651315 (+  3216us) log.cc:844] Serialized 26 byte log entry
0709 13:08:51.007558 (+356243us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:51.007607 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.033411 (+1025804us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400633dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:52.033464 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:52.033514 (+    50us) raft_consensus.cc:1685] finished
0709 13:08:52.033531 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:52.033747 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28825,"raft.queue_time_us":112620,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11977,"spinlock_wait_cycles":536540032,"thread_start_us":811102,"threads_started":2,"wal-append.queue_time_us":598705}
I20260709 13:08:52.143702 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:52.144292 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 105452us
I20260709 13:08:52.134951 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3246) took 1216 ms. Trace:
I20260709 13:08:52.139541 21036 leader_election.cc:400] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:08:52.144304 21795 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:52.145418 21141 rpcz_store.cc:276] 0709 13:08:50.918936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.132341 (+213405us) service_pool.cc:224] Handling call
0709 13:08:51.184557 (+ 52216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.374522 (+189965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:51.374551 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:51.374591 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:51.374623 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.134730 (+760107us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54002959d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:52.134779 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:52.134895 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436370304}
I20260709 13:08:52.151264 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.145520 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:52.154716 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.128053 27218 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.157050 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.146693 21036 leader_election.cc:403] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:52.158830 21147 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:52.111794 27795 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.137547 27533 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:52.155665 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.160990 21145 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:52.162179 27611 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.131950 27739 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.130278 27580 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:52.133901 27812 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:52.139716 27694 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:52.169528 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:52.152580 27598 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:52.121801 27776 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:08:52.165827 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 70207.3us
I20260709 13:08:52.136951 27802 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:52.140002 27771 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.133085 27726 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:52.170578 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60480.4us
I20260709 13:08:52.133005 27733 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.173804 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:52.178393 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.178884 21138 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:52.175566 27599 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:52.180102 27599 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:52.180550 27599 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:52.129815 27531 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.168416 27565 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:08:52.169126 27765 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:52.185439 27765 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.185835 27765 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:52.177536 27378 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:08:52.190294 27726 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:52.190804 27726 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:52.191774 27726 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.173882 27598 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.193439 27726 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:52.193831 27598 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:52.194584 27802 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:52.171176 27790 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:52.198022 27639 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:52.200628 27802 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:52.200858 27647 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:52.202509 27790 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:52.203135 27790 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:52.204531 27790 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.179333 27779 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.205215 27688 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.207757 27779 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.209398 27531 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.209319 27812 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.212136 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.212563 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3390) took 2178 ms. Trace:
I20260709 13:08:52.212982 21146 rpcz_store.cc:276] 0709 13:08:50.034262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.034485 (+   223us) service_pool.cc:224] Handling call
0709 13:08:51.874958 (+1840473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.875586 (+   628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.212321 (+336735us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54002e3e50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:52.212462 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4231748352}
I20260709 13:08:52.213476 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:52.162906 27767 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:52.171864 27533 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:52.215713 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:52.221411 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:52.222575 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:52.173674 27694 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.210728 27812 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:52.235898 27706 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:52.236399 27599 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.232141 27733 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.240311 27583 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.227111 27620 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:52.243849 27620 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:52.244266 27620 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:52.247752 27413 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:52.249452 27413 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:52.249953 27413 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:52.252336 27795 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:52.254267 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.255472 27733 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.256070 27733 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.257632 27770 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.259181 27733 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:52.223493 27779 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:08:52.220774 27767 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:52.261317 27779 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:52.261724 27779 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:52.262684 27779 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.164310 27218 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.265697 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.221745 27665 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:52.222667 27584 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:52.222365 27820 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:52.231699 27080 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:52.214946 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:52.283954 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69071.8us
I20260709 13:08:52.261247 27583 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.224309 27688 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:52.229269 27694 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:52.270251 27779 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:52.277009 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:52.321790 21795 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:52.285954 21798 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:08:52.273273 27665 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.324163 21070 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:08:52.334025 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.335031 21215 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:52.292340 27749 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.293215 27583 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.287688 27524 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:52.283023 27080 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.298621 27218 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.369282 21794 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:52.301960 27764 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:52.306372 27620 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.307423 27765 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.309145 27770 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.380759 21065 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:52.381481 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:52.310083 27325 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:52.386155 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.310913 27767 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.280197 27599 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.369374 27080 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:52.271456 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.289394 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.340205 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.340986 27665 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:52.351646 21215 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:52.353557 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:52.364143 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.368088 27776 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.373175 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:52.426225 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 136930us
I20260709 13:08:52.426941 21220 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:08:52.251164 27413 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.430543 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.383977 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.375801 27764 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:52.390501 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:52.442493 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52113.9us
I20260709 13:08:52.386994 21065 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:52.445279 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:52.446295 21065 raft_consensus.cc:2368] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18.
I20260709 13:08:52.392477 21072 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:52.397038 21795 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:52.411216 21800 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:08:52.317338 27667 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.413125 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.413271 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.420192 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.459124 21071 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:52.369903 27762 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:52.424384 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 248166us
W20260709 13:08:52.427748 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.429428 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.389279 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:52.432444 21217 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:52.431140 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.464704 21218 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:52.464555 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.434859 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.465617 21220 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:52.465905 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.466501 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:52.435173 21067 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
W20260709 13:08:52.468107 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.435580 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.435511 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.469321 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:52.449671 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 178356us
I20260709 13:08:52.452937 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.476949 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.478775 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:52.479688 21215 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:52.459444 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.451642 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:52.448961 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:52.485230 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.457692 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:52.489015 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 118963us
I20260709 13:08:52.491247 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.474116 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:52.493846 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.497854 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:52.425406 27533 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:08:52.498060 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 49205.1us
I20260709 13:08:52.510249 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.511051 21072 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:52.431434 27770 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:52.374992 27583 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:52.502240 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:52.508628 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.505409 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.489976 27565 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:52.462134 27761 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:52.441138 27647 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:08:52.514026 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.503636 27795 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:52.515358 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.463438 27483 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:52.424970 27676 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
W20260709 13:08:52.518558 27689 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.566s	user 0.003s	sys 0.007s
I20260709 13:08:52.520974 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.540102 21071 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:52.520758 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.540720 21795 raft_consensus.cc:2368] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18.
I20260709 13:08:52.486269 27771 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:52.520670 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.519186 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.542836 21801 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:52.497263 27524 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:08:52.529014 27761 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.523319 21142 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
W20260709 13:08:52.560262 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58121.6us
I20260709 13:08:52.440797 27702 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:08:52.512923 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.506040 27748 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:52.497357 27794 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:52.505630 27605 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:52.514456 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.518435 27384 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:08:52.509807 27560 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:52.512279 27584 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:52.567693 27794 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:52.573812 27584 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:52.574033 27794 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:52.570451 27605 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.579742 27605 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:52.583752 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:52.584573 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.585016 27669 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:52.535326 21215 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:52.529482 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:52.507546 27620 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.485731 27762 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.546766 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:52.510872 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.547712 21071 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:08:52.591580 21071 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:52.591657 27667 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.597225 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1337) took 1384 ms. Trace:
I20260709 13:08:52.597671 21800 rpcz_store.cc:276] 0709 13:08:51.212451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.212569 (+   118us) service_pool.cc:224] Handling call
0709 13:08:51.212954 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:52.448297 (+1235343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.596183 (+147886us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b54006615a0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42493dc0 00007f7f4251fa02 00007f7f4251bdd6 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:08:52.596986 (+   803us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54006612d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:52.597162 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2791297792}
I20260709 13:08:52.599216 27694 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:52.600687 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 809) took 1114 ms. Trace:
I20260709 13:08:52.600935 27765 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.601109 21794 rpcz_store.cc:276] 0709 13:08:51.486508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.486697 (+   189us) service_pool.cc:224] Handling call
0709 13:08:52.600457 (+1113760us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54006c11d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:52.600596 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026810112}
I20260709 13:08:52.601833 27532 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.602505 27532 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:52.590565 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2965) took 1512 ms. Trace:
I20260709 13:08:52.529548 27686 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:52.544492 27406 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.515372 27690 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:08:52.610576 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.524454 27757 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:08:52.611713 27690 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:52.611349 27406 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.612126 27690 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:52.612267 27669 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:52.612704 27669 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:52.613011 27690 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.604496 27677 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:52.533814 27795 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.613456 27669 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.614156 27677 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:52.518898 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:52.614674 27677 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:52.614379 27740 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:52.615231 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 68530us
I20260709 13:08:52.616178 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.617151 27483 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:52.574929 27584 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.617841 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.619469 27667 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:52.620273 27667 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:52.621858 21212 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:08:52.563386 27776 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:52.516683 27533 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:52.587141 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 57758.1us
I20260709 13:08:52.564576 27761 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:52.603852 21796 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:52.519654 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:52.565466 27565 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:52.614779 27669 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:52.569428 21217 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:52.560240 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.627303 27776 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:52.628618 27764 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:52.630007 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.648628 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:52.631819 21138 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:08:52.650031 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.632637 27533 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:52.632956 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 114144us
I20260709 13:08:52.651504 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:52.633543 27820 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:52.663122 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:52.604377 21063 rpcz_store.cc:276] 0709 13:08:51.077891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.386535 (+308644us) service_pool.cc:224] Handling call
0709 13:08:51.386842 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:51.396932 (+ 10090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:51.396974 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:51.397523 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:08:52.308581 (+911058us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b1800355de0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:52.527027 (+218446us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:52.527089 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.527192 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:52.587015 (+ 59823us) raft_consensus.cc:1685] finished
0709 13:08:52.587034 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:52.590503 (+  3469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9578,"raft.queue_time_us":11879,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":80904448,"thread_start_us":218175,"threads_started":1,"wal-append.queue_time_us":1143343}
I20260709 13:08:52.636590 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.624812 21111 leader_election.cc:400] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18.
I20260709 13:08:52.666996 21111 leader_election.cc:403] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:52.634706 21035 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:52.614284 27795 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:52.644059 21220 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:52.653273 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:52.653416 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 93316.8us
I20260709 13:08:52.654393 21215 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:52.656376 21187 leader_election.cc:400] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18.
I20260709 13:08:52.656354 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:52.681402 21187 leader_election.cc:403] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:08:52.664796 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:52.665033 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.666198 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:52.668754 21110 leader_election.cc:400] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18.
W20260709 13:08:52.669148 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44521.5us
I20260709 13:08:52.676215 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:52.676527 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:52.681099 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.605732 27598 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.655148 27825 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:52.660187 27325 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:52.699102 21063 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:08:52.699373 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 62886.4us
I20260709 13:08:52.660892 27560 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:52.624269 27080 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.665570 27770 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.740603 21219 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:08:52.668241 27583 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.692196 27533 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:08:52.737998 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:52.741226 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.714406 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.665247 27771 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.742085 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.692293 27767 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.699795 27524 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:52.747654 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.662376 27706 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:52.749157 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 114561us
I20260709 13:08:52.662883 27736 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:08:52.701035 27688 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:52.768349 21145 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:52.700140 27741 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18.
I20260709 13:08:52.745927 21066 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:52.757278 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.692783 27565 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:52.778616 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.701668 27792 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:52.765810 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:52.787410 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:52.788020 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:52.739187 27325 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.789726 27325 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:52.791532 27620 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:52.792457 27620 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:52.792914 27620 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:52.793360 27620 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:52.761535 27583 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:52.761683 27749 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:52.701388 27812 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:08:52.810421 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:52.774717 27483 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:52.809412 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:52.814224 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.754734 27770 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.820538 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:52.823215 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:52.780334 27565 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:52.763212 27771 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.826776 21071 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:52.803164 27733 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:52.817116 27767 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:52.832315 27767 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.834429 27767 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.834538 27532 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.838356 27413 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:52.843842 27580 consensus_meta.cc:287] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.666s	user 0.014s	sys 0.001s
I20260709 13:08:52.844324 27580 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.845661 21216 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:52.845701 27620 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.847558 27580 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:52.848273 27325 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.850291 27325 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.851555 27620 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:52.851792 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.852500 21065 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:52.816027 27688 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:52.826318 21142 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:08:52.818219 27694 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.754763 27776 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.812939 27667 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.857585 21065 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:08:52.869971 21065 pending_rounds.cc:85] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 12
I20260709 13:08:52.817806 27583 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:52.861044 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:52.858633 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 98742.4us
I20260709 13:08:52.870401 21065 pending_rounds.cc:107] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:08:52.829413 27565 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:52.827229 27779 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:52.872663 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.872978 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:52.873997 27667 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.879106 27378 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:52.879592 27378 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.879952 27378 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:52.880502 27667 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.871323 27688 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:52.881716 27688 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.818424 27792 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:08:52.883646 27598 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.863145 27740 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:52.831039 27733 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:52.886090 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:52.886368 27733 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:52.876449 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:52.890477 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:52.892055 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:52.892738 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:52.889096 27761 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.880102 27794 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.875939 21065 raft_consensus.cc:2889] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:52.901600 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:52.863842 27839 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:52.902537 21071 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:52.972707 27779 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.972980 27677 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.973429 27779 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:52.974080 27080 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:52.974406 21801 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
W20260709 13:08:52.977288 27665 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.521s	user 0.005s	sys 0.005s
I20260709 13:08:52.977751 27665 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:52.978466 27677 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:52.979557 21146 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:52.923859 27839 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:52.980914 27839 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:52.901172 27749 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:52.900583 27792 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:52.985049 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:53.012337 27583 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:53.015683 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 74698.8us
I20260709 13:08:53.016191 27736 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:53.016633 27736 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.017006 27736 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:53.018970 27413 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:53.019574 27413 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:08:53.020609 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.021212 21071 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:53.022116 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.022455 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:53.024101 27771 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.024516 27614 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:53.024917 27614 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.025269 27614 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:53.026032 27736 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:53.028124 27794 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.029381 27761 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.023473 27413 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:08:53.038959 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.100282 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:53.099730 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 201543us
I20260709 13:08:53.106416 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:53.114316 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.131634 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.133427 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.134402 21071 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:08:53.135986 27831 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.387s	user 0.004s	sys 0.000s
I20260709 13:08:53.139631 27665 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.140825 27580 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:53.140810 27740 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:53.141708 27580 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.141880 27740 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.144167 27580 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:53.145635 27770 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.148327 27639 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:53.149125 27639 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:53.152580 27853 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:53.155445 27605 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.000s
I20260709 13:08:53.158043 27378 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:53.196681 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.197304 21072 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:53.158404 27605 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.160710 27533 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18.
I20260709 13:08:53.103868 27779 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.161991 27531 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:53.198282 27531 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.200557 27531 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:53.161259 27740 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:53.166970 27565 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.202768 27565 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.203483 21063 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:53.172458 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.205610 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:53.206323 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 573293us
I20260709 13:08:53.173964 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.176525 27736 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:53.184182 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.210819 21067 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:53.212522 21067 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:53.212971 27406 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:53.187175 27614 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.215529 27378 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.207001 21064 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:53.217175 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:53.217770 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:53.217921 27614 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.188886 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.218532 21214 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:53.218698 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:53.193717 27795 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.219341 21213 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:53.220165 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:53.220260 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:53.220710 27605 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:53.150652 27639 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.220782 27795 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.133653 27733 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.221019 21215 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:53.240764 21067 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:53.217789 21219 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:53.207082 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:53.232601 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:53.243749 21221 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:08:53.244167 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:53.245291 21215 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:53.235195 21213 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:53.246762 21213 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:53.236402 27762 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:53.260891 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53880.1us
I20260709 13:08:53.261831 21072 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:53.262414 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.237897 27673 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:53.269001 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:53.105543 27744 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:53.270239 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.254083 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.275385 21213 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.294924 21213 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:53.298461 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44502.1us
I20260709 13:08:53.299352 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:53.297667 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 483515us
I20260709 13:08:53.301692 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:53.301776 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:53.300987 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:53.302486 21801 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:53.296936 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:53.301878 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:53.313712 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.272141 27795 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:53.318578 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:53.323333 21215 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:53.324651 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.326162 21214 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:08:53.272851 27533 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:53.328078 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:53.333184 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:53.334110 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:53.334492 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.335676 21219 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:53.336798 21212 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:08:53.335328 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:53.311313 27665 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:53.274078 27762 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:08:53.332101 27080 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:53.340970 27560 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:53.344672 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.345752 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:53.321609 27689 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:53.295794 27406 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:08:53.320614 27565 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:53.347126 27560 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:53.357571 27560 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:53.347553 21072 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:53.349225 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:53.347328 21221 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:53.362800 27771 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:53.364171 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.338294 27598 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.367468 27667 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:53.367698 27598 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.332438 27533 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.369777 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.370301 27080 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:53.371052 27080 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.343528 27677 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:53.372206 27677 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:53.285832 27744 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.372671 27677 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:53.352478 27689 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.371076 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.374056 27689 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:53.356029 27761 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:53.297950 27669 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:08:53.341939 27770 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:53.376269 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.379827 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.380146 21140 raft_consensus.cc:2368] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:53.353266 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.380934 21068 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:53.343925 27848 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:53.378916 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:53.311197 27832 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:08:53.342625 27795 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:53.396124 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.381645 21146 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:53.382761 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.344693 27688 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:53.369107 27667 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.378355 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.388307 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.391665 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:53.411579 21221 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:08:53.390763 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:53.392652 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:53.412356 21145 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:53.392916 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:53.413384 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.327709 27779 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:08:53.399864 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.402235 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.420900 21146 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:08:53.421730 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:53.422117 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:53.422631 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.422920 21218 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:53.402748 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.423900 21144 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:53.425040 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.426007 21144 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:53.429416 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.427044 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.390930 21213 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:08:53.407974 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.437425 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:53.438114 21216 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:53.438349 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.438943 21145 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:53.439419 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.440694 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:53.394568 27614 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:53.414738 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.377739 27533 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:53.394201 27771 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:53.411409 21140 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:53.483932 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.418246 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:53.484874 21219 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:08:53.417625 27604 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:53.418707 27847 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:53.442030 21213 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:08:53.489444 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:53.416680 21142 raft_consensus.cc:2368] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18.
I20260709 13:08:53.409791 21072 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:08:53.411898 27848 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:53.477461 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.495106 21141 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:53.480108 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:53.495855 21799 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:53.480386 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.496402 21144 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:53.480803 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:53.481621 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:53.499095 21068 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:53.482815 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:53.385558 27761 raft_consensus.cc:2764] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:08:53.440161 27781 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:53.444242 27855 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:53.439957 27378 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:53.492123 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:53.441632 27790 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:53.446014 27497 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.502795 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:53.511232 21145 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:53.641989 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:53.545578 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:53.549619 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:53.550786 27856 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:08:53.477065 27583 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:08:53.477972 27688 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:08:53.557042 21799 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:53.558434 27761 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:53.430996 27667 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.558737 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:53.563230 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.565809 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.650732 21141 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:53.651611 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:53.567989 21800 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
W20260709 13:08:53.574004 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 91260.1us
I20260709 13:08:53.574728 21216 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:08:53.650432 21072 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 21: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:53.635715 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 2988) took 1174 ms. Trace:
I20260709 13:08:53.636461 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.653312 21069 rpcz_store.cc:276] 0709 13:08:52.461386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.605494 (+144108us) service_pool.cc:224] Handling call
0709 13:08:52.605810 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:52.606198 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:52.606228 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:52.612093 (+  5865us) log.cc:844] Serialized 26 byte log entry
0709 13:08:53.146105 (+534012us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b1800464c20. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:53.376269 (+230164us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:53.376329 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.376422 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:53.565019 (+188597us) raft_consensus.cc:1685] finished
0709 13:08:53.565039 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:53.635616 (+ 70577us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3500,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"spinlock_wait_cycles":16896768,"thread_start_us":229933,"threads_started":1,"wal-append.queue_time_us":832194}
I20260709 13:08:53.654340 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:53.655402 21068 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:53.478812 27776 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:08:53.485677 27839 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.170s
I20260709 13:08:53.648679 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:53.644451 21142 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:53.565605 27771 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.564343 27848 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:53.493827 27604 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.659713 27384 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:53.659751 27855 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:53.650385 27497 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.667596 27639 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:53.648186 27378 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:53.648303 27790 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:53.672251 21213 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:53.501581 27847 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.564785 27677 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.673779 21144 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:08:53.675112 21146 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:53.680758 21067 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:53.681025 27774 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:08:53.682248 27821 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:08:53.684528 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 1161) took 1331 ms. Trace:
I20260709 13:08:53.686830 27080 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.766842 21138 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:53.637363 27795 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.655798 27665 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:53.692688 27667 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:53.645107 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 153055us
I20260709 13:08:53.701433 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 2099) took 1046 ms. Trace:
I20260709 13:08:53.694967 27839 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:53.705103 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:53.702432 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:53.729254 21069 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:53.704602 27604 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:53.712512 27790 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:53.732919 21141 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:53.565464 27218 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:53.787423 27218 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.701010 27771 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.790262 27665 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:53.791842 27665 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.792076 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:53.793373 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.794807 27080 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:53.795800 27080 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:53.796710 27080 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.797240 27598 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.687687 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 852) took 1169 ms. Trace:
I20260709 13:08:53.799435 21220 rpcz_store.cc:276] 0709 13:08:52.517950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.518932 (+   982us) service_pool.cc:224] Handling call
0709 13:08:53.687462 (+1168530us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540034c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:53.687620 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232677120}
I20260709 13:08:53.800240 27744 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.802354 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:53.802704 27218 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.786681 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.712077 27378 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.705034 27855 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.706640 27497 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.770248 27599 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:53.770965 27639 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:53.771415 27677 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.771979 27774 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:53.772097 27690 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:53.703472 27384 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:53.773079 21144 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:53.766624 27620 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:53.775030 21066 rpcz_store.cc:276] 0709 13:08:52.353009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.353256 (+   247us) service_pool.cc:224] Handling call
0709 13:08:53.684345 (+1331089us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54002ad0d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:53.684461 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634131328}
I20260709 13:08:53.840669 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.842226 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.780388 21063 rpcz_store.cc:276] 0709 13:08:52.654649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.740984 (+ 86335us) service_pool.cc:224] Handling call
0709 13:08:53.701266 (+960282us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54003a2ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:53.701364 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029596032}
I20260709 13:08:53.718181 21142 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:53.844643 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:53.845301 21066 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:53.779299 27795 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.804322 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:53.768913 21146 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:53.846730 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:53.849474 21035 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:53.804561 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 99551.4us
I20260709 13:08:53.853066 21145 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:53.715549 27847 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:53.705760 27733 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.855942 21185 leader_election.cc:400] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:53.857563 21185 leader_election.cc:403] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:53.858136 27771 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:53.858711 27855 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.858857 27771 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:53.860208 27771 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.866430 27689 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.868453 27497 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:53.810901 21069 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:08:53.888728 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.889009 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.733834 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.891273 21067 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:08:53.893534 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.894448 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:53.697758 27761 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.814594 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1167) took 1252 ms. Trace:
I20260709 13:08:53.784471 27839 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:53.806452 27794 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:53.807595 27765 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:53.832974 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:53.903196 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.833338 21070 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:53.914731 21069 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:53.869323 27795 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:53.869822 27620 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.871269 27598 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:53.920845 21797 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 15: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:53.873623 27764 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:53.781145 27560 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.874922 27384 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:53.879734 27740 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:53.735123 27832 raft_consensus.cc:2764] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:08:53.895682 21065 rpcz_store.cc:276] 0709 13:08:52.562372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.853204 (+290832us) service_pool.cc:224] Handling call
0709 13:08:52.853546 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:52.897903 (+ 44357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.814278 (+916375us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b540001b3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:53.814515 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8470784}
W20260709 13:08:53.974207 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.000s	sys 0.010s
W20260709 13:08:53.974757 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.000s	sys 0.010s
I20260709 13:08:53.916077 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.976068 21070 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:53.977339 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.918855 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.918411 27261 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:53.920837 27598 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.928290 27825 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:53.928607 27620 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:53.922289 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 89408us
I20260709 13:08:53.926815 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.930048 27761 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:53.896781 27497 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.932621 21064 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:54.013294 21064 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:53.834484 27774 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:53.890800 27533 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:53.898203 27599 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:08:53.953966 27841 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
W20260709 13:08:53.954216 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.033s	sys 0.045s
I20260709 13:08:53.955153 21797 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
W20260709 13:08:54.022101 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.033s	sys 0.045s
W20260709 13:08:54.022563 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 602458us
I20260709 13:08:53.966449 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:54.024871 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58564.5us
I20260709 13:08:53.973941 27384 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:53.908268 21215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.010s	sys 0.028s
W20260709 13:08:54.029450 21215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.010s	sys 0.028s
W20260709 13:08:53.976091 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 532339us
I20260709 13:08:53.980032 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:53.978174 21070 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:54.031652 21070 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:54.009143 27598 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:53.998109 21139 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.000s	sys 0.014s
I20260709 13:08:54.011884 27821 raft_consensus.cc:2764] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:08:53.956058 27733 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:53.954487 27847 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:54.010893 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:53.964195 27740 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:54.070060 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 59285.5us
I20260709 13:08:54.017403 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:53.950860 21767 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:54.020942 21147 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.010s
W20260709 13:08:53.997216 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78475.3us
I20260709 13:08:54.062215 21140 raft_consensus.cc:2393] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:54.072854 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:54.074038 21147 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.008s	sys 0.010s
W20260709 13:08:54.071261 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 120484us
I20260709 13:08:54.039903 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:54.023835 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:54.078773 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55047.2us
I20260709 13:08:54.078872 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:53.963465 27611 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:08:53.997287 21071 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:54.016604 27497 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:54.080027 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:54.082021 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:54.078058 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:54.082228 21220 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:08:54.090384 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:54.047977 27620 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:54.083009 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:54.088680 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.013s	sys 0.028s
I20260709 13:08:54.083015 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:54.085602 21065 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:54.092094 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:54.095075 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:54.095793 21070 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 13:08:54.099313 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.013s	sys 0.029s
I20260709 13:08:54.105576 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:54.106410 21219 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:54.107141 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:54.107805 21213 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:54.109580 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:54.111397 21185 leader_election.cc:400] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 17. Current term is 18.
I20260709 13:08:54.112528 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:54.113914 21185 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:54.098644 21212 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:54.116605 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:54.061128 27733 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:54.119688 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:54.076174 27744 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:54.080201 27749 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:54.135926 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:54.049155 27761 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:08:54.090332 27725 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:54.122983 21145 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:54.091357 27686 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:54.055496 27565 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:08:54.086716 27779 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:54.078603 27847 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:54.153576 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:54.105768 27855 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:54.105130 27647 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:54.147423 27848 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:54.083693 27802 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:54.104121 27620 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.105278 27531 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:54.120092 27821 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:54.124487 27688 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:54.091310 27853 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:08:54.122407 27604 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:54.161753 27725 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.160048 27749 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:54.158350 27605 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:08:54.259310 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:54.200177 27779 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:54.272591 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:54.173318 27870 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:54.220203 27647 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.119159 27774 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.303788 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:54.281095 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 149234us
I20260709 13:08:54.226668 27620 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:54.244221 27847 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:54.227777 27689 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:54.279935 21797 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:54.264879 27639 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:54.269023 27725 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:54.308833 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:54.288034 27779 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:54.271231 27748 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:54.272864 27604 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:54.320526 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:54.302930 27647 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:54.294724 27870 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.274214 27611 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:08:54.303730 27560 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:54.167754 27740 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.358856 21138 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:54.288640 27677 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:54.307047 27218 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:54.348627 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:54.320705 27774 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:54.326640 27605 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:54.348625 27870 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:54.364645 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:54.360088 27862 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
I20260709 13:08:54.329892 27639 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:08:54.378959 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.367113 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 263433us
W20260709 13:08:54.344117 27861 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 13:08:54.341557 27667 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:54.320806 27794 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:08:54.357402 27771 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:54.368854 27864 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:08:54.357710 27790 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:54.394502 27863 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:08:54.362739 27677 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.375890 27865 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:54.340919 27748 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.376194 27533 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:54.399935 27761 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:54.423585 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:54.441618 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 863) took 1113 ms. Trace:
I20260709 13:08:54.443190 21217 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:54.399806 27605 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:54.446067 21142 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:54.447966 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:54.447160 21139 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:08:54.406504 27740 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:54.444208 21215 rpcz_store.cc:276] 0709 13:08:53.328320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.328465 (+   145us) service_pool.cc:224] Handling call
0709 13:08:53.328877 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:53.329167 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:53.329206 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:53.329902 (+   696us) log.cc:844] Serialized 26 byte log entry
0709 13:08:54.029991 (+700089us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:54.030052 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:54.030147 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:54.441285 (+411138us) raft_consensus.cc:1685] finished
0709 13:08:54.441308 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:54.441529 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":689802,"threads_started":1,"wal-append.queue_time_us":682201}
I20260709 13:08:54.453706 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:54.455056 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:54.455850 21146 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:54.456278 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:54.403069 27565 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:08:54.422605 27865 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:54.421710 27677 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:54.461812 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:54.450876 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 147168us
I20260709 13:08:54.409128 27688 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:08:54.466244 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:54.467222 21765 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:54.477067 27891 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:54.465656 27605 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.486514 27748 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:54.489459 27873 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
W20260709 13:08:54.491344 27878 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 13:08:54.494297 27872 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:08:54.320003 27757 consensus_queue.cc:1175] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9384358ce43f4e3fac2ae44bdf11c389" responder_term: 19 status { last_received { term: 18 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:08:54.496989 27875 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:08:54.501343 27876 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:08:54.503093 27881 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 13:08:54.517968 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.523831 27874 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:08:54.422073 27614 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 13:08:54.526743 27877 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:08:54.392359 27749 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.534703 27639 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:54.542084 27884 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:08:54.543869 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 925) took 1063 ms. Trace:
I20260709 13:08:54.414526 27771 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.544409 21140 rpcz_store.cc:276] 0709 13:08:53.480242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.483713 (+  3471us) service_pool.cc:224] Handling call
0709 13:08:54.543618 (+1059905us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54002963d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:54.543781 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1327867264}
I20260709 13:08:54.546424 27771 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:54.547746 21142 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 13:08:54.543988 27871 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:08:54.408717 27667 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:54.552246 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 90780.6us
I20260709 13:08:54.565237 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:54.413673 27604 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.567960 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:54.568168 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.568825 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:54.572978 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:54.578945 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 120928us
I20260709 13:08:54.468065 27865 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:54.583726 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:54.584995 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:54.587224 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:54.588861 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:54.589609 21215 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:08:54.564069 27770 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:54.465571 27689 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.111s
I20260709 13:08:54.570407 27583 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:54.596125 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 460305us
I20260709 13:08:54.576399 27611 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:54.612362 27802 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.613139 27802 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.617233 27802 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:54.620374 27882 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:54.567775 27565 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:08:54.590411 27560 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:08:54.639906 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 52748us
I20260709 13:08:54.566294 27883 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:08:54.596004 27770 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:08:54.641983 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51088.7us
I20260709 13:08:54.595471 27688 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:08:54.602041 27761 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:54.680657 27902 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:54.608904 27583 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:54.699311 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 58362.9us
I20260709 13:08:54.700129 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:54.651427 27894 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:54.713012 27384 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.658s	user 0.008s	sys 0.004s
I20260709 13:08:54.759945 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:54.774932 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.722177 27887 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 13:08:54.720278 27690 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.865s	user 0.013s	sys 0.000s
W20260709 13:08:54.737651 27764 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.783s	user 0.013s	sys 0.000s
I20260709 13:08:54.792637 21799 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEARNER]: Advancing to term 20
I20260709 13:08:54.781327 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:54.781594 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:54.735491 27667 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:54.791615 27689 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:54.830771 21796 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:08:54.832083 27583 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:54.863355 21798 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:08:54.832017 27882 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:54.838075 27847 consensus_meta.cc:287] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.001s
I20260709 13:08:54.838883 21799 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEARNER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:08:54.840641 27725 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
W20260709 13:08:54.841936 27598 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.002s
I20260709 13:08:54.700126 27770 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.772141 27894 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:08:54.772686 27688 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:54.715690 27620 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.829363 27761 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:54.856324 27497 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.750s	user 0.007s	sys 0.005s
I20260709 13:08:54.861145 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:54.862504 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:54.727769 27889 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.857818 21212 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:54.873319 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:54.734688 27865 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.878091 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:54.878242 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:54.875687 21801 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:54.815697 27647 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.887019 27667 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:54.887914 27639 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.888587 27910 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:54.874413 27779 consensus_meta.cc:287] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.004s
I20260709 13:08:54.865893 27843 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.865491 27792 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.858639 27560 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:54.924777 21801 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:54.884286 27669 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:54.875946 27847 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:54.925516 27870 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.003s
I20260709 13:08:54.894804 27688 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.875492 27725 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.875842 27882 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.892797 27598 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:55.032377 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:54.890374 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:54.931586 21214 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:08:55.044458 21142 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21.
I20260709 13:08:55.045866 21221 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:54.927322 21219 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:55.158293 27598 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:54.944681 21801 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.160238 21801 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:54.923879 27497 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.163378 27667 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.164263 27667 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.950433 27560 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.956686 27620 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.174856 21141 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:54.937595 27639 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.972056 27688 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:54.945636 27779 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:54.950775 27792 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.965399 27821 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:55.191387 27869 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:55.191895 27869 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:55.193559 27869 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:54.950584 27843 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.195403 27378 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.989215 27847 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:54.960717 27865 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.196344 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:54.972205 27870 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:55.197213 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.156587 27866 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.199728 27870 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:54.990206 27882 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.199947 27866 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.201581 21068 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:08:54.966837 27885 log.cc:927] Time spent T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.482s	user 0.000s	sys 0.001s
I20260709 13:08:54.925498 27889 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.202875 27866 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:54.988723 27901 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:55.232957 27901 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:55.234310 27790 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:54.945905 27910 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.235204 27790 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.067711 27080 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.068018 27647 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.237928 27910 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.238514 27080 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.234628 27901 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.240531 27901 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:55.078567 27733 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.242972 27733 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:55.049031 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:55.067081 27725 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.110704 27895 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:55.113397 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2185) took 1027 ms. Trace:
I20260709 13:08:55.280050 27895 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:55.280759 27895 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:55.282979 27866 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.283995 27866 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.120430 21146 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:55.075268 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:55.288264 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:55.288926 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.289611 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:55.058343 27839 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.292470 21066 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:55.293814 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:55.291018 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:55.294939 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1360) took 1342 ms. Trace:
I20260709 13:08:55.168835 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:54.955900 27677 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.176921 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:55.185941 27779 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.188599 21213 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:55.205336 27639 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.205494 27889 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:54.955137 27900 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:08:55.369446 21071 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:55.370702 27888 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.206203 21145 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:55.191932 27821 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:55.196309 27378 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.209285 27667 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:55.209857 27690 consensus_queue.cc:1175] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" responder_term: 15 status { last_received { term: 15 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 15 index: 12 } }
I20260709 13:08:55.186079 27560 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.211596 27497 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.225142 21214 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:08:55.227386 27896 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.642s	user 0.004s	sys 0.000s
I20260709 13:08:55.229297 21212 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:55.387964 21146 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:08:55.231559 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:55.231578 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:55.241848 27080 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.392333 21219 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:08:55.392704 21071 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:55.394678 21144 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:55.245373 27869 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:55.080875 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1308) took 1002 ms. Trace:
I20260709 13:08:55.261128 27770 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:55.279675 21216 rpcz_store.cc:276] 0709 13:08:54.085520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.085668 (+   148us) service_pool.cc:224] Handling call
0709 13:08:54.085945 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:54.280516 (+194571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:54.280566 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:54.280629 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:54.280687 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.112788 (+832101us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400102554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:55.112890 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.113181 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446940416}
I20260709 13:08:55.145067 21143 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:55.098081 27774 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.394511 27821 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.403601 27774 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.404280 27821 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:55.298024 27614 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.406483 27614 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:55.407387 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 65245.2us
I20260709 13:08:55.409662 27614 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.237960 27790 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.300191 27647 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.410770 27647 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.413030 27647 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.300721 27853 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:55.413954 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.413823 27853 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.310302 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:55.315106 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.316659 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:55.317119 27792 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.319669 27843 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.319995 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:55.327261 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.336231 21141 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:55.334066 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1185) took 1081 ms. Trace:
I20260709 13:08:55.364353 21801 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:55.350718 21798 rpcz_store.cc:276] 0709 13:08:53.952209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.979693 (+ 27484us) service_pool.cc:224] Handling call
0709 13:08:55.294586 (+1314893us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b5400701a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:55.294738 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1794599296}
I20260709 13:08:55.337780 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:55.323163 27886 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.835s	user 0.003s	sys 0.000s
I20260709 13:08:55.372480 27888 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.282488 27895 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.338656 27384 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:55.430052 27384 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:55.430610 27895 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:55.431340 27384 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.433259 27384 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:55.380652 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:55.435930 21070 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:55.436275 21145 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:55.380120 21797 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:55.437642 21067 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:55.441689 27639 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.442489 27639 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.445075 27639 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.449687 21143 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:55.396202 21795 rpcz_store.cc:276] 0709 13:08:54.078407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.078581 (+   174us) service_pool.cc:224] Handling call
0709 13:08:54.081545 (+  2964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.081900 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:54.081934 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:54.082397 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:08:54.779707 (+697310us) spinlock_profiling.cc:243] Waited 7.63 ms on lock 0x7b1800428ce0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:54.932791 (+153084us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:54.932833 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:54.932898 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:55.080234 (+147336us) raft_consensus.cc:1685] finished
0709 13:08:55.080279 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.080661 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":12427,"spinlock_wait_cycles":17546112,"thread_start_us":152881,"threads_started":1,"wal-append.queue_time_us":901904}
I20260709 13:08:55.451475 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 867) took 1006 ms. Trace:
I20260709 13:08:55.323642 27866 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.387108 27889 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.399500 27643 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.335155 27689 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:55.425608 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:55.429416 27888 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:55.455653 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.418497 21068 rpcz_store.cc:276] 0709 13:08:54.252635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.253041 (+   406us) service_pool.cc:224] Handling call
0709 13:08:54.257034 (+  3993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.202135 (+945101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.333647 (+131512us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b5400215554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:55.333948 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170998144}
I20260709 13:08:55.463356 27733 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:55.463915 27643 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.464233 27733 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.466289 27733 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.467737 27889 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.468816 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.470129 27889 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.451915 21213 rpcz_store.cc:276] 0709 13:08:54.445291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.445591 (+   300us) service_pool.cc:224] Handling call
0709 13:08:54.445879 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.374840 (+928961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.451179 (+ 76339us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b5400104354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:55.451420 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1467768064}
I20260709 13:08:55.453680 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:55.449690 27790 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:08:55.483893 27843 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:55.457353 27890 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.496874 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:55.497666 21068 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:55.498965 27853 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.500555 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:55.501315 21072 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:55.503048 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:55.504734 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.505553 21064 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:55.512247 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 929) took 1353 ms. Trace:
I20260709 13:08:55.512429 27599 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:55.513064 27599 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:55.512789 21138 rpcz_store.cc:276] 0709 13:08:54.158417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.158605 (+   188us) service_pool.cc:224] Handling call
0709 13:08:54.158934 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:54.359488 (+200554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.511704 (+1152216us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54002585d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:55.512020 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459157888}
W20260709 13:08:55.503058 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.500590 21067 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:55.466301 27643 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:55.528010 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:55.527102 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 92960.6us
W20260709 13:08:55.489231 27790 raft_consensus.cc:1068] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:55.489509 27889 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.583328 21066 raft_consensus.cc:2410] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:08:55.585515 27677 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.586483 27890 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.588425 21065 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:55.589890 27890 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.590737 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:55.514556 27599 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.600075 27599 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:55.603520 21800 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:55.610538 27889 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.613138 27889 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:55.618181 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 201111us
I20260709 13:08:55.620277 27839 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:55.620867 27790 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:55.621306 27790 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.621333 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:55.621851 27790 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:55.622680 27406 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:08:55.616655 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 871) took 1029 ms. Trace:
W20260709 13:08:55.643108 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 112364us
I20260709 13:08:55.654624 21067 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:55.650555 21064 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:08:55.642295 27888 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:55.657913 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:55.625603 21218 rpcz_store.cc:276] 0709 13:08:54.460461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.460610 (+   149us) service_pool.cc:224] Handling call
0709 13:08:54.460945 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.466352 (+  5407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:54.466384 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:54.466860 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:08:55.114563 (+647703us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b180040ad40. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:55.249882 (+135319us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:55.363329 (+113447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.376945 (+ 13616us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:55.478611 (+101666us) raft_consensus.cc:1685] finished
0709 13:08:55.478629 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.489623 (+ 10994us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8026,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":147,"spinlock_wait_cycles":74622208,"thread_start_us":51727,"threads_started":1,"wal-append.queue_time_us":775111}
I20260709 13:08:55.664008 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:55.640817 27643 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:08:55.667794 21185 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21.
I20260709 13:08:55.670285 21185 leader_election.cc:403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:55.671049 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.676859 21069 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:55.661134 27406 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.682718 27771 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:08:55.684005 27771 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:55.684584 27771 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:55.685957 27771 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.688553 27406 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.682356 27688 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.682898 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1363) took 1082 ms. Trace:
I20260709 13:08:55.689775 21796 rpcz_store.cc:276] 0709 13:08:54.567230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.567700 (+   470us) service_pool.cc:224] Handling call
0709 13:08:55.575530 (+1007830us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54005d5f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:55.649206 (+ 73676us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602091264}
W20260709 13:08:55.703852 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:55.709728 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.653813 27889 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:08:55.761197 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.718449 21221 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:55.771647 21068 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:55.772468 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:55.773309 21144 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:55.776805 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:55.767798 21067 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:55.778868 27853 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:55.779368 27853 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:55.782639 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.783982 21071 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:55.771327 21072 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:55.768808 21798 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEARNER]: Advancing to term 21
W20260709 13:08:55.792367 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 80332us
I20260709 13:08:55.793326 21800 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:55.790517 27843 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.794343 27843 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.797120 27843 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:55.798755 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 182808us
I20260709 13:08:55.799809 27888 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:55.801011 27677 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:55.802099 27677 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:55.802523 27677 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:55.804543 27677 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.805670 27774 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.805873 27890 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.768354 27725 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.792248 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:55.805709 21063 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:55.814563 27901 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.807727 27725 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.815150 27901 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.807866 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:55.816848 21793 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:55.817469 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.818329 21065 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 15: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:08:55.808105 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.819064 27790 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.819957 27378 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.821285 27790 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:55.821648 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.810946 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:55.808831 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:55.822625 21801 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:55.823016 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.824195 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.824823 21065 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:55.826385 27902 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.827013 27902 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.828738 21140 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: [17.11-18.12]
I20260709 13:08:55.829357 27902 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:55.829774 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.821698 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:55.809031 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:55.831310 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.832604 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:55.832805 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 211564us
I20260709 13:08:55.833312 21221 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:55.834764 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:55.835050 27744 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:55.835659 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:55.834760 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:55.836212 27839 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:55.836539 21218 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:55.836700 21216 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:55.829965 27598 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:55.808133 27770 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.822654 21071 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:08:55.837778 27931 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:08:55.829815 27261 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.837582 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:55.839581 27261 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:55.840206 21213 raft_consensus.cc:2410] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:08:55.835747 27744 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.831928 21800 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:55.859707 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.861028 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.861122 27901 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.862478 27895 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.806322 27677 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:55.863973 27901 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:55.842410 27744 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.836690 27839 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:55.843655 27733 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.867024 21767 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:55.846611 27848 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:08:55.867601 27733 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.868737 27869 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:55.869125 27869 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.869479 27869 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:55.870635 27733 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.847366 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 868) took 1401 ms. Trace:
I20260709 13:08:55.847743 27847 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.780545 27853 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.871701 27839 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:55.871709 27847 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.871447 21220 rpcz_store.cc:276] 0709 13:08:54.445707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.445860 (+   153us) service_pool.cc:224] Handling call
0709 13:08:54.446174 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.326084 (+879910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.326136 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:55.326684 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:08:55.396227 (+ 69543us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:55.396269 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.499162 (+102893us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b540013b854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:55.499212 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:55.837017 (+337805us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:08:55.846913 (+  9896us) raft_consensus.cc:1685] finished
0709 13:08:55.846937 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.847126 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14406,"raft.queue_time_us":15960,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":164,"spinlock_wait_cycles":2022864640,"thread_start_us":69092,"threads_started":1,"wal-append.queue_time_us":298151}
I20260709 13:08:55.874550 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.849275 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.872160 27757 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:08:55.849956 27771 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:55.876948 27847 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:55.877723 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 77470.6us
I20260709 13:08:55.811708 27406 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.336930 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:55.879432 27898 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:55.881001 27898 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:55.811383 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:55.883926 27690 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:08:55.885147 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 73939.6us
I20260709 13:08:55.813153 27688 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.856001 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.857009 27725 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.857750 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.806753 27890 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.886821 21071 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:55.887382 27853 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:55.888447 27890 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.863134 27895 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.821708 27378 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.889401 27690 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:55.889714 27378 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.890583 27895 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.838701 27770 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.891834 27378 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.892915 27770 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.837518 27598 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.893100 27406 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:55.894042 21221 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:08:55.894390 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:55.895228 27598 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.841984 27261 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.896004 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:55.896742 21147 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:55.899180 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:55.899911 27406 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:55.900391 27406 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.900810 27406 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:55.902300 21138 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:55.905238 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:55.905501 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:55.906035 21800 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:55.907667 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:55.908610 27690 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:55.909345 21144 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:55.910180 27901 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:55.915167 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:55.915727 21069 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:55.916715 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:55.918109 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:55.918510 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:55.911616 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:55.923403 27639 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:55.911217 21065 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:08:55.926811 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:55.928973 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:55.931154 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.939380 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:55.893258 27839 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.942636 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.943531 21138 raft_consensus.cc:2410] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:08:55.950184 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:55.953794 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.954500 21063 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:08:55.956813 27639 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:55.957182 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:55.958045 27847 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:55.960059 27611 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:55.960749 27611 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.961498 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:55.938710 27770 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:55.963263 27847 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:55.964129 27848 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21.
I20260709 13:08:55.963938 27847 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.930470 27757 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:55.971241 21146 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:55.974093 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:55.975049 27599 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:55.975627 27599 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:55.975679 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:55.976255 27599 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:55.976858 27639 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:55.980830 27869 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:55.983981 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.984784 21066 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:55.986251 21066 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:08:55.986588 21066 pending_rounds.cc:85] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 11
I20260709 13:08:55.986828 21066 pending_rounds.cc:107] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:08:55.963130 27611 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:55.987565 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.988261 21063 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:08:55.989312 27847 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:55.989522 21063 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:08:55.990327 27643 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:55.990559 27889 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:55.952690 27922 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:55.992462 27922 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:55.994058 27690 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:55.983657 27902 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.994944 27922 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:55.987108 21066 raft_consensus.cc:2889] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:55.990646 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:55.999096 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:55.987470 27890 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:56.001706 21800 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:08:55.977263 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:55.965827 27895 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:56.007719 27890 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:56.008263 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:56.009434 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:56.010102 27892 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:56.009922 27406 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.011439 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:56.012346 27384 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:56.014102 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:56.015753 21221 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:55.960069 27839 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:56.017217 21066 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:56.019901 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.020848 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:56.014499 27643 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:56.025416 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:56.028710 21213 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:56.014178 27378 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:08:56.029681 27843 raft_consensus.cc:2764] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:08:56.028741 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:56.027935 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50739.4us
I20260709 13:08:56.031361 21066 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:08:56.035862 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:56.037495 21072 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:56.037757 21796 raft_consensus.cc:2368] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 20. Current term is 21.
I20260709 13:08:56.038745 27378 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:56.040954 21798 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:56.041188 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.042050 21146 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:56.025329 27889 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:56.037402 27643 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:56.048240 21213 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:56.050421 21035 leader_election.cc:400] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 20. Current term is 21.
I20260709 13:08:56.050798 21035 leader_election.cc:403] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:08:56.051759 27839 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.055037 27889 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.055521 27888 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:56.055871 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:56.056074 27888 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:56.056614 21221 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:56.057039 27888 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.039206 27378 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:56.059958 27378 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.006210 27902 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.060976 27882 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:56.061324 27378 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.061421 27882 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.040304 27847 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.061846 27882 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.062201 27847 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.055532 27889 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.030987 27892 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.065138 27643 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:56.066115 27643 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.078675 27689 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:56.079295 27689 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.039986 27848 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:56.092890 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.096745 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:56.104176 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 96003.3us
I20260709 13:08:56.111465 27744 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.114239 21221 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:56.092317 27599 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.120007 27869 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:56.121143 27599 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:56.121383 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:56.121968 27770 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.122465 27744 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.123243 27902 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:56.124320 21216 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:56.124650 27744 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:56.125372 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.125998 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.126509 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.127776 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.127907 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.123273 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:56.129216 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:56.111526 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.129942 21072 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:56.115054 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:56.130745 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:56.117884 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.129045 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:56.079159 20963 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 168) took 22255 ms (client timeout 15000 ms). Trace:
I20260709 13:08:56.146767 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:56.147037 21767 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:56.147902 21069 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:56.148696 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:56.117695 27690 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:56.152105 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:56.156961 21066 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:56.154186 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.157655 21215 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:56.151221 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.158885 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.163223 27384 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:56.144168 27733 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:56.112536 27848 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.122476 27770 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:56.170114 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55132.9us
I20260709 13:08:56.174235 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:56.163381 27869 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:56.158537 27689 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:56.162756 27406 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:56.193094 21220 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.177824 27839 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 20. Current term is 21.
I20260709 13:08:56.210951 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.215859 21214 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:56.184787 27869 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:56.230584 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.217814 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:56.235435 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.235548 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:56.238092 21214 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:56.204186 27847 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:56.250475 21220 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:56.256722 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.257385 21218 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:56.254338 27639 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.261349 27639 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.236361 27869 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:56.275319 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.278126 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.211792 27922 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:08:56.278870 21216 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 13.
I20260709 13:08:56.232525 21221 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:56.255079 27599 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:56.191906 27890 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.307989 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.318864 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.319552 21220 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:56.320957 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.321826 21219 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:56.333560 27892 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:56.350616 27848 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:56.352218 27770 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:56.354079 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:56.355664 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.302101 27688 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:56.356806 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:56.356488 27688 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.357359 21218 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:08:56.288687 27839 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:08:56.363252 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:56.365906 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.306206 27599 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:56.371379 27599 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.375180 21064 raft_consensus.cc:2410] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:08:56.375173 27869 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.377799 27689 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.377899 27869 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:56.378477 27689 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.380686 27689 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:56.383606 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:56.385915 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:08:56.386889 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 279842us
I20260709 13:08:56.387821 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:56.386972 27725 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:56.388697 27725 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.389053 27725 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:56.394838 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.397745 21065 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:08:56.398967 21072 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:08:56.367007 21214 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:56.369098 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:56.405402 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.406473 21220 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:56.326709 27889 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.417689 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.424229 27665 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
W20260709 13:08:56.424753 27665 raft_consensus.cc:1068] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:56.427182 27406 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:56.427772 27406 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.428892 27889 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:56.429857 27406 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:56.430718 27531 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:56.431180 27598 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:56.431677 27598 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.431368 27531 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.432086 27598 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:56.433818 27531 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:56.431385 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 62383.1us
I20260709 13:08:56.435304 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:56.436043 27744 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:56.436476 27744 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.416241 21767 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:56.436833 27744 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.445119 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.446187 21220 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
I20260709 13:08:56.437552 27689 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:56.414691 27639 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:56.487826 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:56.447723 27261 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:08:56.471794 27689 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:56.522657 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:08:56.529747 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 436731us
I20260709 13:08:56.506203 27901 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:56.534870 21113 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:56.540720 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:56.535849 27643 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:56.543040 27643 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:56.543496 27643 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:56.553395 27948 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:56.554118 27948 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:56.555076 27948 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:56.140058 20963 rpcz_store.cc:269] 0709 13:08:33.824488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.825121 (+   633us) service_pool.cc:224] Handling call
0709 13:08:56.078869 (+22253748us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:48.898269 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:48.898907 (+   638us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:48.898986 (+    79us) write_op.cc:435] Acquired schema lock
0709 13:08:48.899008 (+    22us) tablet.cc:662] Decoding operations
0709 13:08:49.495822 (+596814us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:49.516812 (+ 20990us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:49.522372 (+  5560us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:08:49.580447 (+ 58075us) tablet.cc:701] Row locks acquired
0709 13:08:49.580487 (+    40us) write_op.cc:260] PREPARE: finished
0709 13:08:49.580667 (+   180us) write_op.cc:270] Start()
0709 13:08:49.580830 (+   163us) write_op.cc:276] Timestamp: P: 1783602529580595 usec, L: 0
0709 13:08:49.580868 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:08:49.581645 (+   777us) log.cc:844] Serialized 6394 byte log entry
0709 13:08:51.669253 (+2087608us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f3f4120ac 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b
0709 13:08:52.150406 (+481153us) op_driver.cc:464] REPLICATION: finished
0709 13:08:52.599814 (+449408us) write_op.cc:301] APPLY: starting
0709 13:08:52.599896 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:53.769159 (+1169263us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b1800311020. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f4225b0ec 00007f7f422a7c0a 00007f7f422a74f7 00007f7f422a7490 00007f7f4225ef48 00007f7f45f731bb 00007f7f45f73532 00007f7f45fbab93 00007f7f45e67de3
0709 13:08:53.788594 (+ 19435us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:53.788635 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:53.936544 (+147909us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:53.937028 (+   484us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 13:08:53.937076 (+    48us) write_op.cc:312] APPLY: finished
0709 13:08:53.937501 (+   425us) log.cc:844] Serialized 148 byte log entry
0709 13:08:54.166666 (+229165us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:54.202845 (+ 36179us) write_op.cc:454] Released schema lock
0709 13:08:54.247046 (+ 44201us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:54.315106 (+ 68060us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:54.321004 (+  5898us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3977129,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":439684,"apply.run_cpu_time_us":47396,"apply.run_wall_time_us":1689992,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71039,"num_ops":16,"prepare.queue_time_us":77672,"prepare.run_cpu_time_us":29924,"prepare.run_wall_time_us":2956597,"raft.queue_time_us":204817,"raft.run_cpu_time_us":41758,"raft.run_wall_time_us":434115,"replication_time_us":2537983,"spinlock_wait_cycles":38257536,"thread_start_us":2942011,"threads_started":5,"wal-append.queue_time_us":2515934,"wal-append.run_cpu_time_us":12969,"wal-append.run_wall_time_us":2421150}]]}
I20260709 13:08:56.559428 20968 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:56.564924 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:56.566126 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:56.499090 27853 consensus_meta.cc:287] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.611s	user 0.013s	sys 0.000s
W20260709 13:08:56.564603 27938 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:08:56.574584 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:56.531322 21220 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:08:56.576226 27889 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:56.586186 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:56.586897 21220 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:08:56.590519 21066 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:56.592376 21767 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:56.594259 21796 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:56.599571 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:56.569188 27378 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:08:56.615247 27949 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:08:56.616684 27690 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:08:56.618847 27882 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.002s
I20260709 13:08:56.567926 27725 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.621420 27725 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:56.622224 27725 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:56.622674 27725 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.624571 27882 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.626425 21212 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:56.626972 27688 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:56.631649 27882 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:56.585096 27889 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.632377 27889 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:56.607913 27378 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.632990 27378 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:56.638490 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:56.623085 27725 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.645200 21218 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
W20260709 13:08:56.640954 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 204057us
I20260709 13:08:56.648428 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:56.655601 21793 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:56.610742 27853 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:56.654831 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:56.659745 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:56.640627 27690 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:56.656908 27892 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:56.672989 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:56.675253 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:56.657305 27531 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:56.678021 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:56.679836 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.680545 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:56.677579 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:56.680698 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.680816 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:56.681305 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.662775 27639 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:56.646394 27821 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:08:56.687199 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:56.687237 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:56.699941 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:56.667068 27853 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:56.687328 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.693529 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:56.706322 21147 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.020s	sys 0.023s
I20260709 13:08:56.693763 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.671983 20968 catalog_manager.cc:5697] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 12 to 14, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 14 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.686839 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:56.700954 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:56.708042 21071 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:56.699800 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:56.701254 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:56.706976 21064 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:08:56.708366 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:56.708858 21065 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:56.712460 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:56.713478 21141 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:56.709653 27688 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:08:56.715606 27943 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:08:56.715880 27688 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:56.716449 27688 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:56.717892 27942 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:08:56.693166 27901 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:56.719403 27869 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:56.719626 27901 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.720131 27901 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:56.723677 27690 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.725134 27771 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:56.726827 27771 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.727577 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:56.727797 27892 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:56.728204 21138 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.728261 27892 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:56.731463 27839 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:56.732025 27839 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:56.732920 27839 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.733105 27599 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.736989 27892 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.741117 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:56.706903 21147 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.020s	sys 0.023s
I20260709 13:08:56.742492 27771 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.703017 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:56.742106 21139 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:08:56.706284 21142 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:56.762562 27898 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:56.648233 27689 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.766677 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:56.787829 21068 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:56.687804 27890 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:56.786299 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:56.798993 21138 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
W20260709 13:08:56.791783 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 214086us
I20260709 13:08:56.806927 27643 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.803920 20968 catalog_manager.cc:5697] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:08:56.828145 21219 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:08:56.855723 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 53241.7us
W20260709 13:08:56.887001 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 312503us
I20260709 13:08:56.763767 27598 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.825340 27643 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:56.907263 21068 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:56.901232 20968 catalog_manager.cc:5697] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 7 to 8, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 8 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:08:56.908195 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:56.934579 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 228662us
I20260709 13:08:56.956523 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:56.900077 27790 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:56.978523 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 81848.9us
I20260709 13:08:56.911649 27888 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:08:56.968760 27847 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:56.977964 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 258854us
W20260709 13:08:56.972944 27944 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 13:08:56.995513 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:57.002115 27599 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:56.934569 27853 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:56.976102 27378 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:57.050729 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 152669us
I20260709 13:08:56.979398 27889 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:56.983410 27384 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:08:57.059896 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:57.031603 27965 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:57.080796 21797 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:08:57.065069 27944 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:08:57.055822 27948 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:08:57.126540 27948 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:08:57.082726 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 83502.2us
I20260709 13:08:57.071060 27890 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.084488 27378 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.087114 27978 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.089998 27839 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.084246 27892 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:57.088317 27954 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:08:57.079165 27964 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:57.174067 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:57.176425 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 180976us
I20260709 13:08:57.126988 27948 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:57.084000 27770 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.188184 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:57.199970 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:57.200263 27922 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:57.200755 27922 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:57.201185 27922 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:57.202548 27961 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 13:08:57.203142 27960 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 13:08:57.203462 27979 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:57.204201 27967 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:08:57.204110 27979 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.206233 27531 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:57.206414 27979 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.206825 27531 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:57.207265 27976 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:57.232389 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45746.1us
I20260709 13:08:57.233506 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:57.159132 27688 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.234613 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:57.236390 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:57.238608 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:57.243567 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:57.244115 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:57.246037 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:57.247179 27956 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.286s	user 0.000s	sys 0.002s
I20260709 13:08:57.292219 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:57.421133 21214 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 14: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:57.292074 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:57.302079 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:57.422825 21072 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:57.208633 27774 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:57.180328 27890 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.314412 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:57.430155 21215 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
W20260709 13:08:57.248863 27952 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 13:08:57.207922 27976 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.182293 27978 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:57.321354 27963 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 13:08:57.207868 21145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
W20260709 13:08:57.318785 27958 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
I20260709 13:08:57.329679 27948 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.209679 27770 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.196297 27839 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.244628 27964 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:57.392294 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 146345us
I20260709 13:08:57.193035 27981 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:57.398391 27378 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:57.235502 21217 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:08:57.320325 27970 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.499554 27970 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.500447 27948 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:57.501775 27970 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:57.319754 27982 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:57.463994 28000 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.002s
I20260709 13:08:57.207842 27531 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.503504 27688 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.244323 21216 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
W20260709 13:08:57.411332 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 454832us
W20260709 13:08:57.412211 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 168177us
W20260709 13:08:57.506462 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:57.418913 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 159607us
I20260709 13:08:57.507617 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:57.429754 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2200) took 1342 ms. Trace:
I20260709 13:08:57.391626 27757 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:57.414356 27744 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.977s	user 0.011s	sys 0.011s
I20260709 13:08:57.391971 27979 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:57.419848 27771 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.677s	user 0.014s	sys 0.000s
I20260709 13:08:57.450168 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3326) took 1359 ms. Trace:
I20260709 13:08:57.417989 27966 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:57.399902 27384 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:57.464313 21145 raft_consensus.cc:606] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:57.476047 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:57.540707 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64767.3us
I20260709 13:08:57.541679 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:57.483512 27790 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:57.483819 27690 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.548652 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:57.484529 27978 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.487938 27889 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.489804 27598 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:57.559769 27598 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:57.560184 27598 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.560647 27598 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:57.560228 27966 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.563462 27665 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:57.490499 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2216) took 1174 ms. Trace:
I20260709 13:08:57.492432 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3325) took 1402 ms. Trace:
I20260709 13:08:57.494735 27890 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.497826 27922 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.523943 21147 rpcz_store.cc:276] 0709 13:08:56.090282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.103470 (+ 13188us) service_pool.cc:224] Handling call
0709 13:08:56.127169 (+ 23699us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:56.136003 (+  8834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:56.136033 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:56.164753 (+ 28720us) log.cc:844] Serialized 52 byte log entry
0709 13:08:56.742852 (+578099us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:56.742955 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.743060 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:57.449909 (+706849us) raft_consensus.cc:1685] finished
0709 13:08:57.449931 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:57.450103 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5717,"raft.queue_time_us":14742,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":16627,"thread_start_us":541025,"threads_started":1,"wal-append.queue_time_us":707590}
I20260709 13:08:57.509253 21213 rpcz_store.cc:276] 0709 13:08:56.086981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.087146 (+   165us) service_pool.cc:224] Handling call
0709 13:08:56.091746 (+  4600us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:56.092297 (+   551us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:56.092642 (+   345us) log.cc:844] Serialized 14 byte log entry
0709 13:08:56.262144 (+169502us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b1800466480. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3fd2a6 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1
0709 13:08:56.326103 (+ 63959us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b1800466480. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:56.478425 (+152322us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:56.489721 (+ 11296us) log.cc:844] Serialized 26 byte log entry
0709 13:08:56.489897 (+   176us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:56.490027 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.490096 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:57.429516 (+939420us) raft_consensus.cc:1685] finished
0709 13:08:57.429542 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:57.429689 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3926,"prepare.queue_time_us":1401,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":17558,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":18827,"spinlock_wait_cycles":15480960,"thread_start_us":336462,"threads_started":1,"wal-append.queue_time_us":486864}
I20260709 13:08:57.528955 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:08:57.529239 27972 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:08:57.531211 27882 raft_consensus.cc:1064] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:08:57.532203 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:57.536175 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:57.608518 21801 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:08:57.539408 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:08:57.611471 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 72245.5us
I20260709 13:08:57.477309 27991 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:57.487809 27977 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.489020 27998 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.549974 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:57.486618 27976 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:57.550552 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:08:57.553190 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45664.3us
I20260709 13:08:57.554454 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:57.627305 21068 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:57.535725 27981 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.555230 27984 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:57.564306 21221 rpcz_store.cc:276] 0709 13:08:56.315929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.316077 (+   148us) service_pool.cc:224] Handling call
0709 13:08:56.316389 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:56.362782 (+ 46393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:56.362830 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:56.413669 (+ 50839us) log.cc:844] Serialized 26 byte log entry
0709 13:08:56.650834 (+237165us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:56.650894 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.652458 (+  1564us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:57.490239 (+837781us) raft_consensus.cc:1685] finished
0709 13:08:57.490262 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:57.490429 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1963,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":155,"thread_start_us":192281,"threads_started":1,"wal-append.queue_time_us":378581}
W20260709 13:08:57.622370 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.194:39351 timed out after 1.693s (SENT) [suppressed 5 similar messages]
I20260709 13:08:57.526226 27888 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:57.606613 21072 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:08:57.606979 21069 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:57.539764 27384 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.648829 21066 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: [20.12-22.13]
W20260709 13:08:57.616621 27882 raft_consensus.cc:1068] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:57.533584 27757 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.566253 21140 rpcz_store.cc:276] 0709 13:08:56.089945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.103484 (+ 13539us) service_pool.cc:224] Handling call
0709 13:08:56.103849 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:56.113977 (+ 10128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:56.114013 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:56.120023 (+  6010us) log.cc:844] Serialized 26 byte log entry
0709 13:08:56.566860 (+446837us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18001ca260. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:08:56.892673 (+325813us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:56.893071 (+   398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.893233 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:57.492125 (+598892us) raft_consensus.cc:1685] finished
0709 13:08:57.492149 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:57.492366 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15795,"raft.queue_time_us":3736,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":9662,"spinlock_wait_cycles":27954944,"thread_start_us":282169,"threads_started":1,"wal-append.queue_time_us":895611}
I20260709 13:08:57.627353 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:57.630265 27889 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:57.537819 27771 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.537597 27979 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.552367 27790 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:57.533751 27744 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:57.667176 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:57.670297 21140 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:57.631985 27993 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.670598 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:57.654578 21138 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:57.679744 21801 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:08:57.675550 21141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:57.680702 21141 raft_consensus.cc:606] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:57.660681 27725 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.004s	user 0.011s	sys 0.007s
I20260709 13:08:57.677399 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:57.670567 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:57.682794 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:57.676718 27981 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.683707 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:57.685071 27771 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.686892 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:57.687785 27976 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.688208 27688 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.689890 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:57.724232 21066 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:57.674240 27378 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:57.674033 27406 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.673972 27889 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.671703 27999 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.692435 28004 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.735854 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:57.736406 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:08:57.706029 27901 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.985s	user 0.010s	sys 0.007s
I20260709 13:08:57.678790 21070 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:57.644045 27978 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:57.699950 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:57.699030 21143 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:57.665774 27384 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:57.701778 21072 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:57.680101 21801 pending_rounds.cc:85] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 9
I20260709 13:08:57.701995 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:57.746402 21801 pending_rounds.cc:107] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:08:57.646539 27261 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:57.657285 27898 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.604460 27690 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.643155 27998 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.718602 21220 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:08:57.727267 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:57.705914 27966 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.673362 27888 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.732597 27378 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:57.738595 21068 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:08:57.758327 21068 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:57.704447 27957 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:57.646046 28003 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.746721 21801 raft_consensus.cc:2889] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:57.726195 27970 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:57.704609 27725 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.703881 28007 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:57.750233 21144 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:57.768361 21066 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:08:57.749243 21069 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:57.782125 21072 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:57.706887 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:57.753474 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:57.694284 27979 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:57.726577 27976 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.732376 27688 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.696560 27948 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:57.626948 27977 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.705634 27993 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:57.791407 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 84609.4us
I20260709 13:08:57.761693 27378 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:57.740952 27406 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.796157 21146 raft_consensus.cc:2368] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21.
I20260709 13:08:57.745723 27901 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:57.806699 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 184339us
I20260709 13:08:57.809808 21072 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:08:57.769474 27970 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.769641 27839 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:57.778295 27957 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:57.777665 28007 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:08:57.744225 27999 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.790441 27869 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:57.754825 27898 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:57.808488 27892 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:57.828111 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:57.832863 21144 raft_consensus.cc:606] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:57.743743 28004 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.757390 27978 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:08:57.814049 27992 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 13:08:57.836346 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:57.836680 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:57.966416 21071 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:57.813864 27970 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:57.807122 27981 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:08:57.819240 28007 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:57.967022 21138 raft_consensus.cc:2410] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 13:08:57.968891 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 427260us
I20260709 13:08:57.824572 27869 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.814080 27261 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.153s
I20260709 13:08:57.972198 21143 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:57.761300 28003 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:57.810534 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:57.828059 27922 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:57.976179 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:57.979116 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:57.971963 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:57.966547 28010 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:08:57.999606 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:58.000476 21138 raft_consensus.cc:2368] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 20. Current term is 21.
I20260709 13:08:57.979624 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:57.833637 27957 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:57.985085 27966 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:57.989912 21142 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:08:57.994115 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 183652us
W20260709 13:08:58.004316 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 302363us
I20260709 13:08:57.976117 27839 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.983129 27869 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:57.980141 27744 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.009459 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.010385 21139 raft_consensus.cc:2368] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:58.018031 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.020710 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:58.001086 27821 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:58.007850 27994 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:58.035660 28012 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:08:58.036041 27898 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.009155 27770 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.036904 27821 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.037609 27598 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.043265 27406 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.020112 27790 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:58.046221 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.046666 27790 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.039140 27821 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:58.055925 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 299460us
I20260709 13:08:58.058320 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2220) took 1527 ms. Trace:
I20260709 13:08:58.057447 27598 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.060439 28004 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.061298 27978 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.009558 27848 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:58.062152 28003 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.062110 27848 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.058764 21220 rpcz_store.cc:276] 0709 13:08:56.531295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.597566 (+ 66271us) service_pool.cc:224] Handling call
0709 13:08:56.598045 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:57.754556 (+1156511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:58.058054 (+303498us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400101dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:58.058255 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2576465152}
I20260709 13:08:58.067806 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.068240 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 3336) took 1392 ms. Trace:
I20260709 13:08:58.014622 27922 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:58.068774 21146 rpcz_store.cc:276] 0709 13:08:56.672369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.675434 (+  3065us) service_pool.cc:224] Handling call
0709 13:08:58.064453 (+1389019us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400279a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:58.064576 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2559781248}
I20260709 13:08:58.016619 27757 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.072594 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.076429 21142 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:08:58.076299 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.079054 28013 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:08:58.079335 28014 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:08:58.079577 28013 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.073076 27790 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:58.080004 28013 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:58.080565 27957 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:58.081282 28009 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:08:58.082415 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.083645 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.084075 27977 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.084702 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.085803 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.080420 27790 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.086887 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.089093 27790 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.092087 27999 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.012161 27966 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.092711 27690 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.094688 27976 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.022732 27839 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.096223 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.097074 27966 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.097563 27839 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.097604 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.098489 28003 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.070709 27889 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.098995 28003 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.100430 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:58.101156 28003 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.102162 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.102337 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:58.103106 21144 raft_consensus.cc:2368] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:58.104317 21064 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:08:58.108718 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.106307 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.069617 27892 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:58.076741 27922 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.111408 27998 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.111624 21070 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 19: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:58.110800 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.111613 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:58.120704 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:58.125502 21067 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:58.065594 27598 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:58.072594 27689 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:58.195580 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 75003.3us
I20260709 13:08:58.196066 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:58.196467 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:58.197211 21144 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:58.134583 27948 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:58.200887 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.205356 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:58.201396 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:58.211357 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 190723us
I20260709 13:08:58.212734 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:58.194156 27725 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.135501 27888 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.133920 27598 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.212237 21144 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:58.215503 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.221051 21144 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:58.129063 28003 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 13:08:58.244673 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.245447 21221 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:58.204353 27966 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:58.193858 27889 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:08:58.253798 21213 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:08:58.255307 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.257586 21217 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:58.260005 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47428.7us
I20260709 13:08:58.216584 27901 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.246598 28007 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.252427 27888 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:58.266323 21216 raft_consensus.cc:1275] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:08:58.287048 21221 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:58.274760 28007 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.296942 27901 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.297814 27901 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.299425 28027 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:58.299914 28022 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:08:58.300071 28027 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:58.300777 28024 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:08:58.302322 28027 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.304137 28013 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.304414 27901 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.271865 27922 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.200106 27689 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.292105 27892 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.295469 21217 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
W20260709 13:08:58.314462 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 100843us
I20260709 13:08:58.260092 27790 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:58.228084 27384 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.268637 27599 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:58.266184 27957 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:08:58.321548 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60499.1us
I20260709 13:08:58.275664 27821 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:58.276932 27598 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.324123 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:58.295933 27261 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.326285 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:58.330019 28017 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:58.277413 27888 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.271404 27988 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.119s
I20260709 13:08:58.333031 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.332072 21147 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:08:58.331445 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 105665us
I20260709 13:08:58.329224 27261 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.292038 27869 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.321568 27599 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:58.352567 27599 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.354122 27599 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:58.355945 27970 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.327404 27889 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:08:58.358249 27970 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.325353 27725 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:08:58.367050 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.323559 27957 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:58.330667 27966 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:58.340137 21213 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:58.341607 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:58.344388 21217 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:08:58.345265 27261 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.325706 27790 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.347533 27869 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.330883 27689 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.368152 21067 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:58.371714 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.382426 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.383617 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:58.387353 21065 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:58.386817 21070 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:58.392114 21067 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:58.397682 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.390769 27261 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:58.403414 27966 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:58.403927 27261 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:58.404896 27261 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.406539 21065 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:58.404556 27966 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.407560 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.409507 27384 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.412792 27598 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:58.415146 21067 raft_consensus.cc:2410] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:08:58.415912 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:58.420452 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.425163 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:58.426282 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:58.427103 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.429139 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.433856 21065 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:58.435220 21213 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:58.435253 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:58.436717 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:58.431454 27378 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.637s	user 0.013s	sys 0.000s
I20260709 13:08:58.422426 27869 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:58.438634 28017 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:58.452616 27848 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.455379 28017 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.457257 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.458562 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.455085 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.461071 21218 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:58.461153 21214 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:58.459544 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.462841 21065 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:08:58.463305 21141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c20ca56847f64504a85694d10f47558e"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:58.463943 21141 raft_consensus.cc:606] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:58.464322 21141 raft_consensus.cc:614] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:58.464022 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:08:58.443584 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 107008us
I20260709 13:08:58.465870 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:58.466603 21065 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:58.470356 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.471024 21217 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:58.472424 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.477617 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.479728 21217 raft_consensus.cc:2368] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:58.482432 21217 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:58.498087 21218 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:58.486317 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:58.502179 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.475881 27988 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.513908 21217 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:58.516706 21065 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:58.518299 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:58.518918 21065 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:58.446354 27378 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.436153 27957 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:58.520920 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.521277 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.527463 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.527721 21218 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:08:58.506659 27973 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:58.531222 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:58.533515 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:58.519737 28013 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.534467 21215 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:58.534780 21143 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:08:58.536607 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:08:58.555619 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.558219 21065 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:58.559516 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:58.559607 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.529809 27966 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:58.532871 27973 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.531569 27725 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.561513 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:58.555938 27964 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.579913 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.581959 28017 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:58.588907 21139 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:58.575438 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.525373 27988 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.592311 21218 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:58.594393 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.595100 21065 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:08:58.574754 27973 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:58.596097 21139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce55364f10c348819e7c63c08633b184"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:08:58.597610 21139 raft_consensus.cc:606] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:58.597995 21139 raft_consensus.cc:614] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:58.606391 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.609992 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.610683 21142 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:58.611716 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.612311 21142 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:08:58.613195 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:58.614337 27888 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.610761 27688 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.615917 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:58.613692 21142 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:08:58.617908 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:58.621366 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.622076 27689 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:58.622715 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:58.623759 21139 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:58.625049 27853 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:58.625550 21142 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:08:58.626266 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.627359 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:58.627875 21071 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:58.628275 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.573115 27966 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.634070 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:58.635247 21218 raft_consensus.cc:2368] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15.
I20260709 13:08:58.577754 27725 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.591234 27964 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.646725 21139 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:08:58.657675 27966 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.678030 21071 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:08:58.649622 27378 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.707441 27869 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:58.707244 27688 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:58.712077 27774 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:08:58.721871 28039 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:08:58.740693 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 119046us
W20260709 13:08:58.750415 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 146061us
I20260709 13:08:58.752712 21072 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:08:58.754101 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:58.764209 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.765095 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.765784 21218 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:08:58.770385 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.770675 28052 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:58.768530 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:58.772600 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:58.736963 27869 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.775611 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.775738 21765 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:58.756440 27689 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:58.738648 27688 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.771409 28052 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.746163 27774 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.772202 27988 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.778194 28003 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:58.791755 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:58.801483 27774 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.811081 27869 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.819998 27869 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:58.820638 27869 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:58.821645 27869 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.790989 28003 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.823836 28003 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:58.831929 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:58.784682 28052 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:58.782944 27689 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:58.837212 28050 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:08:58.804047 27725 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:58.861995 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 86343.6us
I20260709 13:08:58.863132 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:08:58.855391 28053 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:58.863116 27948 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:08:58.876292 28058 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:08:58.876924 27964 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.891453 28063 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:58.894441 28063 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:58.902009 27957 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:58.869305 27948 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:58.872259 27688 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:08:58.917366 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 50817.4us
I20260709 13:08:58.918968 21765 leader_election.cc:400] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15.
W20260709 13:08:58.919538 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 409852us
I20260709 13:08:58.920972 27948 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.922380 27948 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:58.922827 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.924124 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.925277 21066 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:58.926595 21066 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:58.932070 27599 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.932386 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.932076 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.933501 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.943042 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.946302 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.949214 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:58.944662 21767 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:58.952965 21184 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:58.953838 28063 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.948485 28061 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:08:58.959098 27973 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.960017 27689 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.960402 27790 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:58.961263 27973 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:08:58.962220 28015 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.946s	user 0.005s	sys 0.001s
I20260709 13:08:58.965366 21144 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:58.965981 21793 raft_consensus.cc:2368] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20.
I20260709 13:08:58.969174 21185 leader_election.cc:400] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21.
I20260709 13:08:58.969566 21185 leader_election.cc:403] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:58.971902 28027 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.972455 28027 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.932725 27599 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:58.975257 28029 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:58.981805 28029 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:58.975728 27955 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21.
I20260709 13:08:58.974964 28027 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.985127 27639 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:58.988174 28020 raft_consensus.cc:993] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:58.988754 28020 raft_consensus.cc:1081] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:08:58.980515 27993 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:58.981031 27599 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:58.993647 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.981105 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.994495 21212 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:08:58.997978 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:58.955776 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:58.999393 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:58.999796 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2175) took 1486 ms. Trace:
I20260709 13:08:59.000840 28042 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:59.001538 28042 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:59.001874 27965 consensus_queue.cc:1048] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:59.003454 28037 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:59.003818 28037 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:59.004328 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.004683 28037 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.005388 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.959524 28061 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:58.998581 28029 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:59.016858 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.017012 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:59.018043 21220 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:59.000211 21063 rpcz_store.cc:276] 0709 13:08:57.513569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:57.696342 (+182773us) service_pool.cc:224] Handling call
0709 13:08:57.701184 (+  4842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:57.816325 (+115141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:57.816374 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:57.816433 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:57.816487 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:58.999508 (+1183021us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540001a4d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:08:58.999575 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:58.999726 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264111360}
I20260709 13:08:59.010972 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.019917 21214 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 22: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:58.999155 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.021469 21070 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:08:59.009009 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.015880 27790 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:59.024963 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.025372 28001 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:59.025774 28001 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:08:59.026037 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:58.966734 28003 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.028580 21185 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.031816 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.026618 21221 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:59.025784 21218 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 22: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:59.006165 21213 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 15.
I20260709 13:08:59.034838 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.035691 21221 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:08:59.035733 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:59.027235 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:59.028759 21067 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:59.032667 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.038810 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.041473 21213 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:08:59.042281 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:59.046895 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.048023 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:59.048070 21070 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:08:59.049286 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:59.016667 28061 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.033504 27847 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:59.058501 21795 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:59.031247 28037 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:59.063623 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.058029 28029 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:59.055843 27964 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:59.078454 21214 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:59.060788 27790 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:59.062107 28061 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:59.060621 27965 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:59.046257 27957 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:08:59.062774 27725 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:59.085589 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 140989us
I20260709 13:08:59.026611 28001 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.085767 21111 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:59.068001 27378 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:59.064177 28065 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:59.058099 28020 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:08:59.096686 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:59.097923 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:59.093611 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:59.090482 21767 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.066455 27889 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:08:59.070346 28062 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:59.085353 27869 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:59.062949 27531 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:08:59.104060 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:59.087568 27965 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:08:59.114959 27869 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.115216 27965 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:59.110491 28013 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:59.107826 27725 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:59.105424 27848 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:59.144021 27848 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.144659 27848 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:59.089704 27790 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.149760 27790 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.095909 28052 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.152338 21797 raft_consensus.cc:2393] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:59.153723 28052 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:59.111223 28001 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:59.117635 27261 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:59.120558 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:59.128722 28003 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:59.159524 27261 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:59.129233 27888 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:59.160475 27888 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.160689 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:59.160957 27261 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.162073 27955 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:59.093451 27774 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:08:59.137107 27966 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:59.162487 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:59.119127 27689 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:59.162636 27955 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:59.115072 28071 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:08:59.163978 27821 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:59.164342 27639 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:59.164285 27966 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.164845 28027 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:59.096987 28065 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.110255 27378 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:59.135470 28013 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:59.162557 27261 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:59.130944 27531 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:59.160876 27888 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:59.192921 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 83039us
I20260709 13:08:59.220054 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:08:59.162765 28043 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.630s	user 0.000s	sys 0.003s
I20260709 13:08:59.221118 28077 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:59.221870 28077 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:59.134827 27725 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.223081 28077 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.223764 27725 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:59.185191 27639 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:59.225067 27639 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:59.196321 28013 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.199937 27955 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.194095 27973 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:59.201757 27378 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.188778 28027 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.206638 28065 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.113289 28062 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.236479 28062 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.229296 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.237008 27948 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:59.237592 27948 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.195389 27531 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.238519 27531 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:59.227770 27869 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.242455 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.252948 27639 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:59.254592 28074 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:08:59.233306 28027 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:59.260352 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 154254us
I20260709 13:08:59.267351 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.269106 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.274034 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.273540 28061 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.318423 27888 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.319761 21800 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:59.284807 27948 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:59.321272 28061 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.328133 27848 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.329689 21221 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:08:59.331770 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.344035 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.345014 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.347862 21067 raft_consensus.cc:2410] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:08:59.347957 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.348908 21071 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 15: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:59.350224 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.350303 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.351162 21070 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 13.
W20260709 13:08:59.355780 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.359817 21070 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:59.317785 27965 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.364828 27965 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.384563 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.384330 28013 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.417048 28013 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.417500 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:59.420619 27725 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.422856 27725 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.423003 27639 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:59.425786 21212 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:59.425849 21216 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:59.426354 21219 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:08:59.427594 21218 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:59.428193 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:59.428740 21216 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:59.434078 21216 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:08:59.344468 27848 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:59.438187 21071 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:08:59.446000 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.446738 21071 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:59.447535 27888 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.438105 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.441337 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.451694 27901 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:59.452358 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:08:59.452579 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 504555us
I20260709 13:08:59.453625 21036 leader_election.cc:400] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:59.453896 21036 leader_election.cc:403] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:08:59.442065 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2167) took 1934 ms. Trace:
I20260709 13:08:59.455440 21064 rpcz_store.cc:276] 0709 13:08:57.507250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:57.509000 (+  1750us) service_pool.cc:224] Handling call
0709 13:08:58.104102 (+595102us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:58.104770 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:59.441861 (+1337091us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b54003cad50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:08:59.442008 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1294381824}
I20260709 13:08:59.456349 27993 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:59.456663 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.456965 27993 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.457332 21064 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:59.457530 27993 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:59.461268 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.462090 21063 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:59.462733 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:59.452323 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.443545 28013 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:59.452260 27901 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.464044 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:59.476948 21801 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:08:59.434676 27531 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.436836 21070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39510
I20260709 13:08:59.478632 21070 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:59.468282 21140 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:59.478801 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:59.479218 21070 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:59.469081 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.469236 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.471869 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.443763 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.481213 21067 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:59.482143 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.471446 27688 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:59.485366 28007 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.485188 28037 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:59.485888 28007 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.488095 28007 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:59.489080 21138 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:59.491554 21143 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:59.497431 27892 raft_consensus.cc:1064] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:08:59.499265 27901 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.499653 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:59.502089 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:59.502206 28075 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:08:59.502736 28075 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.503592 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:59.504812 28075 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.505144 27531 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:08:59.503212 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:59.513103 21064 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:59.515547 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:08:59.510937 21111 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:59.509506 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:59.527156 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83472us
I20260709 13:08:59.487052 28037 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:59.506699 27892 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.528488 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:08:59.529733 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.530875 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:08:59.531232 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49207.6us
I20260709 13:08:59.497267 28027 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.533298 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.540966 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:08:59.534475 21216 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:59.542327 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:08:59.540722 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.545022 21069 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:59.545796 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.548972 21063 raft_consensus.cc:2410] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:08:59.528452 28075 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:59.551599 21144 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:59.538533 27965 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:59.553166 21071 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:08:59.553761 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:08:59.539234 27639 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:59.558578 21071 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:08:59.559520 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:59.562215 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.563061 21144 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:59.563874 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.564658 21143 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 22: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:59.568967 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.545753 28037 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:08:59.529573 27889 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:08:59.540222 27948 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:08:59.573958 27965 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:59.574810 27965 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.576460 21070 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:08:59.577802 28075 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:08:59.579406 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.578969 28075 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.580740 21069 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:59.582553 21067 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:08:59.539049 27901 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:08:59.582795 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.593101 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:59.560576 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3064) took 1133 ms. Trace:
I20260709 13:08:59.533185 27725 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:08:59.601459 21065 rpcz_store.cc:276] 0709 13:08:58.427192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.595884 (+168692us) service_pool.cc:224] Handling call
0709 13:08:58.596768 (+   884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:58.596998 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:58.739014 (+142016us) log.cc:844] Serialized 14 byte log entry
0709 13:08:58.778892 (+ 39878us) log.cc:844] Serialized 14 byte log entry
0709 13:08:58.990908 (+212016us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b18002d5fe0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:59.447417 (+456509us) spinlock_profiling.cc:243] Waited 895 us on lock 0x7b18002d5fe0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287 00007f7f42537208
0709 13:08:59.559810 (+112393us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:59.559867 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:59.559925 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:59.560401 (+   476us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:59.560526 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":473394,"prepare.queue_time_us":219917,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":3550,"raft.queue_time_us":680749,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":2406,"spinlock_wait_cycles":25798272,"thread_start_us":273854,"threads_started":3,"wal-append.queue_time_us":90449}
I20260709 13:08:59.533334 28076 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:08:59.612892 21070 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.612030 21216 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:08:59.616281 21111 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:08:59.619170 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:59.619877 21143 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:08:59.619212 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.613917 27901 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:08:59.624437 28027 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:08:59.624893 27901 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:08:59.629140 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 175628us
I20260709 13:08:59.603737 27882 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:08:59.636495 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 43522.7us
I20260709 13:08:59.641340 27688 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:59.642127 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:08:59.648416 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.655187 21765 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.658345 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.643173 28007 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:08:59.660264 21185 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.660545 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:59.647533 28019 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:59.666005 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.667066 28065 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:59.667591 28075 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:59.645959 27882 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:08:59.650801 27378 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:59.676054 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:08:59.677300 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:59.655262 27901 raft_consensus.cc:2764] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:08:59.651854 27688 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:08:59.701916 28025 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:59.675575 27774 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:59.689672 28082 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:08:59.707720 27882 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.708400 28082 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:59.709178 28082 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:59.709553 28082 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:08:59.710289 28082 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:08:59.675076 27639 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:59.676239 27848 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:08:59.714668 28075 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:59.715317 28037 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:59.735651 21068 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:59.736649 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1227) took 1371 ms. Trace:
I20260709 13:08:59.737056 21068 rpcz_store.cc:276] 0709 13:08:58.365246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.369204 (+  3958us) service_pool.cc:224] Handling call
0709 13:08:59.735095 (+1365891us) spinlock_profiling.cc:243] Waited 931 us on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:08:59.735404 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:59.736251 (+   847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:59.736572 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141568}
I20260709 13:08:59.715355 27261 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:08:59.713827 27993 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:08:59.756978 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:59.773262 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 96048.9us
I20260709 13:08:59.774389 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:08:59.777848 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64435.5us
I20260709 13:08:59.753458 28082 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:59.780067 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.780615 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:59.792079 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:59.787580 21143 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:08:59.796808 27848 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:59.806288 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:59.783893 28007 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:59.806142 27901 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:59.798519 28083 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:59.814738 28089 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:08:59.819357 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:59.821174 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:59.829602 21036 leader_election.cc:400] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 13 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 13. Current term is 14.
I20260709 13:08:59.830988 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:08:59.835973 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:08:59.824955 27901 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:59.841279 21071 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:59.825112 27378 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:08:59.829581 28081 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:08:59.827492 28093 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:59.831795 28083 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:59.862080 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:08:59.864784 21068 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:08:59.840619 27774 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:59.842309 28007 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:08:59.922725 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 86818.3us
I20260709 13:08:59.865720 28096 raft_consensus.cc:2764] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 13:08:59.923153 28091 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:08:59.933061 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.868184 28025 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:08:59.925114 27993 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:08:59.936904 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.938725 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.942840 21219 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:08:59.937968 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:08:59.944283 21217 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:59.942902 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:59.945554 21216 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:08:59.867705 28081 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:08:59.947913 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 299575us
I20260709 13:08:59.949826 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:08:59.956156 21212 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:59.928426 27901 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:08:59.941289 28096 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:08:59.653398 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:59.966316 27888 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:59.968511 27888 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:59.972450 21221 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:59.970057 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:08:59.950435 28090 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:08:59.979085 27888 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:59.981287 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:08:59.983547 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.983688 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.984709 21140 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:08:59.986038 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.992961 21140 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:08:59.993947 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.994436 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.994594 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.995069 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.995373 21147 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:08:59.995676 21212 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:08:59.996524 28110 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:00.001173 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.002086 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.002341 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.002725 21142 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:00.003146 28065 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:00.003643 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.004045 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.004798 21142 raft_consensus.cc:2410] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:09:00.005148 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:00.005831 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.006193 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.006628 21142 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:00.007256 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:00.003646 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:08:59.975922 28007 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:09:00.010850 28108 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:00.010680 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.011359 28108 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:00.012503 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:00.012256 28108 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:00.013669 28108 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:00.013797 21142 raft_consensus.cc:2410] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:08:59.935262 27378 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:00.018699 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.020398 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:00.021677 21139 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:00.021850 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.022418 27948 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:00.020711 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.022923 27948 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:00.021008 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:00.021984 21139 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:00.023893 27948 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.024415 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:00.024614 21139 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:00.025187 27901 raft_consensus.cc:2764] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:09:00.025735 21139 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:00.025619 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.027078 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.028230 28096 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:00.028963 28096 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:00.029079 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:00.029412 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:00.030588 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:00.031754 27957 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:00.032135 27957 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:00.032976 27957 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:00.034000 27957 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:00.035912 27688 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:00.036330 21110 leader_election.cc:400] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20.
I20260709 13:09:00.036798 21110 leader_election.cc:403] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:00.037812 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:00.038084 28096 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:00.036334 27688 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:00.039445 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.039520 27688 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:00.040436 28108 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:00.040828 27688 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:00.042676 21072 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:09:00.042526 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.043347 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3068) took 1047 ms. Trace:
I20260709 13:09:00.043712 21072 rpcz_store.cc:276] 0709 13:08:58.996029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.041978 (+1045949us) service_pool.cc:224] Handling call
0709 13:09:00.042307 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:00.043063 (+   756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.043254 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:00.044516 28096 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:00.044894 27964 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:00.045286 27964 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:00.046093 27964 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:00.046463 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:00.047715 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:00.048460 21217 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:00.048592 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:00.049916 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.050652 27993 raft_consensus.cc:2764] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:09:00.051393 27993 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:00.052938 28108 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:00.054548 28081 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:00.023200 21147 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:00.056432 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:00.059880 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.060086 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:00.061604 28096 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.062136 28096 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:00.063417 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.065485 28096 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:00.066073 28096 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:00.066463 28096 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:00.066794 28096 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:00.061084 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.068413 27993 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:00.069080 27993 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:00.069118 27888 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20.
I20260709 13:09:00.070014 27888 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:00.020059 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:00.074795 28095 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.247s	user 0.004s	sys 0.000s
I20260709 13:09:00.077054 21139 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
W20260709 13:09:00.079903 28099 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:09:00.070451 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.083319 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.083503 28108 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:09:00.083890 28106 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:09:00.084420 28108 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:00.084738 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:00.089058 28108 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:00.089776 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1387) took 1143 ms. Trace:
I20260709 13:09:00.090144 21800 rpcz_store.cc:276] 0709 13:08:58.946657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.946805 (+   148us) service_pool.cc:224] Handling call
0709 13:09:00.089571 (+1142766us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54005429d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:00.089721 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737056000}
I20260709 13:09:00.091470 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.090577 21767 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:00.092000 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:00.096323 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:00.048593 27973 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:00.101536 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:00.103201 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:00.103727 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.100221 21767 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:00.107643 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:00.102137 27988 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:00.108618 28001 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:00.111003 28081 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:00.111778 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:00.154238 21138 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:00.093057 27964 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:00.115272 28096 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.115784 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:00.170647 21793 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:09:00.115993 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:00.116868 28069 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:00.103127 27774 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:00.100287 27948 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:00.118230 27966 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:00.124282 27985 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:00.133385 21792 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:00.138538 21801 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:00.140741 21145 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:09:00.178399 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.141290 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.180420 21139 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:09:00.182643 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.143719 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.183387 28069 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:00.184424 28069 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.144322 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.186159 28081 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:00.186615 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:00.187621 28081 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:00.188107 28081 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:00.188958 28081 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:00.161278 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.190436 28082 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.190626 21138 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:09:00.101992 27973 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:00.100878 28076 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:00.155401 21795 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:09:00.198204 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:00.198649 21795 pending_rounds.cc:85] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 11
I20260709 13:09:00.198974 21795 pending_rounds.cc:107] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:09:00.161798 28100 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:00.154999 28007 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:00.198889 21216 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:09:00.199379 21795 raft_consensus.cc:2889] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:00.154438 27957 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:00.191699 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.209847 21138 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:09:00.152030 27988 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:00.151795 28075 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:09:00.167065 28013 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:00.177242 27985 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:00.213508 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:00.190968 28082 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:00.248802 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 140710us
I20260709 13:09:00.206465 27790 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.202831 28007 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:00.259119 21140 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:09:00.214097 27988 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:00.200512 28025 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
W20260709 13:09:00.326553 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.261528 27688 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:00.321547 28007 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:00.402621 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:00.463519 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 60977.6us
I20260709 13:09:00.411612 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:00.481964 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:00.434171 28102 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.443502 27901 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:00.543094 27901 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:00.208257 28100 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:00.449044 27985 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.390522 28076 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.454501 28082 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:00.319458 27790 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:00.457647 28096 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:00.464643 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.215008 27957 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.538375 28065 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:00.573535 27985 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:00.578663 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.406536 28075 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:09:00.584530 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 119950us
I20260709 13:09:00.612939 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:00.406340 27688 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:00.548794 28111 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
I20260709 13:09:00.622778 21765 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:00.586370 28112 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
W20260709 13:09:00.628525 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 425349us
I20260709 13:09:00.630649 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:00.583323 27901 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:00.640035 21138 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:00.612166 28075 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:00.624931 27688 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:00.645092 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.570672 28102 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.648074 21063 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:00.626814 27853 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:00.657378 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:00.582597 28076 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.684562 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:00.626278 28117 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:00.670648 21221 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:00.642139 28065 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:00.638638 28042 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:00.711911 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:00.729869 28116 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:09:00.697481 27378 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.761607 27378 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.762579 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2266) took 1231 ms. Trace:
I20260709 13:09:00.763839 27378 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:00.762964 21214 rpcz_store.cc:276] 0709 13:08:59.531533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.531648 (+   115us) service_pool.cc:224] Handling call
0709 13:08:59.531943 (+   295us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:59.540598 (+  8655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:59.540629 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:59.541185 (+   556us) log.cc:844] Serialized 52 byte log entry
0709 13:09:00.126203 (+585018us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b18002e4620. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:09:00.254414 (+128211us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:00.254463 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.254543 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:00.762202 (+507659us) raft_consensus.cc:1685] finished
0709 13:09:00.762231 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:00.762518 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97,"raft.queue_time_us":14909,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":6683,"spinlock_wait_cycles":13799296,"thread_start_us":91776,"threads_started":1,"wal-append.queue_time_us":994160}
I20260709 13:09:00.764389 27378 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.731041 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:00.734977 27948 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.000s
I20260709 13:09:00.764902 27378 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:00.745896 27973 consensus_meta.cc:287] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.000s
I20260709 13:09:00.765852 27973 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.766574 27901 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:00.767937 27973 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:00.651613 28069 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 13:09:00.752898 27790 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:00.753412 28075 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:00.771035 28075 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:00.766619 28083 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:00.780764 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.758335 28076 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:00.749046 28102 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:00.827549 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:00.832432 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.850023 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:00.850992 21766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.011s	sys 0.016s
W20260709 13:09:00.868947 21766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.012s	sys 0.016s
I20260709 13:09:00.845849 28065 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:00.950201 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:00.936648 27888 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.866s	user 0.007s	sys 0.004s
I20260709 13:09:00.792394 28120 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.146s
I20260709 13:09:00.791790 28083 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:00.960381 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 118268us
I20260709 13:09:00.977489 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:00.964784 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 323458us
I20260709 13:09:00.991930 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:00.998087 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.000916 21138 raft_consensus.cc:2393] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:00.999516 21141 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:09:01.020800 28122 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.021440 28122 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.023689 28123 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.024304 28123 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.002388 28082 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:01.009380 27882 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.030368 27882 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.032239 27531 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.032977 27882 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.032825 27531 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.034965 27531 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.036126 28123 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.051024 21187 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:00.993738 27888 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:01.057369 21138 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:09:01.057888 21138 pending_rounds.cc:85] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 11
I20260709 13:09:01.060465 21792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1351) took 1549 ms. Trace:
I20260709 13:09:01.062728 21792 rpcz_store.cc:276] 0709 13:08:59.500731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.505323 (+  4592us) service_pool.cc:224] Handling call
0709 13:08:59.505719 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:00.177501 (+671782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:01.050370 (+872869us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b54004934d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:01.050572 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1443000064}
I20260709 13:09:01.066946 28007 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:01.067675 28122 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.069322 28007 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.072654 28025 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:09:01.051537 27964 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.878s	user 0.005s	sys 0.006s
I20260709 13:09:01.043313 27882 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.076980 21138 pending_rounds.cc:107] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:09:01.077268 21138 raft_consensus.cc:2889] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:01.018641 28029 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.045130 27378 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.039911 27848 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.074290 27882 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:01.061969 27988 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.661s	user 0.013s	sys 0.000s
W20260709 13:09:01.098933 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 251969us
I20260709 13:09:01.102499 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.090397 28065 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.040055 28129 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:09:01.079000 28110 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.123693 28110 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.125741 28110 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.091188 27901 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.076725 27531 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:01.093724 27848 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.129316 27988 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.130163 27848 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.131398 27988 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.144515 21147 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:01.148739 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:01.151811 21142 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
W20260709 13:09:01.145562 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 463423us
I20260709 13:09:01.091672 28029 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.159314 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:01.165093 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:01.166442 28122 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:01.159700 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.088377 28007 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:01.126899 27901 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.209640 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:01.210726 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.212041 21213 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:01.214954 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.216055 21217 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 22: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:01.221475 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.222358 21792 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 13:09:01.223814 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 244610us
I20260709 13:09:01.225612 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:01.147557 28102 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.230499 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:01.162189 27639 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:01.233224 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.236848 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.161680 28083 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.178545 28125 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.126282 27688 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:01.233150 27985 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.000s
I20260709 13:09:01.220747 28027 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:01.259770 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:01.266577 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.275494 21215 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:09:01.276151 21215 pending_rounds.cc:85] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 15
I20260709 13:09:01.276455 21215 pending_rounds.cc:107] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:09:01.276793 21215 raft_consensus.cc:2889] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:01.223749 28007 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:09:01.286047 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:01.290463 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 64945us
I20260709 13:09:01.293187 21035 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:01.296903 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 277506us
I20260709 13:09:01.248260 28128 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:01.304265 21141 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:01.262101 28083 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:09:01.304387 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 312888us
I20260709 13:09:01.238898 28102 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.351707 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.263713 28125 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.304935 21138 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
W20260709 13:09:01.339905 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 111090us
I20260709 13:09:01.402786 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 971) took 1300 ms. Trace:
I20260709 13:09:01.432719 21142 rpcz_store.cc:276] 0709 13:09:00.102405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.102965 (+   560us) service_pool.cc:224] Handling call
0709 13:09:01.402552 (+1299587us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54002963d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:01.402690 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239243648}
I20260709 13:09:01.401208 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3394) took 1251 ms. Trace:
I20260709 13:09:01.434226 28082 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:01.411563 28121 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:01.435420 28136 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:09:01.436152 28093 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.436636 27882 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.436887 28093 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.437063 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.437489 28029 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.434521 21147 rpcz_store.cc:276] 0709 13:09:00.149552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.185095 (+ 35543us) service_pool.cc:224] Handling call
0709 13:09:00.185383 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.145084 (+959701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:01.400888 (+255804us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b5400296dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:01.401131 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2177133440}
I20260709 13:09:01.438970 28093 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.433158 28153 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.431989 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:01.447441 21145 raft_consensus.cc:1240] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:01.432478 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:09:01.452508 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 159403us
I20260709 13:09:01.457221 21221 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:09:01.460937 21219 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:01.462069 21216 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:01.462651 21221 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:09:01.465101 28156 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:01.465670 28156 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:01.468422 28147 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:09:01.468640 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.452600 28153 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:01.470134 28075 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.001s
I20260709 13:09:01.470906 28153 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.471508 28153 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.472016 28007 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.471961 28153 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.472658 28007 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.473168 28007 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.473966 28153 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.474468 28153 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:01.460495 28013 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:01.302132 27888 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:01.436182 28121 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.475611 21795 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:01.475950 27888 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.471004 27901 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.477047 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.478204 27888 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.332849 28083 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.478273 21212 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:09:01.484248 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:01.483843 21070 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:01.481892 21071 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:01.405607 28135 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:01.364619 28138 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:01.481859 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:01.479812 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:01.524581 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.475322 28153 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:01.464188 27774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.282s	user 0.004s	sys 0.025s
I20260709 13:09:01.276894 28027 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:01.517380 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 85010.6us
I20260709 13:09:01.521885 21070 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:09:01.515631 21065 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:09:01.525213 21219 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:01.477252 28121 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.526010 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:01.522590 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:01.532341 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.538451 21064 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:01.539359 21219 raft_consensus.cc:2410] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:09:01.532456 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:01.534145 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.468441 28125 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.534822 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.480495 28143 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.534854 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.484706 28029 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:09:01.517966 28157 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:01.535849 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.536080 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.524901 21066 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:01.536541 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:01.538497 21071 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:01.540938 21065 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:09:01.540206 21139 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
W20260709 13:09:01.528928 27774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.349s	user 0.005s	sys 0.025s
I20260709 13:09:01.535171 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.543210 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.546083 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.463837 27882 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:09:01.547360 28027 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.548282 21216 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:01.549734 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.559963 21218 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:01.534165 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.308115 28128 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.531169 28007 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.556394 28013 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.557049 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:01.561058 21066 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:01.567744 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:01.555805 21070 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:01.563520 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:01.569689 21065 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:01.571455 21065 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:01.578680 21212 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:09:01.578871 28128 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.580206 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3505) took 1559 ms. Trace:
I20260709 13:09:01.580622 21140 rpcz_store.cc:276] 0709 13:09:00.020965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.021085 (+   120us) service_pool.cc:224] Handling call
0709 13:09:00.021420 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:00.265484 (+244064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:01.579980 (+1314496us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540036e054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:01.580149 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":540266240}
I20260709 13:09:01.581940 27889 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:01.582373 27889 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:01.583221 27889 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.584332 27889 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:01.586599 21069 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:09:01.587607 28144 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:01.562260 27774 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:01.589457 27774 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.592058 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:01.606316 21069 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:01.535409 28138 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.608902 28138 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.610005 28083 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.614295 21069 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:09:01.607071 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.617770 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:01.541920 28082 consensus_queue.cc:1059] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:09:01.523826 28135 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.591413 27888 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.565092 28157 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:01.628644 21070 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:09:01.631907 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:01.575094 28159 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:01.630023 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:01.612744 28033 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:01.635742 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:01.626001 28142 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:01.649552 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.653285 21221 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:09:01.650717 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:01.643242 27847 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 13:09:01.658560 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:01.554046 28143 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.641170 28156 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.643759 28159 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.605273 28013 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:01.620319 27790 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:09:01.648506 27988 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:01.664184 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.656790 28125 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:09:01.665843 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.626137 28097 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:09:01.654713 27598 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:01.658250 28142 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.668088 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:01.668679 28027 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:01.669479 28135 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:01.714592 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46579.8us
I20260709 13:09:01.643415 28122 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:09:01.671250 28143 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.675884 28102 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.677968 28013 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.644009 27888 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.655361 28033 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:01.681198 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.682416 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.661820 27848 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:01.663307 28083 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.737591 21065 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:09:01.738865 21065 pending_rounds.cc:85] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 11
I20260709 13:09:01.716943 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.717996 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.739183 21065 pending_rounds.cc:107] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:09:01.739132 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:01.726769 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:01.739709 21221 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:01.740108 21068 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:01.735213 28157 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:01.740900 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.741997 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.651765 28148 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:09:01.667778 28159 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:01.743575 28102 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:01.705163 27988 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:01.706060 28129 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.737077 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.706310 27598 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.714344 28142 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:01.663053 28153 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.714740 28027 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.739691 21065 raft_consensus.cc:2889] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:01.725688 27847 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:09:01.763659 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.738397 27848 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.714041 28125 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.737847 27384 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:01.762611 27988 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:01.766059 27598 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:01.758436 28076 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:01.768541 28027 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:01.768047 27847 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:01.762962 28097 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:09:01.773002 27384 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:01.772691 27848 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:01.772768 28153 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.764355 28083 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.794297 27847 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:01.797885 28154 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:09:01.782539 28124 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.155s
I20260709 13:09:01.767943 28129 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.839707 27384 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.841013 27384 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:01.841702 28129 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.846060 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.848851 28083 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.855196 28176 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:01.855958 28176 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.858485 28176 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.864825 27688 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.815927 27957 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.825810 27847 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:01.827775 28166 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:09:01.828892 27888 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.832250 28033 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.792615 28076 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.878485 27847 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.879191 28076 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.879786 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:01.880554 21215 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:01.881536 28115 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.676s	user 0.012s	sys 0.023s
I20260709 13:09:01.882174 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:01.884467 28118 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.340s	user 0.005s	sys 0.012s
I20260709 13:09:01.884341 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:01.885694 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.800757 28153 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.872246 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.889995 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.890543 21213 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:09:01.890579 21035 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:01.891721 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:01.892295 21793 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:01.892956 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2249) took 2851 ms. Trace:
I20260709 13:09:01.893673 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.893344 21220 rpcz_store.cc:276] 0709 13:08:59.035028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.040178 (+  5150us) service_pool.cc:224] Handling call
0709 13:08:59.040590 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:59.058577 (+ 17987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:59.058609 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:59.649601 (+590992us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b18003c8620. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3fd2a6 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a
0709 13:09:00.121568 (+471967us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b2c0011d108. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:09:00.122076 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:09:00.156723 (+ 34647us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:00.156764 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.156833 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:01.856151 (+1699318us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:01.886477 (+ 30326us) raft_consensus.cc:1685] finished
0709 13:09:01.886507 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:01.886660 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17521,"raft.queue_time_us":17908,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4327,"spinlock_wait_cycles":14633984,"thread_start_us":34247,"threads_started":1,"wal-append.queue_time_us":58583}
I20260709 13:09:01.896095 27688 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.896489 28153 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.896247 28159 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.896879 27688 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.897507 21066 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:01.897905 28138 raft_consensus.cc:3055] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:01.800029 28110 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.898360 28138 raft_consensus.cc:740] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:01.898645 28110 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.899473 27688 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.899230 28138 consensus_queue.cc:260] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.900509 28138 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:01.900712 28110 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.901845 28159 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:01.904207 28142 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.905146 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:01.905287 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.906379 28142 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:01.906584 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.270s	user 0.017s	sys 0.021s
W20260709 13:09:01.906975 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.271s	user 0.017s	sys 0.021s
I20260709 13:09:01.916038 27848 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.918385 27848 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.919317 28027 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.920423 21212 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:01.922171 28027 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.926318 21793 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:09:01.929142 21215 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:09:01.930168 27598 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:01.930780 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.932242 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:01.932375 27598 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:01.932745 21792 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:01.932978 28159 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:09:01.933712 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 197416us
I20260709 13:09:01.933871 21035 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:01.935376 21035 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:01.874817 27988 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:01.886929 21799 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:01.938244 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.888756 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.940953 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.940980 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.814252 28125 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:01.889235 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:01.936496 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:01.937544 28153 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:02.011883 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.019496 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.020875 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:02.014966 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79760.8us
I20260709 13:09:02.050839 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.051172 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.053491 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:02.058121 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.059265 21071 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:02.062076 21111 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:01.937844 27688 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:02.072243 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.073215 21145 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:02.014748 27598 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.075419 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.075783 21071 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:09:02.078233 21071 pending_rounds.cc:85] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 11
I20260709 13:09:02.078644 21071 pending_rounds.cc:107] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:09:02.078912 21071 raft_consensus.cc:2889] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:02.078284 21146 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:02.079356 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.080355 21140 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:09:02.082630 21070 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:02.077953 28013 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.083653 28013 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.085839 28013 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:02.087364 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:02.088860 27988 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:02.016058 28144 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.074s
W20260709 13:09:02.089568 28114 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.832s	user 0.008s	sys 0.009s
I20260709 13:09:02.072806 27378 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:09:02.090436 28001 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.094484 21065 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:02.097988 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:02.099381 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:02.102389 28100 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:02.102463 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:02.103610 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:02.075188 28110 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:02.105782 28110 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.106153 28110 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:02.107237 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 977) took 1966 ms. Trace:
I20260709 13:09:02.107668 21143 rpcz_store.cc:276] 0709 13:09:00.140498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.141106 (+   608us) service_pool.cc:224] Handling call
0709 13:09:00.170183 (+ 29077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:00.622576 (+452393us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:09:01.003113 (+380537us) log.cc:844] Serialized 14 byte log entry
0709 13:09:01.338913 (+335800us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:01.339157 (+   244us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:01.339329 (+   172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.107017 (+767688us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54001ead54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:02.107062 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:02.107178 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8024,"prepare.queue_time_us":472221,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":19377,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":1039814400,"thread_start_us":689826,"threads_started":2,"wal-append.queue_time_us":393536}
W20260709 13:09:02.110733 28126 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.036s	user 0.000s	sys 0.001s
I20260709 13:09:02.112033 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:02.112900 27847 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:02.114521 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:02.115446 28104 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:02.115808 28104 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.115993 28013 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:02.078097 28061 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.116181 28104 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:02.116031 21066 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:02.055228 28075 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:09:02.090989 28001 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.050864 28128 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:02.119091 21792 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:02.145469 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:02.089460 27598 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.178448 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.179083 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.179966 21220 raft_consensus.cc:2410] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:09:02.180354 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:02.179992 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:02.221297 21219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.011s	sys 0.023s
I20260709 13:09:02.145443 28120 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:02.222997 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:02.154817 28153 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:09:02.221735 21219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.011s	sys 0.023s
W20260709 13:09:02.226442 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 43499.6us
I20260709 13:09:02.228945 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:02.228405 21069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.015s	sys 0.012s
W20260709 13:09:02.232241 21069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.015s	sys 0.012s
I20260709 13:09:02.154902 27973 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:02.116858 28061 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.145746 27888 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
W20260709 13:09:02.240936 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 477356us
I20260709 13:09:02.250854 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:02.251967 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.219681 28042 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.254395 21146 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:02.256727 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.261655 21139 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:02.257862 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:02.252679 21140 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:02.221220 28144 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.228586 28108 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:02.271035 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.162875 28124 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 13:09:02.228214 28082 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.157413 28128 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.223248 28138 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:02.185981 28142 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:09:02.250993 28153 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:02.261521 28148 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:02.286993 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:02.294725 28138 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.296224 28177 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:02.296703 28177 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.297103 28177 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:02.298533 28110 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.299769 28135 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:02.315506 28135 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:02.316471 28135 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.317691 28135 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:02.267946 28042 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.320324 28178 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:02.321261 28042 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.303546 28131 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:02.322069 28131 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.324316 28131 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:02.236092 28120 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.324648 27882 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:02.325151 27882 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.325516 27882 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:02.326462 28120 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.304690 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.307415 28138 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:02.307962 28173 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.527s	user 0.000s	sys 0.002s
I20260709 13:09:02.311746 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1360) took 1668 ms. Trace:
I20260709 13:09:02.327400 21066 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:02.327682 21795 rpcz_store.cc:276] 0709 13:09:00.643161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.643330 (+   169us) service_pool.cc:224] Handling call
0709 13:09:00.643598 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.476077 (+832479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.311524 (+835447us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54006612d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:02.311689 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912673024}
I20260709 13:09:02.290997 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.312306 21767 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:02.329478 28110 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:02.331975 27598 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:02.306283 21212 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:02.332214 28075 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:02.296285 28061 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.333055 28061 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:02.332995 28120 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:02.333446 28075 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:02.262220 27774 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 13:09:02.334707 28075 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:02.335740 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:02.335981 21067 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:02.336524 28097 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:02.265936 27965 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:02.340219 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.340866 21072 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:02.341686 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:02.342193 21072 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:02.342257 28128 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:02.342659 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.343915 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:02.283865 28108 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:02.345244 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.346105 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.346664 28108 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.349061 28142 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:02.346690 21071 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:02.349635 28142 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:02.350715 21071 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:09:02.325577 20963 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 14 to 15, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 15 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:02.352512 20963 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 21, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 21 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:02.353826 28153 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.351346 27531 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:02.333644 28001 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.342335 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.358613 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:02.360103 21070 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:02.357482 21072 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:02.359422 21792 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEARNER]: Advancing to term 21
I20260709 13:09:02.369524 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:02.349095 28167 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.560s	user 0.010s	sys 0.006s
I20260709 13:09:02.377693 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.259788 27973 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.378612 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.380112 21066 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:02.381431 27384 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:02.381875 27384 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.382200 27384 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:02.291601 28144 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.384846 21065 raft_consensus.cc:2410] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:09:02.355479 27965 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.387163 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:02.387306 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.387599 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:02.388634 21138 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:02.388535 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.390079 21140 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:02.390162 21139 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:02.392052 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.393285 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:02.395784 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:02.405277 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:02.405617 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.406997 21214 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:02.411204 21221 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:02.414597 20963 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 13 to 21, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 21 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:02.398449 27378 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.416913 27378 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.417419 28159 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.417073 20963 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 20, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 20 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:02.417960 28159 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.406265 21220 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:02.419442 27973 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.420084 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.420943 21143 raft_consensus.cc:2410] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:09:02.421576 21217 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: [21.10-21.10]
I20260709 13:09:02.350768 28142 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.291765 28082 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.434959 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:02.396459 28157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.019s	sys 0.029s
I20260709 13:09:02.438032 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.440661 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.443011 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.443784 21070 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:02.443981 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:02.446247 27688 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:02.446698 27847 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:02.419015 20963 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:02.452373 28159 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:02.436408 27966 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:02.453013 27966 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:02.453385 27966 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:02.453874 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.454541 21143 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:02.447024 28100 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:02.455932 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:02.456115 28100 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:02.456969 28100 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.460441 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:02.461838 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:02.462275 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.477851 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:02.487979 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:02.420336 28138 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:02.491899 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.492162 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.492894 21065 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
W20260709 13:09:02.460005 28157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.019s	sys 0.029s
I20260709 13:09:02.505287 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:02.500027 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:02.502060 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.502418 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.506778 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.506915 21220 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:02.512531 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.434893 27378 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.519924 21146 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:09:02.503052 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:02.525660 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:02.422480 28108 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:09:02.479580 28083 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:02.530167 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:02.487041 28159 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:02.428166 27985 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:09:02.487704 27957 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:02.552459 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 46133us
I20260709 13:09:02.497085 28138 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.550109 28148 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:02.494051 28077 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:02.534078 28157 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.623013 28157 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.625991 28157 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:02.596678 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 93728.7us
I20260709 13:09:02.487330 27888 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:09:02.560927 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:02.630017 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:02.512326 27948 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:09:02.554977 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49768.4us
I20260709 13:09:02.598376 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:02.631572 27948 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.634089 27948 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.533327 28027 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:02.477330 28181 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 13:09:02.600840 21185 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:02.627200 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:02.615166 28188 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:09:02.615688 28041 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:02.616823 28144 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.617720 27378 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:02.621836 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3389) took 2474 ms. Trace:
I20260709 13:09:02.599464 27848 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:02.677055 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 76274.6us
I20260709 13:09:02.559734 28129 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:02.643025 21144 rpcz_store.cc:276] 0709 13:09:00.147848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.148002 (+   154us) service_pool.cc:224] Handling call
0709 13:09:00.169444 (+ 21442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:00.169847 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:00.169889 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:00.176859 (+  6970us) log.cc:844] Serialized 26 byte log entry
0709 13:09:01.861522 (+1684663us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18003f1e40. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:09:01.867925 (+  6403us) spinlock_profiling.cc:243] Waited 7.48 ms on lock 0x7b18003f1e40. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f557 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:09:02.264083 (+396158us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:02.264143 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.264288 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:02.621517 (+357229us) raft_consensus.cc:1685] finished
0709 13:09:02.621545 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:02.621732 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3948,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2161,"spinlock_wait_cycles":43601536,"thread_start_us":402300,"threads_started":1,"wal-append.queue_time_us":2442553}
I20260709 13:09:02.596885 27957 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.596782 28138 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:02.685643 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:09:02.688062 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 129065us
I20260709 13:09:02.688864 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:02.640929 28027 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:02.691803 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 431150us
I20260709 13:09:02.693682 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:02.694968 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 63157.8us
I20260709 13:09:02.681717 28129 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.676782 28143 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:02.687429 27957 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:02.715902 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.718384 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:02.692160 28077 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:02.701794 28027 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:02.695672 27847 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:02.694773 28176 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:02.692585 27948 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:02.678435 28041 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:02.680815 27378 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:02.713049 28104 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.002s
I20260709 13:09:02.731963 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:02.732887 21800 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:02.714192 28129 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:02.734020 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:02.736104 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 403977us
I20260709 13:09:02.722254 27985 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:02.718104 28076 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:02.738348 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 250436us
I20260709 13:09:02.738241 21767 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:02.747028 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:02.743516 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:02.731426 27847 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.736013 28128 raft_consensus.cc:2764] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:09:02.758937 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:02.733544 28176 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:02.714354 28130 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 13:09:02.757818 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 258797us
I20260709 13:09:02.724298 28155 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:09:02.765216 21140 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:09:02.768078 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:02.732920 27853 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:09:02.722852 28143 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.771948 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:02.755009 27847 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:02.751893 27774 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:02.738057 27948 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:02.828557 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.755259 28082 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:02.756836 28096 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:02.789225 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:02.736900 28104 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:02.804554 28157 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:02.768594 28176 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:02.826280 28191 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:09:02.846664 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.805738 28194 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:02.757895 28061 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.082s
I20260709 13:09:02.854908 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:02.832623 27774 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:02.854602 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 135144us
I20260709 13:09:02.862828 21794 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
W20260709 13:09:02.860906 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 71776.9us
I20260709 13:09:02.871541 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:02.863829 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 236738us
W20260709 13:09:02.851011 28097 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 13:09:02.875087 28097 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:02.878674 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:02.910311 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:02.912163 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:02.921010 21219 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:09:02.923808 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 216831us
W20260709 13:09:02.877501 28183 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:09:02.869155 28082 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:02.879783 27688 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:02.878969 27774 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:02.863694 28194 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:02.935240 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:02.825184 28077 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:02.923082 28157 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:02.947258 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:02.922814 28198 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:02.955744 28041 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:09:02.956503 28104 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:02.959149 28143 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:02.951483 28157 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:02.968609 28157 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:02.970320 28195 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:09:02.970615 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.971446 21067 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 21: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:02.936568 27639 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.975692 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:02.977192 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:02.978456 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 99866.7us
I20260709 13:09:02.980494 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:02.943845 28194 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:02.990433 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 78367.4us
I20260709 13:09:02.996686 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.996941 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.997701 21140 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:02.998440 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:02.982393 28104 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:03.005525 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.004832 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:03.006800 21795 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:03.012339 21110 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.586s (SENT) [suppressed 14 similar messages]
I20260709 13:09:03.018846 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.020114 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:03.026173 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:02.995512 28200 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:02.982050 27639 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.029642 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.034389 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:03.033725 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:03.046867 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 65147.7us
I20260709 13:09:03.047811 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.051743 27955 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:03.055944 28153 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:03.056687 28153 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:03.056926 28200 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:03.057771 28200 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.057982 28153 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.063301 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:03.059348 28082 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:03.025229 28198 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:03.069773 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.071334 21792 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:09:03.030821 28199 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:09:03.080701 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 68412.1us
I20260709 13:09:03.028352 28205 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:03.032666 27378 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:03.047400 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 109405us
I20260709 13:09:03.038021 27688 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:03.083303 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:03.098536 28210 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:09:03.083055 28192 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:09:03.100471 27384 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.718s	user 0.007s	sys 0.004s
I20260709 13:09:03.030903 28196 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 13:09:03.100900 27384 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.102321 21145 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:03.102921 28027 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.104280 27639 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:03.105387 28027 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:03.105885 27966 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.652s	user 0.009s	sys 0.002s
I20260709 13:09:03.106307 27966 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.081743 28199 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:03.081164 28061 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:09:03.117473 27882 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.791s	user 0.011s	sys 0.001s
I20260709 13:09:03.078203 28200 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:03.113914 28177 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.816s	user 0.015s	sys 0.006s
I20260709 13:09:03.111904 28199 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:03.093015 27948 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:03.135965 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53102.8us
I20260709 13:09:03.138094 28001 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:03.140334 27965 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.103021 27384 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:03.164806 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 110439us
I20260709 13:09:03.102989 28129 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:03.145159 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49392.6us
I20260709 13:09:03.153563 27847 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.161616 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.082073 28082 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:03.164594 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 139339us
I20260709 13:09:03.144019 28208 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:03.111055 28176 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.173588 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.174686 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.179597 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:03.174588 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.112046 28138 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.161310 28104 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:03.180353 21214 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:03.178078 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.112360 28205 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.196466 27957 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:03.197644 28190 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.479s	user 0.002s	sys 0.000s
I20260709 13:09:03.197997 28205 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:03.198652 27957 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.178388 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.204631 21795 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:03.182731 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.177021 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.206380 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:03.207757 28129 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.208664 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:03.126582 27882 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.212466 27882 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:03.185232 21140 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:03.186846 21216 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:03.191475 27966 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.214205 27965 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:03.193117 28201 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:03.193324 28096 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:03.215576 28176 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:03.194729 27888 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:03.216598 28082 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.216670 27888 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:03.217715 27888 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.178176 27948 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.219537 27948 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.220678 28138 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:03.162146 28001 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:03.221479 21140 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:09:03.221518 27948 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.222097 28001 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.223300 28001 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:03.223845 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:03.224560 21140 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:03.226552 28201 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.228683 28201 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.229470 21069 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:09:03.204007 21220 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: [21.10-21.10]
I20260709 13:09:03.177987 27384 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:03.230206 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.231110 21139 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:09:03.231437 27774 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.215332 28096 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.235244 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.236342 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:03.236879 21072 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:03.219165 27888 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:03.161046 28177 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.244143 21794 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:09:03.232604 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:03.188114 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.254581 21144 raft_consensus.cc:2368] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22.
I20260709 13:09:03.255594 28177 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:03.257853 28199 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.259537 28222 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:03.260133 28222 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.261374 27774 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:03.263593 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75540.3us
I20260709 13:09:03.264736 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.264029 21214 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:03.276377 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.289640 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:03.290581 21139 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:03.294713 28153 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:03.295715 28153 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:03.296290 28153 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.297237 28033 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:03.262378 28081 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:03.310935 27955 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:03.311380 27955 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:03.312026 28096 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.312176 27955 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.312940 21139 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:03.315886 27955 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:03.301573 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.299929 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.322407 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:03.323660 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:03.332939 21139 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:03.326774 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:03.333599 21214 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:03.320607 27847 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:03.329084 21072 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:03.326969 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.386373 21220 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:03.385622 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.386819 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.388286 21220 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:03.360397 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 277564us
I20260709 13:09:03.388935 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.389338 21213 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:03.389433 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.385008 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:03.391005 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.391716 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.391827 21139 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:03.392387 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.392856 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:03.393489 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.393744 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.393867 21138 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:09:03.395308 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.396147 21220 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:03.396059 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.397977 21218 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:03.399718 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.399952 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.402586 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.404438 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.405016 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.405771 21215 raft_consensus.cc:2393] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 17: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:09:03.405898 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65463.7us
I20260709 13:09:03.407815 21214 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:09:03.392208 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.408612 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.409359 21765 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.393941 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.411794 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.413092 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.416658 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.410202 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.418345 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:03.434168 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:03.458833 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:03.430344 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:03.435055 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:03.434635 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:03.460321 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.460911 21799 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:09:03.462172 21066 raft_consensus.cc:2410] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 13:09:03.436177 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.429746 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:03.430085 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:03.420843 21072 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:03.415546 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:03.464113 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.334900 28222 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.420670 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.464746 21794 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:03.468995 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:03.481285 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.469435 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:03.494418 28222 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:03.492057 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:03.409749 27985 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:03.499724 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.409926 28077 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.501521 28077 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.504351 28077 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.409727 28177 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:09:03.512020 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2310) took 1028 ms. Trace:
I20260709 13:09:03.512460 21217 rpcz_store.cc:276] 0709 13:09:02.483734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.484798 (+  1064us) service_pool.cc:224] Handling call
0709 13:09:02.485116 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:02.485526 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.485565 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:02.498072 (+ 12507us) log.cc:844] Serialized 26 byte log entry
0709 13:09:03.257660 (+759588us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b18003ca660. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:09:03.407047 (+149387us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:03.407103 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.409829 (+  2726us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.511680 (+101851us) raft_consensus.cc:1685] finished
0709 13:09:03.511700 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.511955 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7351,"raft.queue_time_us":1627,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":19755,"spinlock_wait_cycles":4214144,"thread_start_us":149175,"threads_started":1,"wal-append.queue_time_us":967814}
I20260709 13:09:03.514524 28082 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:03.516317 21212 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 22: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:03.417951 28104 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:03.458190 27973 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:03.416540 28061 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:03.421146 28148 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:03.421036 28096 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:09:03.465723 28201 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:03.493973 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:03.521973 21212 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:03.522660 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:03.519810 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.523571 21066 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 19: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:09:03.496277 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 80861.3us
I20260709 13:09:03.492269 21796 raft_consensus.cc:2376] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17. Re-sending same reply.
I20260709 13:09:03.413532 28153 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:03.516193 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 46860.6us
W20260709 13:09:03.468691 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 50414.4us
I20260709 13:09:03.515113 28082 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.533771 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:03.411322 28138 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.108s
I20260709 13:09:03.549674 28042 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:03.550125 28042 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:03.520648 28077 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:03.549407 21139 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 10 to 14, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:03.531052 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:03.550992 28077 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:03.552204 28207 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.211s
I20260709 13:09:03.553237 28077 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.554248 28233 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:03.554877 28233 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.558413 28096 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:03.558861 21217 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:03.565363 21111 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:03.565747 21799 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:03.524259 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.537055 28215 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:03.463114 28117 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:09:03.540015 28082 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:03.575153 28181 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:03.575660 28181 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:03.576507 28181 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.412209 28157 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.577456 28222 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:03.577718 28181 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:03.577908 21067 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:03.579737 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:03.580837 28199 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:03.582401 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.544255 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:03.584143 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.558333 28233 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:03.584781 21070 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:03.581067 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:03.558982 28096 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:03.551839 21801 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:03.534883 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:03.544803 27985 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:03.534171 28194 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
I20260709 13:09:03.589164 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:03.607180 21067 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:03.607750 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.590619 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:03.590312 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66112.7us
I20260709 13:09:03.613826 27985 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.588925 28117 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:03.589251 28157 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:03.568457 28215 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:03.617395 28215 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:03.617909 28194 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.523522 28104 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.620050 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.587483 27888 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:03.620751 27888 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:03.620886 27973 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:03.621156 27888 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:03.621326 28104 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:03.588609 28065 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:09:03.622339 27973 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:03.622841 28065 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:03.623268 28065 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:03.598369 28138 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:03.615372 28117 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:03.588165 28096 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.625705 28117 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.624277 28065 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.533313 28148 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:03.606241 21072 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
W20260709 13:09:03.662750 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 55082.3us
I20260709 13:09:03.624253 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.663769 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.662834 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:03.662786 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 58397.5us
I20260709 13:09:03.634299 21067 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:03.669317 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:03.626869 28117 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:03.639106 28025 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index 10 to 14, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:03.642112 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2250) took 1272 ms. Trace:
I20260709 13:09:03.648574 21035 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:03.653496 28201 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:03.706348 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 57855.7us
I20260709 13:09:03.660009 21793 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:09:03.660741 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:03.661427 28096 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:03.662021 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57200.4us
W20260709 13:09:03.713260 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49601.9us
W20260709 13:09:03.636368 28225 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.219s	user 0.000s	sys 0.003s
I20260709 13:09:03.672830 21067 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
W20260709 13:09:03.717381 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 51341.2us
I20260709 13:09:03.722615 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.677874 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:03.593148 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:03.680871 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 68261.7us
I20260709 13:09:03.688556 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:03.704641 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 114091us
I20260709 13:09:03.701428 21071 rpcz_store.cc:276] 0709 13:09:02.369449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.370876 (+  1427us) service_pool.cc:224] Handling call
0709 13:09:02.376665 (+  5789us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:02.741445 (+364780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.741492 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:02.742086 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:09:02.935545 (+193459us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:02.935578 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.641694 (+706116us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000fb254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:03.641746 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.641796 (+    50us) raft_consensus.cc:1685] finished
0709 13:09:03.641811 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.642049 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135989,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":37392,"spinlock_wait_cycles":282039040,"thread_start_us":434687,"threads_started":2,"wal-append.queue_time_us":274332}
I20260709 13:09:03.716825 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.728155 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.699564 28001 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:03.732226 21800 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:03.733114 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.737249 21066 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:03.701421 28076 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:03.718703 28201 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:03.721050 28222 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:03.743134 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:03.746565 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.742410 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.749147 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.739163 21071 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:09:03.752270 28082 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:03.752635 28199 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:03.726449 28176 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:03.754657 21035 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.755718 21800 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:09:03.756964 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.757529 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.758467 21113 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:03.760015 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.763756 28153 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:03.587217 28042 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.771533 27965 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:03.772800 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.771647 28143 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:03.776764 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.737219 28097 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:09:03.779860 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51821.5us
I20260709 13:09:03.781159 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.781867 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.747936 28122 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:03.794047 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:03.794253 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.794464 21142 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:03.789062 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.797446 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.751029 28222 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:03.809756 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.742008 28131 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:09:03.774859 28200 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:09:03.835888 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 89416.6us
I20260709 13:09:03.780035 28143 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:03.841164 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:03.842962 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 54008.8us
I20260709 13:09:03.782578 28157 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:03.782389 28001 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:03.844341 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:03.842140 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.784924 28027 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:03.750138 28202 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:09:03.785993 27948 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:03.786567 28076 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:03.800436 28013 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:03.800976 28128 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:03.800642 27598 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:03.859939 21765 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:03.830169 27965 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:03.837635 28222 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:03.844280 27384 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:03.869179 28157 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:03.838953 28208 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:03.884404 28223 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.776605 28148 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:03.884967 28224 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:03.861882 28240 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
W20260709 13:09:03.892333 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.841851 27882 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:03.846557 28143 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:03.852854 27988 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:03.850939 27848 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:03.872383 28129 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:09:03.968154 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 188799us
W20260709 13:09:03.939893 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 179673us
I20260709 13:09:03.872006 27957 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:03.897856 27847 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:03.864737 28200 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:09:03.911209 28177 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:03.969355 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:03.907495 28082 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:03.908958 28241 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:09:03.911171 28033 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:09:03.984704 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 142648us
W20260709 13:09:03.944115 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 102747us
I20260709 13:09:03.874524 28076 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.901157 27598 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:03.973809 28240 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:03.908533 28157 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:03.994624 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 85226us
I20260709 13:09:03.990882 28129 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:03.948254 28223 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.011623 21217 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:09:03.994427 28033 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:04.045410 28033 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.046638 28033 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:04.021137 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:04.021219 28242 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:09:04.021505 28157 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:04.047855 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.023355 28076 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.025177 27598 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.025725 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:03.992609 28082 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:04.027791 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:04.032114 28129 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.051208 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:04.053129 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:04.009743 28155 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:04.048197 21797 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 10 to 14, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:04.056747 27598 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:04.057298 28157 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:09:04.184914 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 127578us
I20260709 13:09:04.186748 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:04.094326 28082 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:04.049216 28153 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:09:04.100876 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47822.2us
I20260709 13:09:04.044220 28233 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.105185 28029 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:04.105175 28027 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:04.178206 28181 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.000s
I20260709 13:09:04.097856 28155 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:04.122644 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.240223 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:04.187763 21799 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:09:04.190564 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 993) took 1193 ms. Trace:
I20260709 13:09:04.242406 21144 rpcz_store.cc:276] 0709 13:09:02.997244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.997994 (+   750us) service_pool.cc:224] Handling call
0709 13:09:04.190314 (+1192320us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54002bbe54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:04.190494 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570706816}
I20260709 13:09:04.243171 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:04.245342 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:04.246088 21216 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:04.247591 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:04.167891 28245 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:09:04.248163 21219 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:04.248440 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54222.7us
W20260709 13:09:04.250830 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:04.226461 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 257208us
I20260709 13:09:04.224829 28076 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:09:04.265133 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:04.255227 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:04.297773 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:04.298833 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:04.301059 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:04.302052 21215 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:04.039258 28201 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.309351 21218 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:04.335464 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3126) took 1852 ms. Trace:
I20260709 13:09:04.336144 21064 rpcz_store.cc:276] 0709 13:09:02.483214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.107948 (+624734us) service_pool.cc:224] Handling call
0709 13:09:03.108245 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.215178 (+106933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:03.215210 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:03.215249 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:03.215316 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.335091 (+1119775us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540026e654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:04.335162 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.335378 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245420800}
I20260709 13:09:04.337517 21216 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:04.170346 28223 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.348033 21219 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:04.264811 28027 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.372195 28027 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.372499 28013 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.372994 28013 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:04.374001 28117 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.673s	user 0.005s	sys 0.005s
W20260709 13:09:04.384312 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 85560.6us
I20260709 13:09:04.376856 21217 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:04.426802 21142 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:09:04.264747 28157 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:04.429510 21066 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 20. Current term is 21. Ops: []
W20260709 13:09:04.430243 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 181072us
I20260709 13:09:04.430580 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:04.432544 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.428002 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:04.338588 28199 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:04.383302 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:04.394980 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:04.235426 28233 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.461848 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 3133) took 1039 ms. Trace:
I20260709 13:09:04.483122 21072 rpcz_store.cc:276] 0709 13:09:03.422206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.466446 (+ 44240us) service_pool.cc:224] Handling call
0709 13:09:04.461615 (+995169us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b540026eb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:04.461765 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126694144}
I20260709 13:09:04.346170 28082 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:04.463879 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:04.484859 21142 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:04.487385 28262 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.488029 28262 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.490415 28262 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.491084 28253 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:09:04.478483 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 50830.1us
I20260709 13:09:04.492708 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:04.503259 21219 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:04.466784 28013 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:04.468354 27965 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.519870 28262 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:04.520424 28200 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:04.520962 28082 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:04.482465 28157 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.374755 28155 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.536758 21217 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
W20260709 13:09:04.500583 27888 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.879s	user 0.013s	sys 0.000s
I20260709 13:09:04.525611 21139 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:09:04.525153 28131 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:04.529866 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:04.557485 21142 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
W20260709 13:09:04.536150 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:04.566540 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:04.536653 28215 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.919s	user 0.013s	sys 0.000s
W20260709 13:09:04.542932 28096 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.823s	user 0.010s	sys 0.002s
W20260709 13:09:04.574698 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.526916 28266 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:09:04.532796 27965 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.532260 28076 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:09:04.610675 27965 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.563870 27888 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:04.614642 27888 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:04.615705 21144 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:04.620764 28233 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:04.543135 28082 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:09:04.622570 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.622385 28082 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.632256 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 999) took 1431 ms. Trace:
I20260709 13:09:04.632870 21143 rpcz_store.cc:276] 0709 13:09:03.200647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.200760 (+   113us) service_pool.cc:224] Handling call
0709 13:09:03.201061 (+   301us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:03.251106 (+ 50045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:03.251136 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:03.251750 (+   614us) log.cc:844] Serialized 52 byte log entry
0709 13:09:04.041885 (+790135us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b180043fc00. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:09:04.334358 (+292473us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:04.334418 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.334520 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:04.600212 (+265692us) raft_consensus.cc:1685] finished
0709 13:09:04.600244 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.632162 (+ 31918us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66155,"raft.queue_time_us":51105,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":26841,"spinlock_wait_cycles":19221248,"thread_start_us":292168,"threads_started":1,"wal-append.queue_time_us":1252990}
I20260709 13:09:04.567515 28155 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:04.534260 28200 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:04.649184 21793 raft_consensus.cc:1275] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:04.566154 28131 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.593983 28215 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:04.641076 28096 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:04.661388 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:04.664978 21072 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
W20260709 13:09:04.680171 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 391370us
I20260709 13:09:04.684444 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:04.681608 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:04.681910 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:04.685833 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:04.649070 28201 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.686474 21067 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:09:04.688529 21765 leader_election.cc:400] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22.
I20260709 13:09:04.691478 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:04.651552 28277 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:04.698253 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:04.693388 21765 leader_election.cc:403] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:04.699081 21071 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:09:04.698292 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:04.704988 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:04.705070 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:04.705745 21793 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:09:04.699751 21072 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 10 to 14, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:04.707331 21035 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:04.711685 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:04.712706 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:04.645711 28253 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:04.719787 21071 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:09:04.720317 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 197172us
I20260709 13:09:04.744973 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:04.749814 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:04.750555 21216 raft_consensus.cc:2393] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:04.694125 28157 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.749575 28254 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:04.748323 21767 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:04.762550 28033 consensus_meta.cc:287] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.715s	user 0.004s	sys 0.008s
I20260709 13:09:04.763564 28253 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:04.767894 28283 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:04.779523 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 125187us
W20260709 13:09:04.773892 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 274281us
W20260709 13:09:04.710877 28025 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.854s	user 0.009s	sys 0.006s
I20260709 13:09:04.782459 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:04.783892 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:04.686949 28096 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:04.787608 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 80418.4us
I20260709 13:09:04.783071 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:04.788658 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:04.797453 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52607us
W20260709 13:09:04.775233 28143 consensus_meta.cc:287] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.810s	user 0.013s	sys 0.003s
I20260709 13:09:04.813251 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:04.806288 21068 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
W20260709 13:09:04.829299 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:04.811949 28222 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.902s	user 0.012s	sys 0.004s
I20260709 13:09:04.777725 28254 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:04.783495 28270 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:04.704312 28277 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:04.821048 27988 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:04.878377 21068 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:09:04.879956 21068 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:04.839766 28285 consensus_queue.cc:1048] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:09:04.850734 28082 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:04.822782 28271 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22.
I20260709 13:09:04.863533 28254 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:04.892534 28230 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.399s	user 0.011s	sys 0.012s
I20260709 13:09:04.862404 28270 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.874104 27988 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:04.903127 28096 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:04.903443 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 998) took 1718 ms. Trace:
I20260709 13:09:04.833905 28143 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:04.908373 28100 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:04.908707 27988 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:04.889309 28082 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:04.787750 28283 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.910447 21147 rpcz_store.cc:276] 0709 13:09:03.184535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.200314 (+ 15779us) service_pool.cc:224] Handling call
0709 13:09:03.200643 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:03.205072 (+  4429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:03.205101 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:03.231967 (+ 26866us) log.cc:844] Serialized 26 byte log entry
0709 13:09:03.334320 (+102353us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:03.334362 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.334431 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:04.891979 (+1557548us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:04.903110 (+ 11131us) raft_consensus.cc:1685] finished
0709 13:09:04.903132 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.903377 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4026,"raft.queue_time_us":4487,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2912,"spinlock_wait_cycles":5913856,"thread_start_us":101976,"threads_started":1,"wal-append.queue_time_us":237460}
I20260709 13:09:04.894122 28199 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:04.908605 28273 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:09:04.864001 28222 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.922382 28278 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:04.923372 28143 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:04.908643 28200 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:04.920468 28082 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:04.893563 28282 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:04.932701 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1273) took 2157 ms. Trace:
I20260709 13:09:04.937172 28283 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.938700 28270 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:04.942878 28215 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:04.943595 28131 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:04.918802 28276 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:09:04.919113 28096 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:04.966665 28122 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.967795 28277 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:04.961356 21069 rpcz_store.cc:276] 0709 13:09:02.774948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.152970 (+378022us) service_pool.cc:224] Handling call
0709 13:09:03.153369 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:03.229859 (+ 76490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.932432 (+1702573us) spinlock_profiling.cc:243] Waited 75.8 ms on lock 0x7b54003a2ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:04.932643 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174379264}
W20260709 13:09:04.991372 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.972780 28282 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.994503 28282 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:04.974888 28278 consensus_queue.cc:1243] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:09:04.995546 21141 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:04.997220 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1000) took 1204 ms. Trace:
I20260709 13:09:04.997587 21141 rpcz_store.cc:276] 0709 13:09:03.793274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.793443 (+   169us) service_pool.cc:224] Handling call
0709 13:09:03.842202 (+ 48759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:04.996001 (+1153799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.997039 (+  1038us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001ead54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:04.997172 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651934592}
I20260709 13:09:04.943527 27988 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.975802 21071 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:04.998375 28122 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:04.990777 28096 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.013495 28096 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:05.013139 28254 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.019486 21216 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:09:05.021971 28082 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.025478 28194 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.026278 28254 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.026538 27988 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.029906 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:05.038363 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.080250 28042 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:05.083636 28204 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.088320 28204 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.090197 28142 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:05.055635 28124 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.095616 28124 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.097407 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1284) took 1481 ms. Trace:
I20260709 13:09:05.097822 28124 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:05.041568 28100 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.099462 27948 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.099908 28100 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.099972 27948 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.102432 28100 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:05.104503 28130 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.104986 28130 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.107545 21138 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
W20260709 13:09:05.112046 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.097836 21066 rpcz_store.cc:276] 0709 13:09:03.615932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.740919 (+124987us) service_pool.cc:224] Handling call
0709 13:09:03.741194 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:04.481766 (+740572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.097134 (+615368us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b5400215554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:05.097339 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579862912}
I20260709 13:09:05.108565 28130 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.125270 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.126180 21792 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:05.102432 27948 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.090656 28204 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:05.119921 28122 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.082827 28201 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:05.140506 21801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 887) took 1706 ms (client timeout 1597 ms). Trace:
W20260709 13:09:05.142122 21801 rpcz_store.cc:269] 0709 13:09:03.433750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.433870 (+   120us) service_pool.cc:224] Handling call
0709 13:09:05.140218 (+1706348us) spinlock_profiling.cc:243] Waited 85.3 ms on lock 0x7b540046aad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:05.140397 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196142336}
I20260709 13:09:05.144604 28096 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.194079 28096 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.199066 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 996) took 2015 ms. Trace:
I20260709 13:09:05.199699 21146 rpcz_store.cc:276] 0709 13:09:03.183944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.185851 (+  1907us) service_pool.cc:224] Handling call
0709 13:09:03.186118 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.231238 (+ 45120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:03.231301 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:03.231345 (+    44us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:03.231380 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.198712 (+1967332us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b5400447ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:05.198797 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.198989 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103330176}
I20260709 13:09:05.176474 28251 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.180166 28131 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.205911 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:05.208851 21219 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:05.150012 28201 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.179854 28117 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.212582 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:05.213272 21212 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:05.223956 21212 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:05.228724 28201 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:05.230422 28259 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.708s	user 0.000s	sys 0.009s
I20260709 13:09:05.214823 28117 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.237569 28117 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.211488 28131 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.254161 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.243098 28131 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.209061 28251 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.260066 21215 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:05.255267 21219 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
W20260709 13:09:05.268805 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.271517 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.260191 28130 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:09:05.268429 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 383167us
I20260709 13:09:05.295873 28130 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:05.297745 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.299036 28130 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.304529 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.305539 21215 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:05.306339 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.309173 21217 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:05.307830 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.313953 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.259694 27948 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.316102 28192 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.317094 28240 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:05.331732 28294 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.123s	user 0.001s	sys 0.001s
I20260709 13:09:05.321903 27948 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.425891 28117 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.427805 28117 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.431216 27948 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.433123 28117 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.434194 28117 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:09:05.396397 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.007s	sys 0.039s
W20260709 13:09:05.435798 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.008s	sys 0.040s
I20260709 13:09:05.437724 28125 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:09:05.438349 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 625182us
W20260709 13:09:05.397836 21065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.000s	sys 0.015s
W20260709 13:09:05.260406 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 428978us
I20260709 13:09:05.283197 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:05.442723 27948 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:09:05.402284 21140 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.015s	sys 0.026s
W20260709 13:09:05.443733 21140 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.015s	sys 0.027s
I20260709 13:09:05.445469 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3539) took 1076 ms. Trace:
W20260709 13:09:05.410161 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 91200.8us
I20260709 13:09:05.448462 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:05.442699 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 159474us
I20260709 13:09:05.446063 21140 rpcz_store.cc:276] 0709 13:09:04.368866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.369016 (+   150us) service_pool.cc:224] Handling call
0709 13:09:04.369484 (+   468us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:04.384473 (+ 14989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:04.384519 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:04.428016 (+ 43497us) log.cc:844] Serialized 52 byte log entry
0709 13:09:04.646144 (+218128us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:04.827168 (+181024us) log.cc:844] Serialized 14 byte log entry
0709 13:09:05.444632 (+617464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.444826 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:05.444947 (+   121us) raft_consensus.cc:1685] finished
0709 13:09:05.444974 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.445289 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45379,"prepare.queue_time_us":285324,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":110023,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":13889,"thread_start_us":932654,"threads_started":3,"wal-append.queue_time_us":313959}
W20260709 13:09:05.401365 21113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.018s	sys 0.028s
W20260709 13:09:05.460397 21113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.018s	sys 0.029s
I20260709 13:09:05.463249 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.468670 27948 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:05.476677 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:05.477010 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.479867 27973 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:05.473533 27948 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.482741 21145 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:05.484148 27966 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.484347 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.485054 27966 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:05.485486 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3542) took 1090 ms. Trace:
I20260709 13:09:05.486418 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.466449 27988 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.487161 28001 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:05.487668 28001 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:05.487763 27988 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.486518 21145 rpcz_store.cc:276] 0709 13:09:04.394618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.394748 (+   130us) service_pool.cc:224] Handling call
0709 13:09:05.485180 (+1090432us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540032c3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:05.485365 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2313720576}
I20260709 13:09:05.489225 28001 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:05.464272 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.508003 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.508687 21110 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:05.438419 28125 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:05.512588 28125 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.513962 28125 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:05.516014 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:09:05.517516 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 66174.4us
I20260709 13:09:05.493081 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.526582 21111 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:05.501351 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.499106 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.530737 21221 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:05.503474 21214 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:09:05.531852 28108 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:05.532832 28108 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.535364 28077 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:05.536006 28130 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:05.536697 28129 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.537083 28130 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:05.537581 28129 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.501741 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.504287 28096 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:05.536059 28077 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.540547 28096 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.504536 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:05.541069 28077 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:05.541191 28096 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:05.541538 28143 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.542351 28143 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.545091 28082 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.545413 28143 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.546197 28082 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.547659 28130 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.538911 28144 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.549261 28122 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.549599 28108 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.547039 28082 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.550364 28108 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.550153 28122 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.503944 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:05.551693 21796 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:05.506994 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:05.518963 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 890) took 1722 ms. Trace:
I20260709 13:09:05.552525 28082 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:05.540771 21138 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3536) took 2145 ms (client timeout 1978 ms). Trace:
I20260709 13:09:05.552839 21800 rpcz_store.cc:276] 0709 13:09:03.795989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.796708 (+   719us) service_pool.cc:224] Handling call
0709 13:09:03.796984 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.864083 (+ 67099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:03.864114 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:03.864155 (+    41us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:03.864192 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.518403 (+1654211us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b5400070154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:05.518506 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.518750 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153833344}
I20260709 13:09:05.458869 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.555598 28100 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.555860 21213 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:05.522656 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:05.556453 28100 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.557403 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.558758 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:05.559561 28117 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.560237 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:05.561293 28122 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:05.562378 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:05.519464 20963 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 11 to 20, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 20 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:05.549026 28144 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.551308 28108 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:05.553231 21138 rpcz_store.cc:269] 0709 13:09:03.395317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.399403 (+  4086us) service_pool.cc:224] Handling call
0709 13:09:05.540341 (+2140938us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54003be554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:05.540524 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3925660032}
I20260709 13:09:05.524608 27966 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
W20260709 13:09:05.575806 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49480.7us
I20260709 13:09:05.563009 27598 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:05.583761 27598 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.583917 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:05.584307 27598 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:05.590293 27888 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:05.331946 28192 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.493537 28001 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:05.528146 28041 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.592461 28041 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.593765 28192 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:05.333660 28240 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.597921 28240 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:05.598031 21797 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:05.357548 28251 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.595201 28041 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.601037 28083 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.601902 28083 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.603641 28201 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.604290 28129 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.604399 28201 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.509708 28097 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:05.518901 27973 consensus_queue.cc:1175] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "46c1676121b64b7d9bcf2227cc639dd3" responder_term: 20 status { last_received { term: 19 index: 14 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:05.492118 27988 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:05.504999 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:05.589207 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 80797.5us
I20260709 13:09:05.589423 21145 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 23: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:05.347355 28270 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.660758 28201 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.661413 28201 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.661530 28083 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.662237 28083 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.662117 28201 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.663569 28097 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.664635 28253 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.665405 28253 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.667011 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:05.562527 28117 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:05.659142 28208 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:05.659751 28135 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:05.662984 28270 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.601496 28100 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:05.666152 28097 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.686100 28144 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.687245 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:05.687855 28222 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.688109 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:05.679956 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:05.680809 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:05.663000 28083 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.692888 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:05.718142 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:05.691732 28208 raft_consensus.cc:697] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:05.728964 28135 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:05.729800 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.729993 21794 raft_consensus.cc:2368] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:09:05.764765 21796 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:09:05.719050 21067 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:05.745368 20963 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 15 to 20, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 20 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:05.747430 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:05.750540 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1286) took 2042 ms. Trace:
I20260709 13:09:05.752264 21138 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:05.755545 28065 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:05.757319 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.780635 21800 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:05.759358 28208 consensus_queue.cc:237] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.781905 21063 rpcz_store.cc:276] 0709 13:09:03.708434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.741398 (+ 32964us) service_pool.cc:224] Handling call
0709 13:09:03.741663 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:05.027868 (+1286205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:05.027909 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:05.028441 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:09:05.398502 (+370061us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:05.398677 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.449770 (+ 51093us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400215054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:05.449847 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:05.750013 (+300166us) raft_consensus.cc:1685] finished
0709 13:09:05.750037 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.750338 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3633,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":35074,"spinlock_wait_cycles":2957050240,"thread_start_us":369082,"threads_started":1,"wal-append.queue_time_us":718725}
I20260709 13:09:05.809085 28316 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:05.809481 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.810075 28316 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.814879 28316 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.811203 28117 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.816606 28117 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.817615 28252 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:05.825475 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:05.706161 28013 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:05.793715 21142 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:05.847915 21140 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:05.732996 28100 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:05.792594 28253 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.818432 28252 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.852385 28082 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.869385 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:05.871449 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:05.877559 28290 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.877941 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:05.878352 28290 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.877815 20963 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 10 to 17. New cstate: current_term: 17 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:05.879441 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:05.880689 28290 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.880048 20963 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 14 to 17, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 17 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:05.882287 28264 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:09:05.885406 28264 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.887784 28013 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:05.888643 28013 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.892336 27847 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:05.892946 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:05.893149 27847 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:05.907949 21065 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:05.908347 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:05.909878 28013 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.919580 28083 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.919973 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.923419 28042 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:05.882635 20963 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 18, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 18 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:05.926549 20963 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 11 to 13, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 13 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:05.922843 28290 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.907899 28262 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:05.930291 28290 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.859205 27598 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.919661 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.878984 21072 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:05.897490 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 291173us
I20260709 13:09:05.923339 28082 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:05.854562 28144 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.896878 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:05.877733 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 159170us
I20260709 13:09:05.928805 20963 catalog_manager.cc:5697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 19. New cstate: current_term: 19 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:05.970194 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 363169us
I20260709 13:09:05.973798 21138 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:05.928828 28135 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:05.971834 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:05.980002 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:09:05.984686 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 88029.4us
I20260709 13:09:05.967634 20963 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index -1 to 10, term changed from 12 to 19, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 19 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:05.983006 28201 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:06.001361 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 86630.4us
I20260709 13:09:06.000592 21142 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:05.984864 27847 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:05.998451 28117 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.022084 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:06.022440 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:06.024331 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:06.026105 28257 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:06.026822 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.027418 28142 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:06.028228 28142 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.030550 28142 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.035665 27598 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.036496 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.037472 21068 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:09:06.039753 28312 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 13:09:06.040119 20963 catalog_manager.cc:5697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 15 to 23, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 23 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:06.041009 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:06.042291 20963 catalog_manager.cc:5697] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 8 to 16, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 16 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:06.045261 20963 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:06.051960 20963 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index -1 to 11, term changed from 14 to 18, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:06.048969 28290 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:05.996842 28144 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:06.054095 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:06.154675 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:06.083130 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75886.2us
I20260709 13:09:06.154202 21795 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:06.156062 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.078166 21072 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:06.024971 21138 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
W20260709 13:09:06.182672 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 59016.3us
I20260709 13:09:06.296581 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:06.520434 21139 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:09:06.120002 21064 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
I20260709 13:09:06.322265 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:06.525147 21142 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 16.
W20260709 13:09:06.317269 28077 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.775s	user 0.006s	sys 0.009s
I20260709 13:09:06.523626 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:06.224862 20963 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 15 to 18, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 18 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:06.528457 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 521175us
I20260709 13:09:06.525455 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.532096 21064 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:06.533643 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.534439 21064 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:06.526564 28077 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:06.235913 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.130240 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:06.156842 21110 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:06.551254 21799 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:09:06.273851 20968 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 225) took 73025 ms (client timeout 15000 ms). Trace:
I20260709 13:09:06.275102 21794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 1395) took 1245 ms. Trace:
I20260709 13:09:06.081677 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:06.347779 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 137924us
I20260709 13:09:06.154448 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.209837 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:06.555088 21794 rpcz_store.cc:276] 0709 13:09:05.029265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.029571 (+   306us) service_pool.cc:224] Handling call
0709 13:09:06.274686 (+1245115us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54005d5f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:06.275015 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1368551168}
W20260709 13:09:06.082252 28325 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:09:06.573607 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:06.574507 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44490.1us
I20260709 13:09:06.499964 28252 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.020159 28108 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.573673 27531 raft_consensus.cc:2764] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:09:06.592820 28144 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.275574 28260 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.593448 27531 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:06.594167 28260 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:06.579149 28320 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:06.596740 28260 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.182013 27853 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.465036 28061 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.597615 28061 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.598886 27853 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.600188 28061 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:06.054425 27847 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.601142 27847 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:06.602198 27853 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.602797 21065 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:06.587913 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 896) took 1126 ms. Trace:
I20260709 13:09:06.603701 21035 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:06.156848 28135 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:09:06.081677 28314 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:09:06.092948 28065 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.496s
I20260709 13:09:06.125916 28082 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.589200 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.218389 28041 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:06.589949 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 535876us
I20260709 13:09:06.590147 28075 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:06.592087 21141 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 10.
I20260709 13:09:06.596235 28320 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.605688 21066 raft_consensus.cc:2410] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:09:06.586613 27774 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:06.612046 21140 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.703s	user 0.008s	sys 0.008s
I20260709 13:09:06.593561 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:06.424495 28258 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:06.630606 21140 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:06.617643 21064 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:09:06.618096 21068 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:06.614920 27847 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:06.622586 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:06.625450 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:06.603664 21793 rpcz_store.cc:276] 0709 13:09:05.461843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.471159 (+  9316us) service_pool.cc:224] Handling call
0709 13:09:05.471815 (+   656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:05.693899 (+222084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:05.693934 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:05.713036 (+ 19102us) log.cc:844] Serialized 26 byte log entry
0709 13:09:05.986936 (+273900us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:05.986988 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.987077 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:06.587555 (+600478us) raft_consensus.cc:1685] finished
0709 13:09:06.587578 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:06.587719 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14338,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":234,"thread_start_us":375605,"threads_started":2,"wal-append.queue_time_us":788072}
I20260709 13:09:06.594501 27531 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:06.296403 28303 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.218530 28117 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.398s
W20260709 13:09:06.633419 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 66487.5us
I20260709 13:09:06.639402 27774 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.645061 28207 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:06.645869 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:06.646152 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.651226 21072 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:06.626673 28075 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.636382 28135 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:09:06.681849 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.683145 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:06.569893 28229 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:06.686087 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.686446 27531 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:06.686970 21070 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:06.687667 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:06.688369 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 119770us
I20260709 13:09:06.653734 28075 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:06.689700 28155 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:06.690325 28303 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:06.691601 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.692531 21072 raft_consensus.cc:2410] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:09:06.693552 21797 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:06.640358 27853 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:09:06.694954 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1423) took 1198 ms. Trace:
I20260709 13:09:06.654815 28207 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:06.696347 28207 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:06.633836 28041 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.695477 21801 rpcz_store.cc:276] 0709 13:09:05.496363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.498688 (+  2325us) service_pool.cc:224] Handling call
0709 13:09:05.499205 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:05.562569 (+ 63364us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:09:05.699180 (+136611us) log.cc:844] Serialized 14 byte log entry
0709 13:09:05.851487 (+152307us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:05.851562 (+    75us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:05.856464 (+  4902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:06.343971 (+487507us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b1800354fa0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:09:06.694675 (+350704us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54003bb854. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:06.694740 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:06.694896 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138488,"prepare.queue_time_us":301701,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"raft.queue_time_us":138627,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5813,"spinlock_wait_cycles":150796032,"thread_start_us":132458,"threads_started":1,"wal-append.queue_time_us":16484}
I20260709 13:09:06.699491 28279 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:06.699752 28135 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.700134 28279 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.700583 28041 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.696972 27853 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:06.026824 28257 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.654703 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:06.648150 28082 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:09:06.683641 28001 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.079s	user 0.008s	sys 0.008s
W20260709 13:09:06.708802 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 552087us
W20260709 13:09:06.683914 28125 consensus_meta.cc:287] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.169s	user 0.016s	sys 0.004s
I20260709 13:09:06.648550 28065 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:06.706765 21143 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
W20260709 13:09:06.706954 28096 consensus_meta.cc:287] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.165s	user 0.005s	sys 0.013s
I20260709 13:09:06.708202 28148 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:06.717760 21765 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:06.704608 27853 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:06.723094 21801 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:06.725924 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:06.704536 28258 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.646919 27892 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
W20260709 13:09:06.554796 20968 rpcz_store.cc:269] 0709 13:07:53.157306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.157712 (+   406us) service_pool.cc:224] Handling call
0709 13:09:06.182248 (+73024536us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:58.056215 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:58.056931 (+   716us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:58.057004 (+    73us) write_op.cc:435] Acquired schema lock
0709 13:08:58.057028 (+    24us) tablet.cc:662] Decoding operations
0709 13:08:58.057668 (+   640us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:58.057758 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:58.057785 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:08:58.058325 (+   540us) tablet.cc:701] Row locks acquired
0709 13:08:58.058368 (+    43us) write_op.cc:260] PREPARE: finished
0709 13:08:58.058528 (+   160us) write_op.cc:270] Start()
0709 13:08:58.058680 (+   152us) write_op.cc:276] Timestamp: P: 1783602538058463 usec, L: 0
0709 13:08:58.058719 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:08:58.126761 (+ 68042us) log.cc:844] Serialized 1713 byte log entry
0709 13:08:58.280278 (+153517us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f3f4120ac 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b
0709 13:08:58.628905 (+348627us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b1800314c20. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:08:58.915067 (+286162us) op_driver.cc:464] REPLICATION: finished
0709 13:08:59.092356 (+177289us) write_op.cc:301] APPLY: starting
0709 13:08:59.106118 (+ 13762us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:59.155090 (+ 48972us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:59.155126 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:59.217875 (+ 62749us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:59.218231 (+   356us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:08:59.218283 (+    52us) write_op.cc:312] APPLY: finished
0709 13:08:59.218624 (+   341us) log.cc:844] Serialized 49 byte log entry
0709 13:08:59.218828 (+   204us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:59.218932 (+   104us) write_op.cc:454] Released schema lock
0709 13:08:59.218989 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:59.266318 (+ 47329us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:59.267415 (+  1097us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30213,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":148024,"apply.run_cpu_time_us":8973,"apply.run_wall_time_us":145882,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"num_ops":4,"prepare.queue_time_us":330971,"prepare.run_cpu_time_us":43326,"prepare.run_wall_time_us":397072,"raft.queue_time_us":108634,"raft.run_cpu_time_us":53073,"raft.run_wall_time_us":111354,"replication_time_us":856064,"spinlock_wait_cycles":10206464,"thread_start_us":515637,"threads_started":4,"wal-append.queue_time_us":295131,"wal-append.run_cpu_time_us":19529,"wal-append.run_wall_time_us":1623974}]]}
I20260709 13:09:06.727490 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:06.727810 21798 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:06.737454 28177 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:06.736475 21797 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:06.715572 28284 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.717083 28082 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:06.719326 28041 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.715646 28065 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:06.690354 28155 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.724779 28252 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:06.728037 28001 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:06.747443 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:06.748589 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:06.752724 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:06.736157 28148 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:06.722651 28096 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.743837 28284 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:06.766433 27774 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.767192 28290 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:06.747632 28001 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.768003 28001 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:06.767920 28290 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.768471 28082 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.770359 28290 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.770961 28252 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.744896 28065 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:06.779580 28255 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:06.779934 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:06.780999 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 1008) took 1116 ms. Trace:
I20260709 13:09:06.781338 21146 rpcz_store.cc:276] 0709 13:09:05.665072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.688862 (+ 23790us) service_pool.cc:224] Handling call
0709 13:09:06.779565 (+1090703us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:09:06.780911 (+  1346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241266944}
I20260709 13:09:06.782092 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:06.782785 21069 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:06.758543 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:06.761336 28279 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.753597 21801 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:06.723191 28229 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.790290 21799 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:06.784260 28255 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:09:06.826812 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:06.858769 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:06.807595 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 57375.4us
W20260709 13:09:06.827711 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 83021.4us
W20260709 13:09:06.841365 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 86157us
I20260709 13:09:06.823673 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.788509 28077 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.785682 28096 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.867777 21069 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:06.867642 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.818320 28229 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.817160 28155 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.816897 28257 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.868230 28065 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.870187 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:06.815197 28332 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:09:06.875445 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 2287) took 1147 ms. Trace:
I20260709 13:09:06.806483 28255 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:06.880342 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:06.876075 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:06.881075 21070 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:06.881292 21799 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:06.876868 21065 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:06.806159 28041 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:06.879042 21064 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:09:06.880146 21069 rpcz_store.cc:276] 0709 13:09:05.728448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.849787 (+121339us) service_pool.cc:224] Handling call
0709 13:09:06.781826 (+932039us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:09:06.875387 (+ 93561us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2484992}
I20260709 13:09:06.796401 28290 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:06.825353 28148 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.825582 28303 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.874284 28117 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.843211 28258 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.891598 21071 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:06.901697 28065 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:06.904558 28041 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:06.875103 28207 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.909181 28207 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.910008 21070 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:09:06.873005 28075 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.917752 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.919153 21070 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:06.825551 27853 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.897578 28320 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:06.922734 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:06.893698 28117 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.929865 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.931237 21070 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:09:06.944712 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:06.945557 21799 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:06.915513 28290 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:06.944583 27853 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:06.928267 28279 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.951030 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:06.955117 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64159.2us
W20260709 13:09:06.953670 28318 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.163s	user 0.001s	sys 0.004s
W20260709 13:09:06.956127 21795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1413) took 3294 ms (client timeout 1895 ms). Trace:
I20260709 13:09:06.956663 21801 raft_consensus.cc:2393] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:09:06.956650 21795 rpcz_store.cc:269] 0709 13:09:03.661746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.668981 (+  7235us) service_pool.cc:224] Handling call
0709 13:09:06.954017 (+3285036us) spinlock_profiling.cc:243] Waited 2.37 s on lock 0x7b54004b7dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:06.956038 (+  2021us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5460974976}
I20260709 13:09:06.956820 21767 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:06.959201 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:06.980791 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:06.953737 28075 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:07.005067 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:07.005285 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:07.000743 28255 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.000062 28272 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:07.010893 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.991515 28144 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:07.013092 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:07.013964 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:06.954805 28279 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.020628 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:07.024307 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 2288) took 1168 ms. Trace:
I20260709 13:09:07.027856 21063 rpcz_store.cc:276] 0709 13:09:05.847224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.852643 (+  5419us) service_pool.cc:224] Handling call
0709 13:09:07.010713 (+1158070us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b5800003398. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:09:07.015832 (+  5119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12751232}
I20260709 13:09:07.028908 21071 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:07.014918 28075 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:07.037751 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.020126 28144 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.043184 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.037540 28075 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.043951 28075 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:07.045621 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.048668 28279 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:07.052290 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:07.052423 21068 raft_consensus.cc:2393] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:07.053393 21069 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:07.053570 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.054893 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.035622 28108 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:07.021692 28272 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:07.059551 28108 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:07.059988 28061 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:07.060499 28061 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:07.060297 28272 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.061537 28061 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.062561 28061 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:07.063822 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 105831us
I20260709 13:09:07.064625 28255 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:07.055194 28144 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:07.065030 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:07.057437 21219 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:07.011909 28001 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.065125 21798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1425) took 1564 ms. Trace:
I20260709 13:09:07.069108 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.072840 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.074958 21217 raft_consensus.cc:2410] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:09:07.076872 21212 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 13:09:07.071113 21798 rpcz_store.cc:276] 0709 13:09:05.500313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.502503 (+  2190us) service_pool.cc:224] Handling call
0709 13:09:05.503035 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:06.736439 (+1233404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:07.064880 (+328441us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54002eaed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:07.065063 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2804044032}
I20260709 13:09:07.078235 21214 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 23: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:07.079607 21219 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:09:07.060024 28108 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:07.093060 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.093418 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.104156 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:07.104759 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.160902 21140 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:07.166656 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1014) took 1475 ms. Trace:
I20260709 13:09:07.167321 21145 rpcz_store.cc:276] 0709 13:09:05.691776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.691946 (+   170us) service_pool.cc:224] Handling call
0709 13:09:07.066619 (+1374673us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:09:07.072121 (+  5502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:07.107305 (+ 35184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:07.166275 (+ 58970us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:07.166333 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:07.166390 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:07.166465 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:07.166592 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35966720}
W20260709 13:09:07.148346 21213 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.003s	sys 0.006s
W20260709 13:09:07.168988 21213 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.003s	sys 0.007s
I20260709 13:09:07.160668 21143 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:07.170471 21144 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:07.169737 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.185741 21217 raft_consensus.cc:2393] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:07.185315 21214 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
W20260709 13:09:07.169881 21187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.003s	sys 0.007s
W20260709 13:09:07.187741 21187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.003s	sys 0.007s
W20260709 13:09:07.188239 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 596184us
I20260709 13:09:07.195477 21214 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:09:07.198127 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.199093 21139 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:09:07.201164 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.200263 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.204803 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 2298) took 1051 ms. Trace:
I20260709 13:09:07.206449 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.204879 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.208603 21140 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 25.
I20260709 13:09:07.211047 21143 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:07.205343 21066 rpcz_store.cc:276] 0709 13:09:06.153850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:06.154015 (+   165us) service_pool.cc:224] Handling call
0709 13:09:07.204593 (+1050578us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b54002acbd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:07.204740 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102644480}
W20260709 13:09:07.214069 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 286789us
I20260709 13:09:07.217446 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:07.219210 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.215164 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:07.221889 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.223173 21214 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:07.224243 21185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.030s	sys 0.055s
W20260709 13:09:07.224819 21185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.030s	sys 0.055s
W20260709 13:09:07.226050 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.207360 21146 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
W20260709 13:09:07.228586 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 197678us
W20260709 13:09:07.239488 21215 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.004s	sys 0.023s
W20260709 13:09:07.240764 21215 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.005s	sys 0.023s
I20260709 13:09:07.244377 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:07.246430 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.247215 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:07.247345 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 182326us
W20260709 13:09:07.219367 28264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.001s	sys 0.022s
I20260709 13:09:07.252918 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.253011 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.249142 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:07.220011 28290 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:07.255484 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:07.404500 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 112366us
W20260709 13:09:07.396669 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.257539 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.407558 21145 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
W20260709 13:09:07.407205 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.369416 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:07.293514 21066 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:09:07.267452 21186 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.000s	sys 0.007s
I20260709 13:09:07.293833 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:09:07.417052 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.222817 28082 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:07.255847 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.293745 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.420468 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2358) took 1237 ms. Trace:
I20260709 13:09:07.408658 28290 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:07.287637 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.414929 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.255484 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:07.526348 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:07.597146 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 75674.6us
I20260709 13:09:07.530843 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:07.605122 21143 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:07.524217 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:07.608229 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:07.609594 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.610142 21145 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:07.590364 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:07.511580 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.525779 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:07.510308 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.416857 21068 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:09:07.589028 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 337812us
I20260709 13:09:07.616103 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:07.416072 21186 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.004s	sys 0.010s
W20260709 13:09:07.255785 28264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.002s	sys 0.022s
I20260709 13:09:07.614269 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:07.271396 28258 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:07.404501 28335 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.359s
I20260709 13:09:07.417639 28117 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:07.421113 21215 rpcz_store.cc:276] 0709 13:09:06.218872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:06.230937 (+ 12065us) service_pool.cc:224] Handling call
0709 13:09:06.554262 (+323325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:06.591197 (+ 36935us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:07.246662 (+655465us) log.cc:844] Serialized 14 byte log entry
0709 13:09:07.419732 (+173070us) log.cc:844] Serialized 14 byte log entry
0709 13:09:07.420102 (+   370us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:07.420165 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:07.420209 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:07.420277 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:07.420386 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":165,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":152,"thread_start_us":676446,"threads_started":2}
I20260709 13:09:07.628165 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:07.631031 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.393642 28041 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.633953 28176 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:07.634271 27378 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:07.636142 28343 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:09:07.637830 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3549) took 1910 ms. Trace:
I20260709 13:09:07.638278 21142 rpcz_store.cc:276] 0709 13:09:05.727715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:06.569082 (+841367us) service_pool.cc:224] Handling call
0709 13:09:06.625655 (+ 56573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:06.648994 (+ 23339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:06.649027 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:06.710055 (+ 61028us) log.cc:844] Serialized 26 byte log entry
0709 13:09:07.206681 (+496626us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:07.206725 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:07.206792 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:07.637597 (+430805us) raft_consensus.cc:1685] finished
0709 13:09:07.637619 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:07.637764 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3257,"raft.queue_time_us":13636,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":9096,"thread_start_us":456344,"threads_started":1,"wal-append.queue_time_us":549497}
I20260709 13:09:07.639734 28252 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:07.640264 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.589258 28082 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:07.641464 28082 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:07.641618 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:07.635160 21187 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:07.636071 28233 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:07.645804 28233 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:07.646891 28233 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.648608 28233 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:07.653558 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.654083 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1012) took 1987 ms. Trace:
I20260709 13:09:07.658897 28334 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.659660 28334 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.660214 28347 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:07.660869 28347 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:07.662025 28334 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:07.662611 28334 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:07.662720 28284 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:07.605607 28122 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.253521 28207 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.416s
I20260709 13:09:07.654568 21147 rpcz_store.cc:276] 0709 13:09:05.666698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.691192 (+ 24494us) service_pool.cc:224] Handling call
0709 13:09:06.524806 (+833614us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b58001f3b98. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:09:06.524991 (+   185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:06.751975 (+226984us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:09:07.185495 (+433520us) log.cc:844] Serialized 14 byte log entry
0709 13:09:07.254241 (+ 68746us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:07.254583 (+   342us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:07.255079 (+   496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:07.653031 (+397952us) spinlock_profiling.cc:243] Waited 48.1 ms on lock 0x7b180034a6e0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3fd2a6 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419
0709 13:09:07.653693 (+   662us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540027a454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:07.653743 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:07.653911 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32888,"prepare.queue_time_us":883499,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":2293,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":27351,"spinlock_wait_cycles":650658560,"thread_start_us":456704,"threads_started":2,"wal-append.queue_time_us":412655}
I20260709 13:09:07.663026 28334 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.673352 28334 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:07.663146 28347 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:07.667866 28344 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.231s	user 0.000s	sys 0.003s
I20260709 13:09:07.675756 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:07.485888 28320 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.421512 28290 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:07.661504 28100 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:07.670079 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:07.632808 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45331.2us
I20260709 13:09:07.393608 27847 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:07.686818 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:07.533823 28181 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:07.691659 28100 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.696664 28100 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:07.700166 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:07.700459 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:07.700886 21797 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:07.701144 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:07.701903 21800 raft_consensus.cc:2368] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20.
I20260709 13:09:07.702564 27847 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.634053 28041 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:07.705325 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:07.705966 21797 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
W20260709 13:09:07.706609 28075 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.662s	user 0.005s	sys 0.009s
W20260709 13:09:07.664489 28333 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.943s	user 0.000s	sys 0.006s
I20260709 13:09:07.707123 27847 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:07.688500 28122 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:07.686939 28117 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:07.708254 28117 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.709573 28122 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:07.707090 28075 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.710270 28117 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:07.711800 28075 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:07.713155 28061 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.650s	user 0.005s	sys 0.011s
I20260709 13:09:07.713764 21794 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:07.715999 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:07.705344 28041 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:07.717494 27957 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:07.720536 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:07.723024 28001 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:07.724076 28108 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.001s
I20260709 13:09:07.724570 28108 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:07.725764 28082 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:07.728034 28082 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:07.629122 28258 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:07.729439 28258 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:07.730571 21142 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:07.730083 28100 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.731626 28108 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:07.733377 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:07.730232 21144 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.002s
I20260709 13:09:07.736467 21144 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 24.
I20260709 13:09:07.732772 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:07.742151 21796 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:07.742893 28290 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:07.743997 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 218325us
I20260709 13:09:07.745203 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:07.687090 21146 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:07.740482 21765 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:07.767057 21143 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:07.768343 28001 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:07.768855 28001 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:07.769286 28001 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:07.766485 28061 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:07.771970 28061 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.772426 28338 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:07.773398 21141 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:09:07.773866 21141 pending_rounds.cc:85] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 7
I20260709 13:09:07.774266 21141 pending_rounds.cc:107] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:09:07.774667 21141 raft_consensus.cc:2889] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:07.776657 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:07.692041 28181 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.742836 28100 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:07.778681 21147 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:07.787930 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2304) took 1632 ms. Trace:
I20260709 13:09:07.788401 21067 rpcz_store.cc:276] 0709 13:09:06.155693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:06.604650 (+448957us) service_pool.cc:224] Handling call
0709 13:09:06.619817 (+ 15167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:06.630748 (+ 10931us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:07.129467 (+498719us) log.cc:844] Serialized 14 byte log entry
0709 13:09:07.786946 (+657479us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b180028aec0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3a9c0a 00007f7f48ae9219 00007f7f3f431a40 00007f7f3f42c4e4 00007f7f3f42c455 00007f7f42514060 00007f7f42526716 00007f7f42537287 00007f7f42537208
0709 13:09:07.787417 (+   471us) log.cc:844] Serialized 14 byte log entry
0709 13:09:07.787589 (+   172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:07.787635 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:07.787686 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:07.787753 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:07.787871 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97040,"prepare.queue_time_us":700600,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"spinlock_wait_cycles":45119360,"thread_start_us":505746,"threads_started":2,"wal-append.queue_time_us":476963}
I20260709 13:09:07.783917 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:07.792912 21068 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:07.743660 28041 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:07.798887 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:09:07.802196 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.779682 21765 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:07.805804 28355 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:07.772423 28061 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:07.807353 21214 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.541s	user 0.015s	sys 0.003s
I20260709 13:09:07.812330 21214 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
W20260709 13:09:07.812661 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 193118us
I20260709 13:09:07.811342 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:07.808066 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:07.794021 28148 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:07.791711 28181 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:07.801101 28100 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.802454 28041 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:07.816489 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:07.800179 27957 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:07.810621 28236 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:07.740305 28320 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.821321 28338 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:07.822000 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:07.826515 21796 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:07.849464 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:07.812446 28096 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:07.820489 28177 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:09:07.819824 28264 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:07.827440 28076 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:09:07.863148 28290 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:07.864128 28290 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:07.864490 28290 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:07.865468 28290 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.866684 28290 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:07.809052 28355 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:07.867699 28355 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:07.869212 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.832908 27957 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.833060 28083 raft_consensus.cc:1027] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: attempt to promote peer b70fc04de37e4239a8336534f0c9c0fb: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:09:07.839454 27853 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:07.852825 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:07.822229 28252 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:09:07.833320 28041 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:07.813958 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:07.841536 28236 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.851922 28338 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:07.853047 28075 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:07.859566 28342 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:07.856081 28207 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:07.888571 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 74908.9us
I20260709 13:09:07.881800 27957 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:07.910537 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:07.899431 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 264425us
I20260709 13:09:07.847586 28100 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:07.912307 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55191.8us
I20260709 13:09:07.891958 28236 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:07.860985 28096 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.893919 28338 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:07.897909 28207 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.893167 28076 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:09:07.928671 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:07.928654 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 98958.4us
I20260709 13:09:07.881907 28148 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:09:07.916950 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:09:07.901412 28272 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.009s	sys 0.015s
I20260709 13:09:07.941215 27378 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:07.893021 27853 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:07.908553 28075 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:07.909368 28176 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:07.956722 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:07.952661 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 703411us
I20260709 13:09:07.964627 21146 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
W20260709 13:09:07.964908 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:07.952574 28272 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.009s	sys 0.016s
I20260709 13:09:07.946728 28207 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:07.970907 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 159628us
I20260709 13:09:07.949019 28076 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:07.970427 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:07.988945 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:07.989841 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50290.3us
I20260709 13:09:07.953522 28135 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:07.959431 28364 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:07.992574 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:07.961866 28351 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:07.995651 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:07.999454 28351 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.001797 28351 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:07.995446 27853 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.998533 28359 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:07.998428 28364 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:07.973663 28284 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:07.992741 28076 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:08.030387 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:08.011667 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 94807us
I20260709 13:09:08.033491 21065 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:08.033721 21184 leader_election.cc:400] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17.
I20260709 13:09:08.034704 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:08.037385 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 75402us
I20260709 13:09:08.042526 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:08.042963 28364 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.008493 28359 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:08.043736 28359 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:08.045028 28342 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:08.052493 28335 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.053117 28335 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.056798 28122 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.057636 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.058302 21215 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:08.058426 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:08.059161 21214 raft_consensus.cc:2393] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:08.059490 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.060159 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.059425 28075 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.030772 27892 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:08.061350 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.061734 27892 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:08.062738 21217 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:08.063638 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.063082 27892 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.064395 28025 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:08.064786 28025 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:08.064879 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.065680 28025 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.066731 28025 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:08.066097 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.067401 28264 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:08.068158 28264 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:08.069340 28264 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:08.067456 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 459985us
I20260709 13:09:08.072350 28100 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.073468 28359 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.075177 28347 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:08.076716 28320 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.078629 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.080138 21217 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:08.079483 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.079051 28334 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.083139 28334 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.003144 28355 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:08.086002 28355 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.085886 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:08.087893 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.088933 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.089084 21072 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:08.088934 21066 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:08.090204 21798 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:08.032302 28135 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.057480 28122 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.087996 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.095708 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.084108 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.100800 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:08.101617 21143 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:08.102572 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.103112 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.103581 21063 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
W20260709 13:09:08.104156 28361 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:09:08.104627 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.105327 21063 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:08.105471 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3548) took 2391 ms. Trace:
I20260709 13:09:08.105337 28082 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.105985 21138 rpcz_store.cc:276] 0709 13:09:05.714047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:06.568273 (+854226us) service_pool.cc:224] Handling call
0709 13:09:06.603665 (+ 35392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:06.603981 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:06.604003 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:07.640890 (+1036887us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b2c00010828. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:09:07.641340 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:09:07.901947 (+260607us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:07.902003 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:07.902104 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:08.105179 (+203075us) raft_consensus.cc:1685] finished
0709 13:09:08.105198 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:08.105413 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30097792,"thread_start_us":260198,"threads_started":1,"wal-append.queue_time_us":299219}
I20260709 13:09:08.106938 28082 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:08.108007 28360 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:09:08.108722 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.103132 28122 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.110018 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.110556 21069 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:09:08.102504 28355 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:08.111610 28355 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.111984 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.112669 21072 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:08.113283 28370 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:08.113659 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.113852 28355 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.113909 28370 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:08.009248 28284 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:08.110051 28347 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:08.121491 28347 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:08.121302 28370 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.122421 28347 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.115439 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:08.120757 28355 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:08.097757 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:08.123899 27378 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:08.126405 27378 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.126739 27378 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:08.129415 28347 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:08.144516 28373 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:08.145505 28373 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.149391 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:08.152498 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 77194.6us
I20260709 13:09:08.152900 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:08.153512 21145 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:08.154407 28258 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.155561 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.097499 28335 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.156833 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.157555 21072 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:08.158042 28041 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.158349 28368 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:08.158804 28320 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:08.159116 21142 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:08.160367 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.160715 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.161021 21212 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:08.161340 21216 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:09:07.989938 28365 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.149s
I20260709 13:09:08.165925 28082 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.166211 28365 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:08.166608 28365 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:08.167567 28365 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.168718 28359 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.169087 28365 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:08.170820 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.171617 21066 raft_consensus.cc:2368] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20.
I20260709 13:09:08.173031 28176 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:08.173357 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.173597 28176 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.174320 21214 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:09:08.175374 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.175648 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:08.176383 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.176972 21215 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:08.177959 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.178220 21071 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:08.172662 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.173451 21140 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:08.179368 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.181295 21113 leader_election.cc:400] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20.
I20260709 13:09:08.181689 21113 leader_election.cc:403] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:09:08.183768 21138 raft_consensus.cc:2368] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:08.123818 28284 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:08.192050 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.194396 21144 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
W20260709 13:09:08.195772 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 225083us
I20260709 13:09:08.191007 27378 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.208061 21069 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:08.209367 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.200584 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:08.210004 21069 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:08.199065 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:08.210819 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.217939 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:08.218691 21794 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:08.220801 27378 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:08.197814 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1390711599999577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 13:09:08.221325 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1237711600000466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 13:09:08.221730 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0373711600000206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
I20260709 13:09:08.158394 28135 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.223994 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.189308 28177 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20.
I20260709 13:09:08.224861 21066 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:08.231422 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.211541 28356 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:08.232084 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:08.219484 28082 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.152122 28338 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.234568 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.219893 28334 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:08.252822 28334 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.253351 28334 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:08.254212 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.254845 21071 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:08.256268 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.257079 21142 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:08.245000 28335 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:08.258857 28335 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:08.260445 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.260114 28335 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.265722 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:08.185730 28001 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:08.274900 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.274917 21035 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:08.277346 28135 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:08.277805 28135 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.278187 28135 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:08.280721 28181 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.285063 28207 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.286206 28236 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.287601 28176 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.287331 28135 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.288775 28061 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.289278 28001 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.291128 28135 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.296407 28356 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.308182 28207 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.299010 28356 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:08.243600 28144 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.178s	user 0.019s	sys 0.004s
I20260709 13:09:08.325408 28144 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.329139 28373 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.329664 28144 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.330564 21070 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:08.330885 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.331238 28284 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:08.332072 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.242792 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.332599 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.332989 28025 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:08.333195 21066 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:08.333573 28025 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.334144 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.247786 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:08.336037 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 94899.7us
I20260709 13:09:08.336263 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.337180 21065 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:08.337435 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:08.337601 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:08.247749 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.338261 21065 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:08.338578 21143 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:08.338776 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:08.250051 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.339394 21141 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:08.339794 21138 raft_consensus.cc:2410] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:09:08.340036 28041 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.340793 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.341461 21138 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:09:08.250478 21147 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:09:08.342796 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:08.343850 28355 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:08.344277 28355 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:08.344753 28355 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:08.344846 28096 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:08.345089 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 364645us
I20260709 13:09:08.345522 28096 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:09:08.346958 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.346632 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:08.347671 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:08.239013 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.348453 28320 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:08.251366 28082 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.349514 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:08.349843 21072 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:08.283079 28181 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.350988 28082 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.349766 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:08.351716 28236 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.351894 21071 raft_consensus.cc:2410] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 21 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:09:08.352643 28122 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:08.353055 28122 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.353412 28122 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:08.159026 28368 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.355480 28359 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:08.355629 21068 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:08.356068 28359 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.356491 28359 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:08.240091 28233 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:08.324925 28083 raft_consensus.cc:1027] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: attempt to promote peer b70fc04de37e4239a8336534f0c9c0fb: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:09:08.327447 28258 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.348914 28320 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:08.358389 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.323930 21213 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:09:08.335274 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.347024 28096 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:08.361235 21145 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:08.372674 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.235258 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.365664 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:08.432349 21072 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:08.397835 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.430713 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:08.433473 21068 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:09:08.393515 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.435810 21065 raft_consensus.cc:1275] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:09:08.376825 21141 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:08.332644 21070 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:08.392987 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:08.439495 21140 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:08.438261 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:08.344168 28338 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.440526 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.448168 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:08.450001 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57141.8us
I20260709 13:09:08.450371 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.350123 28364 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:08.291119 28061 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:08.392520 28290 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.525s	user 0.003s	sys 0.010s
W20260709 13:09:08.444204 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 78609.9us
W20260709 13:09:08.455904 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 463380us
I20260709 13:09:08.460958 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:08.408564 28320 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:08.398783 21066 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:08.255048 28076 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.392729 28369 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:08.444535 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.355449 28368 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.466266 21068 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:08.482777 28338 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:08.483518 28338 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.483956 28338 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:08.484002 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:08.485301 28376 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:08.485932 28376 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:08.490156 28364 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:08.490904 28364 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:08.491401 21110 leader_election.cc:400] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20.
I20260709 13:09:08.492107 28364 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.493175 21110 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:08.494199 27378 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:08.494642 27378 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:08.494997 27378 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:08.495637 21145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:09:08.496572 21145 raft_consensus.cc:606] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:08.496992 21145 raft_consensus.cc:614] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:08.500917 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:09:08.486861 28376 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.501674 21144 raft_consensus.cc:606] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:08.507930 21140 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:09:08.493247 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.464849 21218 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:08.513057 21218 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:08.469444 21113 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:08.468580 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:08.466620 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:08.473575 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:09:08.520588 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54035.4us
W20260709 13:09:08.521327 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 94504.7us
I20260709 13:09:08.522214 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:08.392694 28365 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:09:08.526748 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:08.453575 28207 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:08.534299 21072 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:08.527951 21139 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:08.536062 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:08.401122 28233 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.538532 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:08.512254 28369 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:08.493314 28284 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:09:08.533079 28368 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:09:08.549301 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 111147us
I20260709 13:09:08.542726 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:08.556252 21796 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
W20260709 13:09:08.544481 27957 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.001s
I20260709 13:09:08.558400 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:08.531651 27378 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.523509 28025 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.524453 21218 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:08.554854 21141 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
W20260709 13:09:08.555264 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.472438 28350 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:09:08.541064 28258 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:08.543042 28096 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:08.543603 28335 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:08.596252 21218 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:08.655622 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:08.559391 21765 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.565s (SENT) [suppressed 6 similar messages]
I20260709 13:09:08.608276 21800 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 24: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:08.649506 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:08.554296 28369 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:08.646332 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 79875.6us
W20260709 13:09:08.665251 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 195807us
I20260709 13:09:08.543897 28347 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:09:08.558444 28368 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:08.669672 21070 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:08.597246 27957 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.597029 28041 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:08.678015 27378 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:08.683065 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:08.552743 28122 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.600793 28258 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:08.562992 27847 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:08.689595 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:08.556269 28181 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.119s
I20260709 13:09:08.696545 27378 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:08.698745 21146 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:08.699878 27378 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.700510 27378 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:08.680169 28368 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:08.702797 21796 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:08.707945 21215 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:08.711725 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:08.712518 21215 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:08.682889 28041 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:08.714190 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.695248 28258 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:08.724237 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:08.690285 28207 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:08.725881 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:08.746316 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.679816 28334 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:08.742077 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 182713us
I20260709 13:09:08.741667 27378 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:08.726509 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 77090us
I20260709 13:09:08.725225 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:08.749645 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.757083 27378 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.759467 21221 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 23: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:08.756937 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:08.761139 21216 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:08.761665 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:08.768889 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:08.770663 21070 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:09:08.725106 28025 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:08.727119 28041 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:08.833262 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.840214 21220 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:08.814145 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:08.841516 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:08.874790 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60767us
W20260709 13:09:08.858574 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 163740us
I20260709 13:09:08.895798 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:08.880693 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:08.908473 21765 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:08.855988 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:08.794531 28284 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.097s
I20260709 13:09:08.926290 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:08.876423 28076 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.930824 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:08.936512 21801 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:08.874325 28122 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.939838 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:08.941972 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:08.930464 28386 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:09:08.916826 28233 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.927767 28379 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:08.931457 28082 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.938287 27957 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.961433 27957 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:08.966802 28355 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.622s	user 0.002s	sys 0.006s
I20260709 13:09:08.974010 28320 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:08.976230 28334 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:08.974813 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:08.939005 28207 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.977744 21799 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 23: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:08.980378 28082 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:08.982667 28082 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:08.938061 28366 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:08.992321 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:08.993376 21800 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:08.955210 28233 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:09:08.955474 28385 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 13:09:08.955850 28387 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:09:08.956178 28400 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:08.956596 28382 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 13:09:09.011972 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 293930us
I20260709 13:09:08.957582 28350 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:08.958297 28371 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:09:08.958755 28384 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
W20260709 13:09:09.017273 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 255662us
I20260709 13:09:09.004591 21215 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:08.945216 28376 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:09.005837 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 63971.9us
I20260709 13:09:09.004722 28233 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.017454 21035 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:09.010941 28338 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.001s
W20260709 13:09:09.027236 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 492549us
W20260709 13:09:08.957418 28393 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:09:08.960338 28390 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:09:09.013335 28335 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:09.031746 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.037590 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:08.992472 28366 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.040371 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.041981 21144 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:09.020123 28371 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:09.005306 28355 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.029963 28233 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:09.055198 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.059900 21140 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 23: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:09.056444 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.061141 21141 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:09.033859 28096 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:09.074908 21801 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:09.078006 21035 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:09.085219 28350 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:09.085386 28355 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:09.086015 28350 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:09.086710 21065 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:09.053236 28371 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:09.090795 21144 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 24.
I20260709 13:09:09.032930 28338 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.074229 28096 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:09.102182 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69817.5us
W20260709 13:09:09.162495 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 222746us
W20260709 13:09:09.162567 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.164515 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.164723 21113 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:09.147794 28402 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:09:09.171478 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2327) took 1021 ms. Trace:
I20260709 13:09:09.139756 28096 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:09.172650 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:09.178921 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 429504us
I20260709 13:09:09.173843 21071 rpcz_store.cc:276] 0709 13:09:08.150314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.353058 (+202744us) service_pool.cc:224] Handling call
0709 13:09:08.353362 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:08.419120 (+ 65758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:08.419165 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:08.460685 (+ 41520us) log.cc:844] Serialized 26 byte log entry
0709 13:09:08.596133 (+135448us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:08.596175 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:08.596267 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:09.170464 (+574197us) raft_consensus.cc:1685] finished
0709 13:09:09.171220 (+   756us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:09.171415 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20064,"raft.queue_time_us":76664,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4956,"thread_start_us":114231,"threads_started":1,"wal-append.queue_time_us":284907}
I20260709 13:09:09.183380 21187 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:09.177066 28359 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.820s	user 0.013s	sys 0.000s
I20260709 13:09:09.185285 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:09.091886 28350 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.191231 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:09.103847 28408 consensus_queue.cc:1059] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:09:09.177951 28082 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:09.204733 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:09.204974 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.211066 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:09.211313 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.211930 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.212867 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:09.214210 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:09.190833 28335 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:09.215399 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:09.223362 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:09.224324 21801 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 22: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:09.218321 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.210986 28025 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:09.226346 28025 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:09.226894 28025 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:09.225894 21071 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:09.219050 28338 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:09.218326 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:09.229563 21799 raft_consensus.cc:2368] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 23. Current term is 24.
I20260709 13:09:09.191588 28359 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.231094 28416 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:09.232080 28359 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:09.216151 28342 consensus_queue.cc:1048] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:09:09.233078 28404 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:09.233484 28404 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:09.234269 28404 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.202543 28258 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.235471 28404 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:09.235597 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.236240 21064 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:09.232402 28368 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.002s
I20260709 13:09:09.203362 28355 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:09.243746 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.244514 21070 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 22: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:09.201184 28347 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:09.189461 28320 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:09.242920 21065 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:09.259173 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:09.259526 28420 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:09.238871 28368 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.261444 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.214342 28388 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:09.265462 21767 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:09.269946 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 497051us
I20260709 13:09:09.250439 21064 raft_consensus.cc:2410] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:09:09.250190 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:09.254627 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.239751 28391 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:09.273097 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:09.246973 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.266552 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.267858 21066 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:09:09.187414 28407 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:09:09.274582 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.275251 21140 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:09.245844 28347 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:09.260950 28416 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:09.269583 28082 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:09.275203 28236 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:09.276681 28075 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:09.298388 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.275419 28391 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.299918 28236 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:09.302843 28391 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:09.302968 28423 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.303272 28236 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:09.303640 28423 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:09.299225 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52349.3us
I20260709 13:09:09.306910 21071 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:09.305908 28423 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.307646 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.308284 28423 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:09.308377 21215 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:09.307983 28025 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.282138 28347 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:09.313724 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.330720 21140 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:09.331341 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.313592 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.320982 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:09.333720 21069 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:09.335016 21036 leader_election.cc:400] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:09.335449 21036 leader_election.cc:403] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:09.335208 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.294229 28366 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:09.337919 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.339619 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:09.339879 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.336181 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.339920 21144 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:09.338291 21216 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:09.336665 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.341296 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.300213 28075 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:09.297835 28264 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:09:09.346789 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:09.337198 28369 consensus_meta.cc:287] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.000s
I20260709 13:09:09.307682 28417 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:09.338433 28378 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:09.349112 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.353744 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.362123 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:09.349942 28403 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:09:09.346626 28075 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:09.379632 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.349859 28258 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.336210 28233 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.380368 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.351876 28378 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:09.380373 21219 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:09.348157 28421 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:09.351069 28368 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:09.350386 28416 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.387841 28368 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:09.390213 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:09.391043 21064 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:09.348145 28425 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:09.399097 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.352087 28369 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:09.397311 28041 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 13:09:09.386122 28378 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:09.361851 28096 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:09.402189 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 129147us
I20260709 13:09:09.407979 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.408299 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.413923 21064 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:09.410989 21068 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:09.412477 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:09.421450 28364 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:09.421949 28364 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:09.422384 28364 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:09.424109 28025 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:09.409844 28411 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:09:09.431804 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.407234 28041 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.426152 28369 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.446288 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.447012 21217 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:09.447168 21145 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:09.448030 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.448721 28391 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:09.449409 28405 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.245s	user 0.003s	sys 0.000s
I20260709 13:09:09.414469 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.450018 28041 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:09.452263 28371 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.453299 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.500597 21219 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 24.
I20260709 13:09:09.454613 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.449286 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.503974 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.463132 21064 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:09.465054 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.506989 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:09.508074 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.508898 21064 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:09.523317 28236 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.524454 28096 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.525833 28236 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:09.494206 28366 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.527350 28148 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:09.527529 28428 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:09:09.528934 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 267564us
I20260709 13:09:09.530530 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.533545 21221 raft_consensus.cc:2368] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:09.532995 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.541499 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.533627 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.533071 21110 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:09.540500 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 125720us
I20260709 13:09:09.573545 21064 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:09.585214 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:09.542438 21795 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:09.548501 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.550365 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.583223 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:09.628556 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 45457us
W20260709 13:09:09.629730 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 81327.3us
I20260709 13:09:09.630677 21111 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.632508 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:09.633203 21064 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:09.635568 21216 raft_consensus.cc:2393] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:09:09.636091 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 103120us
I20260709 13:09:09.636888 21110 leader_election.cc:400] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 23. Current term is 24.
I20260709 13:09:09.637178 21110 leader_election.cc:403] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:09:09.528013 28148 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:09.643890 21142 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:09.644434 21143 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:09:09.646013 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.646695 21798 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:09:09.644730 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.647166 21184 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.648681 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.533994 28181 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:09.650537 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:09.502583 28075 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.654265 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.655238 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.641904 28379 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:09.663647 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:09.640484 28427 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:09:09.640208 28423 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:09.662056 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.641672 28407 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:09.641680 28411 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.657366 28425 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:09.665949 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:09.649672 28284 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:09.650758 28416 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 23. Current term is 24.
I20260709 13:09:09.526552 28366 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.658639 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.668891 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.670585 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.672469 21143 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:09.661639 28041 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:09.678455 21141 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:09:09.652561 28320 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:09.685312 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 350405us
I20260709 13:09:09.685432 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.683239 21064 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:09.689479 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:09.670799 28181 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:09.652932 28334 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:09:09.660837 28359 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:09:09.694236 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.694605 28424 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:09.695099 28412 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:09.699347 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.725211 21220 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:09:09.700374 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.694926 28437 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.694240 28013 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:09.712209 28440 raft_consensus.cc:2764] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 13:09:09.712364 28436 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:09.714094 28391 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.714483 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 965) took 1474 ms. Trace:
I20260709 13:09:09.678787 28379 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:09.716173 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:09.726436 21219 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:09.736574 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:09.739948 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 89694.5us
I20260709 13:09:09.688920 28041 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.756129 21218 rpcz_store.cc:276] 0709 13:09:08.239757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.239941 (+   184us) service_pool.cc:224] Handling call
0709 13:09:08.295715 (+ 55774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:09.019086 (+723371us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:09.019134 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:09.019193 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:09.019246 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.714250 (+695004us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400102f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:09.714298 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:09.714428 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3056,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":3989,"spinlock_wait_cycles":388600704}
I20260709 13:09:09.761615 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.742935 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:09.690457 28423 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:09.758268 21036 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.764091 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.673559 28425 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.717382 27853 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.769640 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.767961 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.681661 28411 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.770632 21220 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:09.775805 28347 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.765545 28440 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:09.777997 28440 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:09.778106 28347 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:09.778606 28365 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:09.778965 21215 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:09.779951 28379 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.774097 27853 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.785454 27853 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:09.786058 27853 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:09.779026 28365 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:09.760953 28436 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.787637 28365 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.788622 28365 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:09.788972 21794 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:09.790066 28442 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:09.790923 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1325) took 1303 ms. Trace:
W20260709 13:09:09.791077 21065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 3176) took 1358 ms (client timeout 1515 ms). Trace:
I20260709 13:09:09.791468 28442 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:09.791584 21065 rpcz_store.cc:269] 0709 13:09:08.433143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.450914 (+ 17771us) service_pool.cc:224] Handling call
0709 13:09:09.790930 (+1340016us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b540041d554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:09.791024 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1300960512}
I20260709 13:09:09.754204 28437 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.794205 28437 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.797986 21071 raft_consensus.cc:2393] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:09.777385 28411 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.798321 28364 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.800467 28320 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:09.800879 28320 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:09.801086 28378 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.801626 28320 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:09.772212 28404 consensus_meta.cc:287] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.007s
I20260709 13:09:09.803951 28371 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:09.804162 21140 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:09.805138 21796 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:09.805390 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:09.806120 21793 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:09.806216 28075 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:09.806677 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 1418) took 1058 ms. Trace:
I20260709 13:09:09.764531 28041 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:09.807202 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1419) took 1058 ms. Trace:
I20260709 13:09:09.820760 21064 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:09.816277 27378 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:09.840112 27378 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:09.827427 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:09.807056 21793 rpcz_store.cc:276] 0709 13:09:08.748545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.748901 (+   356us) service_pool.cc:224] Handling call
0709 13:09:09.805224 (+1056323us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:09:09.806608 (+  1384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67207424}
I20260709 13:09:09.840235 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.839159 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:09.842317 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:09.833477 21219 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:09.818164 21068 rpcz_store.cc:276] 0709 13:09:08.488001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.488311 (+   310us) service_pool.cc:224] Handling call
0709 13:09:08.488631 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:09.414746 (+926115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.790692 (+375946us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b54000dd254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:09.790871 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2120771456}
I20260709 13:09:09.838624 21071 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
W20260709 13:09:09.842485 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 183917us
I20260709 13:09:09.840569 21796 rpcz_store.cc:276] 0709 13:09:08.748862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.749012 (+   150us) service_pool.cc:224] Handling call
0709 13:09:09.649310 (+900298us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7a22 00007f7f48310c09 00007f7f4746aee0 00007f7f475340ab 00007f7f47534131 00007f7f474f5fc0 00007f7f474e0929 00007f7f474e03d2 00007f7f40a57bd4
0709 13:09:09.649516 (+   206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:09.805652 (+156136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.806998 (+  1346us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400700154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:09.807150 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679193600}
I20260709 13:09:09.846179 21069 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:09:09.768963 28369 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:09.856820 21219 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:09.849011 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.814903 28359 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.861212 27378 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.807528 28366 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.759655 28148 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.809088 28425 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:09.841181 28408 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.841317 28334 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.915504 28334 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.916972 28388 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:09.841545 28436 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.865252 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:09.867303 21219 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 13:09:09.921026 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60987.8us
I20260709 13:09:09.921406 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:09.922158 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:09.844439 28355 raft_consensus.cc:2764] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:09:09.874274 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:09.874606 27378 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:09.858702 28013 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:09.877436 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49338.6us
I20260709 13:09:09.904469 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.924727 28076 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:09.925915 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.926308 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.926959 21141 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 26: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:09.858968 28437 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:09.927757 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.928038 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.871115 28407 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:09.903010 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.928965 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.894658 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:09.929682 21147 raft_consensus.cc:2368] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24.
I20260709 13:09:09.929771 21799 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:09.930490 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.878342 28391 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:09.928544 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.928634 21138 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:09.866650 28320 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:09.932766 21138 raft_consensus.cc:1240] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:09.934053 21138 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:09:09.904675 28280 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:09:09.906267 28424 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:09.900177 28378 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:09.910461 21072 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:09:09.911199 21145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:09:09.935817 28424 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:09.936820 21145 raft_consensus.cc:606] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.937333 21145 raft_consensus.cc:614] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:09.938817 28424 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:09.779052 28440 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.955634 28436 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:09.957376 28424 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:09.957924 28424 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:09.918380 21218 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:09.960211 28177 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:09.961280 28177 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:09.961622 28177 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:09.960222 28424 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.962443 28177 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.963097 28424 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:09.868360 28100 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:09:09.875037 28359 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:09.929323 21142 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 22: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:09.938726 21138 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 25. Current term is 26. Ops: []
W20260709 13:09:09.902103 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 165595us
I20260709 13:09:09.941906 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.943001 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.970081 21145 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:09.943651 28407 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:09.977638 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55556.4us
I20260709 13:09:09.979329 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:09.977855 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.900687 28412 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:09:10.004703 28366 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:10.005316 28366 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:10.009857 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.945230 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:09.927834 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:09.963616 28177 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:09.912130 28364 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:09.917434 28388 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:10.018587 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:09.965910 28290 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:09.983943 28320 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:09:09.984277 28355 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:09.985395 28347 consensus_queue.cc:1059] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:09.987361 28437 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:09.988823 28334 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:09.989377 28439 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:09:09.968019 28391 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:10.013185 21143 raft_consensus.cc:2368] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:09:09.969388 21765 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:09.901185 28408 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:10.023427 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 78279.8us
I20260709 13:09:10.048144 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.030135 21218 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:10.048530 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3166) took 2021 ms. Trace:
I20260709 13:09:09.951978 21799 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:10.034403 21144 raft_consensus.cc:2410] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:09:10.006502 28366 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.036550 21145 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:10.038470 28388 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:09:10.039961 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56961us
W20260709 13:09:10.046562 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 77279.1us
I20260709 13:09:10.050082 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:10.050233 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.050765 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:10.009689 28365 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:09:10.048954 21067 rpcz_store.cc:276] 0709 13:09:08.027401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.273799 (+246398us) service_pool.cc:224] Handling call
0709 13:09:08.353612 (+ 79813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:08.436768 (+ 83156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:08.436819 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:08.523268 (+ 86449us) log.cc:844] Serialized 26 byte log entry
0709 13:09:08.985896 (+462628us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b4800080358. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f47445aa1 00007f7f4743cb37 00007f7f4743c9ca 00007f7f4753426f 00007f7f47533f37 00007f7f4754855c 00007f7f47548508 00007f7f47548488 00007f7f47548443 00007f7f475476ec 00007f7f46471a96
0709 13:09:09.011616 (+ 25720us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:09.011685 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.011921 (+   236us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:09.448811 (+436890us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:09.526813 (+ 78002us) raft_consensus.cc:1685] finished
0709 13:09:09.526839 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:10.048357 (+521518us) spinlock_profiling.cc:243] Waited 79.5 ms on lock 0x7b540026eb50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:10.048473 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65172,"raft.queue_time_us":76528,"raft.run_cpu_time_us":16141,"raft.run_wall_time_us":548505,"spinlock_wait_cycles":196561792,"thread_start_us":451174,"threads_started":1,"wal-append.queue_time_us":619651}
I20260709 13:09:10.052729 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.033689 28423 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:10.056171 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:10.057261 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:10.059862 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 241935us
I20260709 13:09:10.040766 28437 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:10.076826 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.064042 21797 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:10.067243 28359 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:10.080281 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.080796 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 1417) took 1332 ms. Trace:
I20260709 13:09:10.069204 28366 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:10.069968 28391 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:10.070582 28335 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:10.097286 28335 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:10.098428 28448 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 13:09:10.099833 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.098814 28335 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:10.101737 28409 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.924s	user 0.005s	sys 0.000s
I20260709 13:09:10.072490 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.102814 21141 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:10.104880 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:10.105564 21216 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 11.
I20260709 13:09:10.105798 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2393) took 1402 ms. Trace:
I20260709 13:09:10.106312 21214 rpcz_store.cc:276] 0709 13:09:08.703203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.703333 (+   130us) service_pool.cc:224] Handling call
0709 13:09:08.703644 (+   311us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:08.717136 (+ 13492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:08.717175 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:08.750341 (+ 33166us) log.cc:844] Serialized 52 byte log entry
0709 13:09:08.953900 (+203559us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:09.095102 (+141202us) log.cc:844] Serialized 14 byte log entry
0709 13:09:09.095378 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.095459 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:09.990136 (+894677us) spinlock_profiling.cc:243] Waited 5.22 ms on lock 0x7b18002c1400. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3fd2a6 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db
0709 13:09:10.101116 (+110980us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:10.105533 (+  4417us) raft_consensus.cc:1685] finished
0709 13:09:10.105554 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:10.105717 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1009,"prepare.queue_time_us":222323,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":865,"raft.queue_time_us":136527,"raft.run_cpu_time_us":459,"raft.run_wall_time_us":72068,"spinlock_wait_cycles":20983424,"thread_start_us":172154,"threads_started":2,"wal-append.queue_time_us":280220}
I20260709 13:09:10.108932 21214 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:10.112633 28364 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:10.113070 28364 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:10.114168 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2402) took 1406 ms. Trace:
I20260709 13:09:10.114319 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.115048 21144 raft_consensus.cc:2393] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:10.115825 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:10.076205 28408 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:10.116346 21144 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:10.117689 21070 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:10.119880 28359 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:10.120572 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:10.121021 28334 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:09:10.121237 21068 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:10.121398 28334 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:10.121519 28451 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:10.121968 28451 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:10.122248 28334 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:10.123548 28334 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:10.078478 21220 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:10.068492 28388 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:10.081360 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.048002 28452 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:10.084933 28320 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:10.131970 28359 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:10.132272 28320 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:10.133389 28320 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:10.080073 28437 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:10.081295 21797 rpcz_store.cc:276] 0709 13:09:08.748244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.749118 (+   874us) service_pool.cc:224] Handling call
0709 13:09:10.050649 (+1301531us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b5800174998. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7b71 00007f7f3f3f7eee 0000000000375939 00007f7f4746b1e0 00007f7f475340c4 00007f7f47534131 00007f7f474e20a0 00007f7f474e1984 00007f7f40a5c6d4 00007f7f40a5c662
0709 13:09:10.080713 (+ 30064us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19090816}
I20260709 13:09:10.113827 28364 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:10.123590 21147 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
W20260709 13:09:10.131261 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49966.1us
I20260709 13:09:10.084380 27892 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:10.136893 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:10.114594 21212 rpcz_store.cc:276] 0709 13:09:08.707793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.900115 (+192322us) service_pool.cc:224] Handling call
0709 13:09:08.924173 (+ 24058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:09.077027 (+152854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:09.077071 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:09.077127 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:09.077171 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:10.113972 (+1036801us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54004443d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:10.114013 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:10.114121 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350877952}
W20260709 13:09:10.198815 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 141639us
I20260709 13:09:10.060241 28207 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:10.074584 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:10.214620 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 77804.8us
I20260709 13:09:10.201215 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:10.104933 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:10.078056 21143 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:10.215786 21036 leader_election.cc:400] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24.
I20260709 13:09:10.216575 21036 leader_election.cc:403] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:09:10.220557 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:10.268869 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48557.8us
I20260709 13:09:10.270303 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:10.271174 21144 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:10.271618 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2338) took 1064 ms. Trace:
I20260709 13:09:10.272055 21063 rpcz_store.cc:276] 0709 13:09:09.206877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.207020 (+   143us) service_pool.cc:224] Handling call
0709 13:09:09.217898 (+ 10878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:09.246298 (+ 28400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:09.246346 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:09.246402 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:09.246453 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:10.244972 (+998519us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b54003ca354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:10.271425 (+ 26453us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:10.271557 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64692608}
I20260709 13:09:10.122704 28451 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:10.202286 28428 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:10.377591 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:10.379197 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 75729.1us
I20260709 13:09:10.388837 21798 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:10.379416 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:10.380198 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.218834 28350 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.222772 28290 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:10.386519 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 116343us
I20260709 13:09:10.404004 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:10.307070 21220 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:10.306085 28280 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24.
I20260709 13:09:10.405416 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:10.388103 21794 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:10.396376 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.221905 27892 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:10.400378 21797 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:10.407521 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:10.409207 21036 leader_election.cc:400] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:09:10.409554 21036 leader_election.cc:403] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:10.378271 28364 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:10.406818 21141 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:10.410990 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:10.412298 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:10.415946 27892 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.417908 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:10.418503 28144 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:10.419219 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.441370 21796 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:10.382220 28207 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:10.398757 28082 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:10.440063 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:10.407099 28041 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.007s
I20260709 13:09:10.463754 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:10.464059 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:10.442297 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56414.6us
I20260709 13:09:10.387617 28452 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:10.446231 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:10.467486 21141 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:10.308853 28334 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.127s
I20260709 13:09:10.380682 28420 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:10.464777 21799 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:10.408543 28440 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.465649 21219 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:10.443599 28233 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:10.452405 28451 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:10.493840 28456 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:09:10.499972 21146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3565) took 2404 ms (client timeout 2058 ms). Trace:
I20260709 13:09:10.499761 28207 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:10.500424 21146 rpcz_store.cc:269] 0709 13:09:08.095153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.102838 (+  7685us) service_pool.cc:224] Handling call
0709 13:09:10.499773 (+2396935us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54001ec3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:10.499904 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1367996672}
I20260709 13:09:10.501645 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.502228 21146 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:10.427978 28431 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:09:10.426995 28350 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.503885 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:10.504462 28440 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.505563 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.505868 28350 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:10.506698 28440 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:10.507069 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:10.512200 27378 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:10.512657 27378 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:10.513005 27378 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:10.463613 28260 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:10.451807 28280 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:09:10.481843 28373 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.463721 28041 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.489495 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:10.490972 28421 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:09:10.518154 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.503136 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:10.522015 21141 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 24.
I20260709 13:09:10.548872 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.527100 21213 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 11.
I20260709 13:09:10.529506 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.558668 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:10.559230 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.560339 21142 raft_consensus.cc:2368] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22.
W20260709 13:09:10.561415 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 152331us
I20260709 13:09:10.564225 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:10.567507 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:10.563495 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:10.569280 21213 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:10.570187 21146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c918605cdf4c39b832a695968308a3"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:09:10.572186 21146 raft_consensus.cc:606] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Received request to transfer leadership to TS 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:10.572646 21146 raft_consensus.cc:614] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:10.520396 28260 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.480526 28379 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.068s
I20260709 13:09:10.543535 28284 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:10.484236 28391 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:10.531569 28373 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.582185 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:10.592679 21212 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:10.569602 28424 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.606s	user 0.011s	sys 0.000s
I20260709 13:09:10.575973 28290 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:10.480829 28452 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.529181 28374 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:09:10.575762 27378 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.482127 28428 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.077s
W20260709 13:09:10.599753 28455 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:09:10.599565 28284 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:10.481961 28061 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.085s
I20260709 13:09:10.601588 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:10.605012 28177 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.002s
I20260709 13:09:10.577518 28260 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:10.601974 21138 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
W20260709 13:09:10.602850 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:10.603410 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.557209 28395 consensus_queue.cc:1059] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:09:10.584077 28392 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:10.600436 28440 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.580979 28401 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:10.633579 21215 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:09:10.563397 28421 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:10.612820 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.604555 28290 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:10.672971 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.636796 21066 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:09:10.638139 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.638567 21072 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:09:10.610431 21213 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:10.603549 28391 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.648461 28373 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:10.642716 28401 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:10.652326 28458 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:09:10.656774 28041 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:10.660364 28437 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:10.683511 28441 log.cc:927] Time spent T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.781s	user 0.003s	sys 0.005s
I20260709 13:09:10.683815 28284 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:10.630373 28417 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:10.685397 28417 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:10.688215 28417 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:10.688882 28391 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:10.689783 28437 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:10.690580 28452 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:10.668617 21212 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:10.627105 28347 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:10.670639 21145 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:10.691984 28347 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:10.694269 28347 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:10.695195 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:10.695851 21212 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:10.696069 28421 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:10.697116 28421 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:10.695231 21138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a473266451804da089da06e9c927e7ee"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:09:10.700390 21138 raft_consensus.cc:606] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.701606 21138 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:10.702064 21142 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:10.702716 21138 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:10.704227 21142 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:10.705426 21144 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 13:09:10.706403 21142 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:09:10.706918 21144 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:09:10.707497 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:09:10.711001 28449 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.800s	user 0.002s	sys 0.007s
I20260709 13:09:10.699090 21219 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:10.712621 21138 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:09:10.712978 21219 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:09:10.714632 21219 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:09:10.715468 21221 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:10.717195 21221 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:10.718808 21219 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 13:09:10.720294 21142 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:10.721992 28335 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:10.723641 21212 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:10.729313 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:10.731053 21767 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.731259 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:10.732164 21064 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:09:10.733457 28428 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:10.734328 28428 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:10.735584 28428 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:10.736645 28379 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:10.737147 28379 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:10.737335 28437 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:10.738306 28379 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.636385 21063 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:09:10.635003 27378 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:10.676769 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:10.660765 28420 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:10.681007 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:10.787719 21072 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:10.791890 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:10.792557 28453 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.805s	user 0.004s	sys 0.006s
I20260709 13:09:10.794559 21801 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:10.794920 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2345) took 1437 ms. Trace:
I20260709 13:09:10.795217 28452 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:09:10.795356 21065 rpcz_store.cc:276] 0709 13:09:09.357798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.792655 (+434857us) service_pool.cc:224] Handling call
0709 13:09:09.792968 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:09.793337 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:09.793376 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:09.793842 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:09:09.867971 (+ 74129us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:09.868032 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.870067 (+  2035us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:10.791842 (+921775us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:10.794624 (+  2782us) raft_consensus.cc:1685] finished
0709 13:09:10.794648 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:10.794861 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1170,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":69101952,"thread_start_us":69694,"threads_started":1,"wal-append.queue_time_us":105857}
I20260709 13:09:10.682696 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:10.798170 21066 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:10.691442 21220 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:10.791114 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:10.749511 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:10.762884 21184 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:10.747025 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:10.739087 21068 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:10.746308 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:10.747773 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3576) took 1413 ms. Trace:
I20260709 13:09:10.638643 28440 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.742161 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.753579 28401 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:10.750487 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.771104 21794 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:10.808063 21140 rpcz_store.cc:276] 0709 13:09:09.334397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.335956 (+  1559us) service_pool.cc:224] Handling call
0709 13:09:10.747589 (+1411633us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54002e4354. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:10.747701 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066080000}
I20260709 13:09:10.772332 21798 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:10.760118 21147 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:09:10.812079 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:10.812520 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 70470.4us
I20260709 13:09:10.813807 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:10.675554 28290 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:10.815435 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 137757us
I20260709 13:09:10.815665 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:10.816797 21797 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:10.817935 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:10.818740 21217 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:09:10.818879 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:10.670665 21146 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:10.820533 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.668819 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:10.821241 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:10.781325 21213 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
I20260709 13:09:10.786765 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:10.823717 21144 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:10.785894 28144 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:10.808185 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.801163 21064 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:10.809630 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:10.747948 21071 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:10.826004 21800 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:09:10.826763 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 64015.7us
I20260709 13:09:10.828811 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:10.822630 21063 raft_consensus.cc:2376] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24. Re-sending same reply.
I20260709 13:09:10.801640 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:10.831907 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:10.828159 28412 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:10.838866 21187 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:10.840593 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.832728 21070 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:10.838752 21035 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:10.811080 28082 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:10.808698 28100 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:10.838269 28061 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:10.847728 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:10.825688 28117 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:10.835534 28446 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:10.852520 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:10.822531 28148 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.854403 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3388) took 1080 ms. Trace:
I20260709 13:09:10.854529 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.826838 28418 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.856506 27378 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:10.867202 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:10.852205 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:10.868706 21069 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:10.869830 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.857287 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:10.870190 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:10.862181 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:10.846698 28280 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:09:10.866768 21216 rpcz_store.cc:276] 0709 13:09:09.773635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.106178 (+332543us) service_pool.cc:224] Handling call
0709 13:09:10.106498 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:10.106817 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.106845 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:10.140636 (+ 33791us) log.cc:844] Serialized 26 byte log entry
0709 13:09:10.433902 (+293266us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:10.839435 (+405533us) log.cc:844] Serialized 14 byte log entry
0709 13:09:10.853966 (+ 14531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:10.854073 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:10.854134 (+    61us) raft_consensus.cc:1685] finished
0709 13:09:10.854152 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:10.854329 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14102,"prepare.queue_time_us":196215,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":15556,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":4410,"thread_start_us":536666,"threads_started":2,"wal-append.queue_time_us":375547}
I20260709 13:09:10.880188 28431 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:09:10.881100 28445 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.972s	user 0.005s	sys 0.009s
I20260709 13:09:10.863366 28076 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:09:10.863162 28350 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:09:10.883172 28350 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:10.883602 28350 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:10.884577 28350 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.885998 28350 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:10.861326 28117 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:10.879063 28400 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:10.855752 28347 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:10.888218 27853 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:10.879557 28412 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:10.879372 28418 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:10.892753 28418 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:10.893946 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:10.894997 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:10.895656 21069 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:09:10.898583 21069 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:10.899081 21069 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:10.902292 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:10.838517 28436 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 13:09:10.923194 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 417668us
I20260709 13:09:10.900301 21069 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.927268 21069 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:10.920949 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 86031.3us
W20260709 13:09:10.931723 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.845714 28355 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:09:10.933367 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:10.878216 28452 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:10.888053 28117 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:10.856755 28451 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 13:09:10.917156 28446 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:10.861753 28100 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:10.925508 28347 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:09:10.933583 28444 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.025s	user 0.000s	sys 0.011s
W20260709 13:09:11.010360 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 55514.1us
I20260709 13:09:10.876264 28148 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:11.024092 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.025866 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:11.029176 21216 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:09:11.030700 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:11.033325 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:10.967419 28061 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:11.053777 21217 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 11: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:11.020354 28347 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:11.047902 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.932761 28391 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 13:09:10.932147 28400 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 13:09:11.052239 21069 raft_consensus.cc:2410] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 25 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:09:11.019748 28452 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:11.047554 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 310883us
I20260709 13:09:11.088181 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:11.093071 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.093207 21066 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:11.095381 21036 leader_election.cc:400] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22.
I20260709 13:09:11.054881 28366 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.054958 28414 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:11.053246 28061 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.097016 21069 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:11.097548 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:11.098838 21066 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:11.115708 21069 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:11.116571 21069 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:11.097908 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 221218us
I20260709 13:09:11.075294 28347 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:11.104190 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:11.105685 21218 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:11.077346 27853 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:11.102770 27892 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.013s	sys 0.016s
I20260709 13:09:11.076099 28364 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 13:09:11.095762 28420 raft_consensus.cc:2764] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
W20260709 13:09:11.126833 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 231907us
I20260709 13:09:11.087497 28280 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:11.089681 28431 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:11.129139 21185 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:11.141059 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:11.111865 28284 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:11.125581 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:11.087682 28451 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 13:09:11.130023 27892 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.013s	sys 0.017s
I20260709 13:09:11.121407 28419 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:11.119830 28061 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:11.167714 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.124032 28233 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:11.127187 28395 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:11.166971 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:11.175251 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.139417 28464 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:11.114535 28366 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:11.217068 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 128978us
I20260709 13:09:11.222987 28100 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:11.119385 28388 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:09:11.173673 28355 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:11.119576 28414 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:11.232576 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65871.2us
W20260709 13:09:11.331274 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.400669 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:11.435258 21217 pending_rounds.cc:85] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 10
I20260709 13:09:11.435792 21217 pending_rounds.cc:107] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:09:11.436121 21217 raft_consensus.cc:2889] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:11.119999 28446 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:11.448058 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3498) took 2137 ms. Trace:
I20260709 13:09:11.223307 28391 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:11.164904 28280 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:11.535213 27378 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:11.544668 27378 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:11.535225 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:11.227043 28417 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.456846 21139 rpcz_store.cc:276] 0709 13:09:09.255155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.257140 (+  1985us) service_pool.cc:224] Handling call
0709 13:09:09.257486 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:09.364479 (+106993us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:09.364516 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:09.385802 (+ 21286us) log.cc:844] Serialized 52 byte log entry
0709 13:09:09.757019 (+371217us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:09.757055 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.757121 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:10.792676 (+1035555us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:11.196499 (+403823us) raft_consensus.cc:1685] finished
0709 13:09:11.196532 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.392904 (+196372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15969,"raft.queue_time_us":50225,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":108,"spinlock_wait_cycles":9575296,"thread_start_us":437624,"threads_started":2,"wal-append.queue_time_us":510739}
I20260709 13:09:11.392913 28355 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.171695 28468 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.338s
I20260709 13:09:11.557534 28401 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:11.428550 28395 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:11.560075 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 299164us
W20260709 13:09:11.680310 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.591073 28429 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:11.604954 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:11.681697 21065 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:11.682821 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:11.683789 21065 raft_consensus.cc:2410] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:09:11.607522 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:11.556528 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 415973us
I20260709 13:09:11.640456 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:11.685226 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:11.263324 28421 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:11.553504 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 149025us
I20260709 13:09:11.185355 28135 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.273s
I20260709 13:09:11.661701 21144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6beb666b5d504215a141add24a6d632f"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46800
W20260709 13:09:11.559161 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.455735 28427 raft_consensus.cc:3055] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:11.687723 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:11.734292 21144 raft_consensus.cc:606] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Received request to transfer leadership
I20260709 13:09:11.734711 21144 raft_consensus.cc:614] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:09:11.695665 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.732398 27378 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:11.663352 28469 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.698s	user 0.000s	sys 0.001s
I20260709 13:09:11.744537 21801 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:11.258035 28233 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:11.666785 28355 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:09:11.768064 28473 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:09:11.768452 28407 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.769044 28407 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:11.769737 28476 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.543s	user 0.000s	sys 0.000s
I20260709 13:09:11.759243 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46800 (request call id 3) took 1760 ms. Trace:
I20260709 13:09:11.224027 28452 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:11.778615 21144 rpcz_store.cc:276] 0709 13:09:09.975254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.661565 (+1686311us) service_pool.cc:224] Handling call
0709 13:09:11.735168 (+ 73603us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:11.733744 28421 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:11.798058 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.807775 21216 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:11.393942 27853 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:11.731950 28429 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:11.741873 28427 raft_consensus.cc:740] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:11.813397 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.830276 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:11.774124 28148 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:11.838852 21801 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 23. Current term is 24. Ops: []
W20260709 13:09:11.843799 21799 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.003s	sys 0.008s
I20260709 13:09:11.748610 28391 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 13:09:11.813043 28421 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:11.849911 21799 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.004s	sys 0.009s
W20260709 13:09:11.858868 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.006s	sys 0.010s
W20260709 13:09:11.860484 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.010s
I20260709 13:09:11.861932 21799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1456) took 1104 ms. Trace:
I20260709 13:09:11.664569 28417 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:11.856758 28407 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:11.676484 28431 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:11.858067 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:11.889478 21071 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:11.858243 28502 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:11.871726 28495 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:09:11.872694 28474 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:09:11.873569 28480 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 13:09:11.874873 28475 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:09:11.874747 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2368) took 1050 ms. Trace:
I20260709 13:09:11.877177 28082 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:11.877285 21067 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:11.877628 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3207) took 1099 ms. Trace:
I20260709 13:09:11.863541 21799 rpcz_store.cc:276] 0709 13:09:10.757599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.763784 (+  6185us) service_pool.cc:224] Handling call
0709 13:09:10.764213 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:10.764504 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:11.049682 (+285178us) log.cc:844] Serialized 14 byte log entry
0709 13:09:11.860820 (+811138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:11.861184 (+   364us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:11.861471 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.861619 (+   148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.861877 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":407338,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":70,"spinlock_wait_cycles":48000,"thread_start_us":1024501,"threads_started":3}
I20260709 13:09:11.881996 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:11.942502 21066 rpcz_store.cc:276] 0709 13:09:10.778066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.116534 (+338468us) service_pool.cc:224] Handling call
0709 13:09:11.116855 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.117051 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:09:11.542456 (+425405us) log.cc:844] Serialized 14 byte log entry
0709 13:09:11.876527 (+334071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:11.876662 (+   135us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:11.876845 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.877006 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.877448 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":749220,"threads_started":2}
W20260709 13:09:11.854015 28260 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.219s	user 0.007s	sys 0.006s
W20260709 13:09:11.892848 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 205227us
I20260709 13:09:11.850441 28177 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.895692 21801 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:11.894419 21795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.007s	sys 0.015s
I20260709 13:09:11.848939 28412 raft_consensus.cc:2764] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:09:11.913429 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:11.962041 21795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.007s	sys 0.015s
I20260709 13:09:11.918032 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:11.962652 21213 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:11.962714 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1457) took 1204 ms. Trace:
I20260709 13:09:11.858788 28148 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.925242 21221 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:09:11.821606 28499 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.895013 28013 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:11.934790 21070 rpcz_store.cc:276] 0709 13:09:10.824421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.873574 (+1049153us) service_pool.cc:224] Handling call
0709 13:09:11.873978 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.874273 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.874323 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:11.874392 (+    69us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:11.874437 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.874525 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.874676 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:11.891575 28401 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.960023 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:11.963845 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:11.963100 21795 rpcz_store.cc:276] 0709 13:09:10.757982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.765690 (+  7708us) service_pool.cc:224] Handling call
0709 13:09:10.766017 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:10.766245 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:09:11.087778 (+321533us) log.cc:844] Serialized 14 byte log entry
0709 13:09:11.962389 (+874611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:11.962442 (+    53us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:11.962495 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.962552 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.962660 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27792,"prepare.queue_time_us":457442,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"thread_start_us":1150777,"threads_started":3,"wal-append.queue_time_us":556403}
I20260709 13:09:11.978781 21213 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 17: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:11.965731 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:11.969327 21187 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:11.972575 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:11.856948 28491 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:09:11.966991 28100 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:11.952796 28407 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:11.987744 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1340) took 1158 ms. Trace:
I20260709 13:09:11.997833 21065 rpcz_store.cc:276] 0709 13:09:10.829688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.973583 (+1143895us) service_pool.cc:224] Handling call
0709 13:09:11.987229 (+ 13646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.987441 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.987469 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:11.987507 (+    38us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:11.987539 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.987588 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.987691 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:11.931178 28502 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:11.882107 28395 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:11.982805 28366 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:12.016000 28359 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:12.019811 21801 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:09:12.020370 21801 pending_rounds.cc:85] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 13
I20260709 13:09:12.020709 21801 pending_rounds.cc:107] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 21.14
I20260709 13:09:12.022809 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:12.021150 21801 raft_consensus.cc:2917] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Aborting config change with OpId 21.14: Aborted: Op aborted by new leader
I20260709 13:09:12.025115 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1342) took 1194 ms. Trace:
I20260709 13:09:12.025619 21065 rpcz_store.cc:276] 0709 13:09:10.830192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.024104 (+1193912us) service_pool.cc:224] Handling call
0709 13:09:12.024480 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:12.024699 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:12.024735 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:12.024804 (+    69us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:12.024859 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.024931 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.025053 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:12.024544 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.035513 21065 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:12.037514 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1344) took 1199 ms. Trace:
I20260709 13:09:12.038497 21065 rpcz_store.cc:276] 0709 13:09:10.837850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.032671 (+1194821us) service_pool.cc:224] Handling call
0709 13:09:12.034647 (+  1976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:12.036768 (+  2121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.037365 (+   597us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:11.969626 28260 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.002125 28497 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.056241 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1055) took 1384 ms. Trace:
I20260709 13:09:12.056793 21143 rpcz_store.cc:276] 0709 13:09:10.672183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.754105 (+ 81922us) service_pool.cc:224] Handling call
0709 13:09:10.754542 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:10.801712 (+ 47170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.801741 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:10.802229 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.120368 (+318139us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:11.120478 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.120605 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.055946 (+935341us) raft_consensus.cc:1685] finished
0709 13:09:12.055978 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.056162 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46717,"raft.queue_time_us":61323,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7274,"spinlock_wait_cycles":1462144,"thread_start_us":281221,"threads_started":1,"wal-append.queue_time_us":403799}
I20260709 13:09:11.996937 28414 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.025264 21801 raft_consensus.cc:2889] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:12.042708 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:12.040819 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75193.1us
I20260709 13:09:11.998101 28412 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:11.987681 28508 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.975075 28177 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:11.977707 28148 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:11.983067 28013 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.007229 28500 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:11.988785 28401 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:11.941277 28427 consensus_queue.cc:260] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:11.934921 28429 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.021582 28280 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.098331 21140 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 13:09:11.997515 28481 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 13:09:12.050417 28391 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:12.006212 28407 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:12.074267 28335 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.352s	user 0.015s	sys 0.000s
I20260709 13:09:11.983654 28499 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.001638 28100 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.020071 28359 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.104444 28233 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.104626 28500 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:12.067624 28144 consensus_queue.cc:1059] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
W20260709 13:09:12.126959 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 167097us
I20260709 13:09:12.121556 28391 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.162926 21801 raft_consensus.cc:1240] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:12.129643 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54634 (request call id 1428) took 1298 ms. Trace:
I20260709 13:09:12.203701 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.163793 21215 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
W20260709 13:09:12.206753 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.122123 28366 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:12.137420 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 246084us
I20260709 13:09:12.220554 21795 raft_consensus.cc:2393] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 17: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:12.139890 28427 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:12.128822 28280 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:12.082198 28412 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.171838 28498 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:12.203958 21793 rpcz_store.cc:276] 0709 13:09:10.831238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.831465 (+   227us) service_pool.cc:224] Handling call
0709 13:09:10.837709 (+  6244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:10.897002 (+ 59293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.897038 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:10.925169 (+ 28131us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.139546 (+214377us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:11.139608 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.877001 (+737393us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b54002c3654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:11.877052 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.117773 (+240721us) raft_consensus.cc:1685] finished
0709 13:09:12.117795 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.129551 (+ 11756us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":199559,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":10325,"spinlock_wait_cycles":135483648,"thread_start_us":117677,"threads_started":1,"wal-append.queue_time_us":307592}
I20260709 13:09:12.133783 28414 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:09:12.097875 28508 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.182408 28500 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:12.067070 28497 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.180418 28507 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:12.305187 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 87569.7us
I20260709 13:09:12.201025 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:12.330904 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 130007us
I20260709 13:09:12.331694 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:12.217953 28391 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:12.219933 28486 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:09:12.276762 28335 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:12.325605 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 261907us
W20260709 13:09:12.336923 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.280300 28516 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:09:12.375530 21145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 3511) took 2433 ms (client timeout 1994 ms). Trace:
W20260709 13:09:12.411128 21145 rpcz_store.cc:269] 0709 13:09:09.942364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.055920 (+113556us) service_pool.cc:224] Handling call
0709 13:09:12.375334 (+2319414us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54002599d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:12.375448 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304369920}
I20260709 13:09:12.320916 28510 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.328949 28280 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:12.430974 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 209346us
W20260709 13:09:12.414067 28488 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.477s	user 0.000s	sys 0.001s
I20260709 13:09:12.409875 28516 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.412881 28414 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:12.438845 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 107232us
I20260709 13:09:12.419941 28511 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:12.409667 28481 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:09:12.285568 28366 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.435230 28096 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:09:12.440647 28478 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.276s	user 0.001s	sys 0.001s
W20260709 13:09:12.479014 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.433678 28280 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.414382 28335 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.444078 28260 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.344450 28507 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.441740 28502 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:12.501824 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.512120 21795 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:09:12.441222 28516 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:12.461840 28417 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.431303 28510 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.433375 28524 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 13:09:12.503736 28488 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:09:12.524130 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:12.525476 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 71662.6us
I20260709 13:09:12.526443 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:12.505462 28177 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.449319 28414 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.546159 28359 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:12.515808 28528 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.604384 28528 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.606690 28528 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.548254 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:12.607656 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 59511.9us
I20260709 13:09:12.548934 28532 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.555845 28100 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.557188 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.608956 28505 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:12.609050 28532 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.609635 21793 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:12.610014 28509 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.558734 28335 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.610559 28509 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:12.559461 28512 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.611536 28532 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.612111 28532 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:12.611778 28512 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.612752 28509 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:09:12.612617 28532 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.521971 28401 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.616259 28512 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.521982 28096 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:12.617915 28421 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.006s
I20260709 13:09:12.617945 28096 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.530457 28013 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.621690 28096 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.622733 28510 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:12.623636 28096 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:12.624444 28527 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.624788 28096 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:12.625075 28527 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.625698 28096 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:12.627233 28096 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.578073 28497 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:12.583483 28499 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.592815 28507 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:12.460821 28511 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.607559 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.561340 28280 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:12.564993 28290 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.734s	user 0.014s	sys 0.000s
W20260709 13:09:12.565685 28061 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.341s	user 0.010s	sys 0.000s
W20260709 13:09:12.629165 28117 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.607s	user 0.013s	sys 0.000s
I20260709 13:09:12.628916 28144 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:09:12.631232 28350 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.745s	user 0.015s	sys 0.000s
I20260709 13:09:12.635226 28414 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.635972 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.636199 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.637770 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.641896 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:09:12.576542 28533 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:09:12.552379 28508 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.645360 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1053) took 2069 ms. Trace:
W20260709 13:09:12.648689 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 47507.4us
W20260709 13:09:12.648007 21220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 3395) took 2295 ms (client timeout 1771 ms). Trace:
W20260709 13:09:12.591069 28355 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.823s	user 0.009s	sys 0.000s
W20260709 13:09:12.653785 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.655359 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2370) took 1732 ms. Trace:
I20260709 13:09:12.618551 28421 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.680557 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.681573 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:12.688714 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:12.689116 21110 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.009s	sys 0.003s
I20260709 13:09:12.736318 21800 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:12.653352 28359 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.661433 28013 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.695020 21792 raft_consensus.cc:1240] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 16. Current term is 17. Ops: []
W20260709 13:09:12.703796 28347 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 1.577s	user 0.012s	sys 0.001s
I20260709 13:09:12.694098 21801 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
W20260709 13:09:12.703608 21037 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.001s	sys 0.019s
W20260709 13:09:12.740368 21037 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.001s	sys 0.019s
W20260709 13:09:12.706838 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.683784 28407 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:12.729113 21066 rpcz_store.cc:276] 0709 13:09:10.922606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.654272 (+1731666us) service_pool.cc:224] Handling call
0709 13:09:12.654658 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:12.654882 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:12.654922 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:12.654977 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:12.655030 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.655100 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.655267 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:12.740954 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.683416 28497 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:12.742730 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.609683 28505 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:12.729825 21220 rpcz_store.cc:269] 0709 13:09:10.304360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.417622 (+113262us) service_pool.cc:224] Handling call
0709 13:09:12.599121 (+2181499us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b540008fa54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:12.600123 (+  1002us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590960384}
W20260709 13:09:12.734875 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 138906us
W20260709 13:09:12.736395 21110 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.010s	sys 0.003s
I20260709 13:09:12.637507 28532 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.727974 21138 rpcz_store.cc:276] 0709 13:09:10.576298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.713373 (+137075us) service_pool.cc:224] Handling call
0709 13:09:10.713657 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:10.822829 (+109172us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:09:10.846585 (+ 23756us) log.cc:844] Serialized 14 byte log entry
0709 13:09:11.078136 (+231551us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:11.112494 (+ 34358us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.112703 (+   209us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:11.112938 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.113053 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.596530 (+1483477us) raft_consensus.cc:1685] finished
0709 13:09:12.596558 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.645277 (+ 48719us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137281,"prepare.queue_time_us":30535,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":122860,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":22803,"thread_start_us":172110,"threads_started":1,"wal-append.queue_time_us":974343}
I20260709 13:09:12.757133 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:12.739897 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 131376us
W20260709 13:09:12.704504 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 62703.7us
I20260709 13:09:12.732230 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:12.691030 28061 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.690866 28117 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.728004 28350 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:12.761672 28530 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:12.691126 28290 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.685420 28529 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.108s
I20260709 13:09:12.775442 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1345) took 1915 ms. Trace:
I20260709 13:09:12.771167 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:12.767159 21765 connection.cc:256] RPC call timeout handler was delayed by -1.5361966890000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:09:12.741678 28527 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.776876 21065 rpcz_store.cc:276] 0709 13:09:10.860572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.040539 (+1179967us) service_pool.cc:224] Handling call
0709 13:09:12.072842 (+ 32303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:12.097201 (+ 24359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:12.097235 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:12.109397 (+ 12162us) log.cc:844] Serialized 26 byte log entry
0709 13:09:12.310415 (+201018us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:09:12.310465 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.310558 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.775203 (+464645us) raft_consensus.cc:1685] finished
0709 13:09:12.775224 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.775381 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7053,"raft.queue_time_us":447694,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":9721,"thread_start_us":118752,"threads_started":1,"wal-append.queue_time_us":602446}
I20260709 13:09:12.736783 28391 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.733953 28355 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.788823 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.789094 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.766819 28545 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.757787 28524 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.779577 21801 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:12.772617 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:12.782792 21796 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 16.
W20260709 13:09:12.817027 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 44470.7us
I20260709 13:09:12.793154 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.789409 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.773759 28490 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:12.819469 28506 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:12.796155 21071 raft_consensus.cc:1240] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:09:12.819499 21144 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:12.820086 28506 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.820998 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.745672 28347 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.821643 28540 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:12.742614 28359 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.822152 28540 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.765448 28499 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:12.773554 28492 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:09:12.814568 28527 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.823748 28359 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.820644 28521 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.823799 28499 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:12.821125 28548 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.824380 28359 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.824656 28499 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:12.825130 28499 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:12.824841 28548 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.825011 28359 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.825479 28540 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.824132 28527 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.818385 21800 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:12.789803 21145 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:12.823402 21214 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:12.773979 28544 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:12.824476 28521 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.819691 28490 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:12.744855 28013 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.766938 28350 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.747042 28497 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.816392 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:12.837999 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.838384 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.823155 28506 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.779196 28532 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.769879 28335 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:09:12.788239 28280 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.921629 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:12.921366 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 68392.9us
I20260709 13:09:12.924520 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.866348 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:12.924520 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.866703 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:12.800194 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:12.913486 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.961550 21797 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:12.962373 21110 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:12.787405 28395 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:12.868070 21063 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 13:09:12.933418 21144 raft_consensus.cc:2410] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:09:12.771749 28407 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.083s
W20260709 13:09:12.965317 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 182864us
I20260709 13:09:12.965560 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:12.933130 21139 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:12.970403 21793 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
W20260709 13:09:12.975642 28549 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:09:12.816049 28144 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:12.827219 28548 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:12.849018 28366 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:12.923446 28502 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.933508 28350 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:12.827267 28359 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:12.921868 28540 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:12.962677 28543 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.993009 21142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1054) took 2384 ms. Trace:
I20260709 13:09:12.933359 28544 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:12.996420 21142 rpcz_store.cc:276] 0709 13:09:10.608868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.721309 (+112441us) service_pool.cc:224] Handling call
0709 13:09:10.721733 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:10.798498 (+ 76765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.798532 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:10.799009 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.113133 (+314124us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:09:11.113187 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.114004 (+   817us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.992789 (+1878785us) raft_consensus.cc:1685] finished
0709 13:09:12.992815 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.992948 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84978,"raft.queue_time_us":79472,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":7639,"thread_start_us":313702,"threads_started":1,"wal-append.queue_time_us":1895550}
I20260709 13:09:12.999383 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.999418 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.000378 21142 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:09:12.933616 28532 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.981073 28506 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:13.002671 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:13.003621 21036 connection.cc:256] RPC call timeout handler was delayed by -1.1840654500000483s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1771s for a response
I20260709 13:09:13.013069 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.017925 21065 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:13.019584 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.987769 28290 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.020576 21142 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:09:13.025900 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.784585 28530 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.977486 28280 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:12.994997 28096 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.995689 28529 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:12.987794 28144 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:12.774102 28536 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.332s
I20260709 13:09:13.066313 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2371) took 2143 ms. Trace:
I20260709 13:09:13.047887 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:12.849553 28524 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.058388 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.102035 21065 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:13.065444 28280 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:13.074018 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 96231.1us
I20260709 13:09:13.100446 21066 rpcz_store.cc:276] 0709 13:09:10.922927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.741101 (+1818174us) service_pool.cc:224] Handling call
0709 13:09:12.741453 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:12.745159 (+  3706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:12.745192 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:12.756332 (+ 11140us) log.cc:844] Serialized 26 byte log entry
0709 13:09:12.942512 (+186180us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:12.942568 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.942671 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.066029 (+123358us) raft_consensus.cc:1685] finished
0709 13:09:13.066060 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.066235 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124347,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":48308,"thread_start_us":159486,"threads_started":1,"wal-append.queue_time_us":244823}
I20260709 13:09:12.998350 28502 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.105935 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:13.108413 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 60623.5us
I20260709 13:09:12.867442 28545 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.109017 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.109669 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:13.102193 28491 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:09:13.021912 28506 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.118314 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.054548 28290 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.027951 28543 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.126016 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3209) took 2054 ms. Trace:
I20260709 13:09:13.127002 28543 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.055994 28519 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:13.126415 21066 rpcz_store.cc:276] 0709 13:09:11.072176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.122385 (+2050209us) service_pool.cc:224] Handling call
0709 13:09:13.125387 (+  3002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.125682 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:13.125716 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:13.125760 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:13.125804 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.125857 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.125962 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:13.130893 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.131608 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:12.990244 28490 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.136457 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:13.136770 21145 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:13.136457 21142 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:13.134227 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2374) took 2029 ms. Trace:
I20260709 13:09:13.140448 21066 rpcz_store.cc:276] 0709 13:09:11.104999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.133253 (+2028254us) service_pool.cc:224] Handling call
0709 13:09:13.133552 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.133720 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:13.133750 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:13.133793 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:13.133830 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.133881 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.134162 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:13.057610 28529 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.057659 28096 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:13.142819 21066 pending_rounds.cc:85] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 13
I20260709 13:09:13.136737 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:13.137125 28544 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.142999 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.144304 28013 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.120620 28117 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.159996 21066 pending_rounds.cc:107] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 21.14
I20260709 13:09:13.161273 21066 raft_consensus.cc:2917] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Aborting config change with OpId 21.14: Aborted: Op aborted by new leader
I20260709 13:09:13.163412 21142 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 26.
I20260709 13:09:13.164800 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:13.110080 28515 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.364s
I20260709 13:09:13.168546 28545 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.137266 28521 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.150075 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.162750 21066 raft_consensus.cc:2889] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:13.137696 28540 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.137283 28561 consensus_queue.cc:1059] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:09:13.174664 21794 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:09:13.172958 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 67132.9us
I20260709 13:09:13.067951 28501 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.383s
I20260709 13:09:13.180045 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.118834 28541 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 13:09:13.169476 28558 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.146274 28391 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.170182 21145 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:13.132764 28530 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.141448 28527 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.149770 28355 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.132678 28519 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.136879 28290 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.174526 28421 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.169564 28535 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:13.188681 28497 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.149261 28548 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:13.207500 21767 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:13.205984 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:13.269526 21037 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.004s	sys 0.008s
I20260709 13:09:13.170122 28505 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:13.257553 28553 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
W20260709 13:09:13.223965 28559 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:09:13.299695 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2373) took 2376 ms. Trace:
W20260709 13:09:13.261896 28516 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.001s
W20260709 13:09:13.305855 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 98441.8us
I20260709 13:09:13.306073 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:13.277486 28548 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.312072 28552 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:13.312263 28548 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:13.312618 28552 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:13.312682 28481 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:13.313045 28552 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:13.313205 28481 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.312283 21145 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:13.215234 28550 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:09:13.304688 21037 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.006s	sys 0.008s
W20260709 13:09:13.265019 28427 consensus_meta.cc:287] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.976s	user 0.011s	sys 0.001s
W20260709 13:09:13.185317 28539 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 13:09:13.274935 28421 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.314697 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.227058 28545 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:13.238914 28532 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.278020 28527 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.320899 21142 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:13.237380 28506 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.313712 28481 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:13.187947 28540 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.322567 21143 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:13.313710 28290 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.323110 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.308998 21065 rpcz_store.cc:276] 0709 13:09:10.923463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.107871 (+2184408us) service_pool.cc:224] Handling call
0709 13:09:13.123736 (+ 15865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.123943 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:09:13.270001 (+146058us) log.cc:844] Serialized 14 byte log entry
0709 13:09:13.299145 (+ 29144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:13.299228 (+    83us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:13.299327 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.299430 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.299614 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21860,"thread_start_us":138082,"threads_started":2}
I20260709 13:09:13.274014 28543 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.316241 28550 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.337338 21765 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:13.266953 28560 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.323683 28421 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.316047 28546 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:13.335594 28544 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:13.350607 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.349377 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.352319 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:13.352819 21142 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:13.353637 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.354354 21142 raft_consensus.cc:2410] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 25 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:09:13.355127 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.356834 21145 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:13.359467 21145 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:13.338724 28550 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:13.324316 28557 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:13.328683 28545 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.351542 28502 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.373687 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1341) took 2543 ms. Trace:
I20260709 13:09:13.359964 28355 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:13.383165 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.374329 21070 rpcz_store.cc:276] 0709 13:09:10.829948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.975577 (+1145629us) service_pool.cc:224] Handling call
0709 13:09:11.993619 (+ 18042us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:12.124411 (+130792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:12.124456 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:12.134159 (+  9703us) log.cc:844] Serialized 26 byte log entry
0709 13:09:12.437508 (+303349us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:12.437545 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.437614 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.373387 (+935773us) raft_consensus.cc:1685] finished
0709 13:09:13.373422 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.373605 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":219,"raft.queue_time_us":498319,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":11906,"thread_start_us":279364,"threads_started":2,"wal-append.queue_time_us":554774}
I20260709 13:09:13.275486 28558 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.338951 28527 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:13.376582 28503 log.cc:927] Time spent T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.351s	user 0.005s	sys 0.000s
I20260709 13:09:13.344442 28543 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.358059 28544 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.326274 28516 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:13.393731 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 56496us
I20260709 13:09:13.394055 28565 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.361949 28546 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.395857 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.368348 28428 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:13.350430 28560 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.394675 28565 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.456457 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3196) took 2711 ms. Trace:
I20260709 13:09:13.406113 21138 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25. Ops: []
W20260709 13:09:13.482163 21796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1464) took 2681 ms (client timeout 2767 ms). Trace:
I20260709 13:09:13.405882 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:13.479554 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1056) took 2719 ms. Trace:
I20260709 13:09:13.449173 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:13.432488 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3509) took 3490 ms. Trace:
I20260709 13:09:13.399931 28546 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.397284 21070 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
W20260709 13:09:13.399534 28500 consensus_meta.cc:287] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.070s	user 0.012s	sys 0.002s
I20260709 13:09:13.422381 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 984) took 1988 ms. Trace:
I20260709 13:09:13.502427 21140 rpcz_store.cc:276] 0709 13:09:10.759932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.856885 (+ 96953us) service_pool.cc:224] Handling call
0709 13:09:10.857219 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:12.113033 (+1255814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.479226 (+1366193us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400258ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:13.479483 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2853542144}
I20260709 13:09:13.369001 28557 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:13.521166 21140 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:09:13.502200 21796 rpcz_store.cc:269] 0709 13:09:10.800787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.800931 (+   144us) service_pool.cc:224] Handling call
0709 13:09:13.481935 (+2681004us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54006c1e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:13.482088 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4485380352}
W20260709 13:09:13.456789 28569 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:09:13.501930 21067 rpcz_store.cc:276] 0709 13:09:10.744998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.810476 (+ 65478us) service_pool.cc:224] Handling call
0709 13:09:10.810842 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.949871 (+1139029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.456013 (+1506142us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540001aed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:13.456364 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2452101504}
I20260709 13:09:13.453913 28148 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.441267 28519 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.524493 21140 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:09:13.525099 28560 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.459509 28540 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.527134 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.475095 28365 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:13.520052 21217 rpcz_store.cc:276] 0709 13:09:11.434365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.434553 (+   188us) service_pool.cc:224] Handling call
0709 13:09:11.435019 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:11.436851 (+  1832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.436897 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:11.437394 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.810905 (+373511us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:11.971117 (+160212us) log.cc:844] Serialized 14 byte log entry
0709 13:09:11.974569 (+  3452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.974674 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.372642 (+1397968us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:13.422044 (+ 49402us) raft_consensus.cc:1685] finished
0709 13:09:13.422159 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.422316 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3131,"prepare.queue_time_us":308059,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":30523,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":4202,"spinlock_wait_cycles":26793728,"thread_start_us":503375,"threads_started":2,"wal-append.queue_time_us":550000}
I20260709 13:09:13.409482 28379 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:13.449543 28440 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.410053 28421 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.433233 28524 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.420827 28437 raft_consensus.cc:3055] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:13.502671 21141 rpcz_store.cc:276] 0709 13:09:09.941699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.563988 (+622289us) service_pool.cc:224] Handling call
0709 13:09:10.581212 (+ 17224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:10.641574 (+ 60362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.641634 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:10.641691 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:10.641737 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.432201 (+2790464us) spinlock_profiling.cc:243] Waited 60.1 ms on lock 0x7b540036cc54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:13.432254 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.432422 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138258304}
I20260709 13:09:13.475322 28502 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.435902 28572 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:13.480471 28516 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.536779 28545 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:13.531821 28567 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.125s	user 0.001s	sys 0.001s
I20260709 13:09:13.527165 28500 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.542734 28543 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.480460 28527 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.390565 28355 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.556706 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.557226 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:13.543499 28544 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:13.535553 28499 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.003s
I20260709 13:09:13.527719 28236 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
W20260709 13:09:13.556507 28452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.118s	user 0.011s	sys 0.025s
I20260709 13:09:13.564610 21072 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:13.556496 28437 raft_consensus.cc:740] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:13.533305 28365 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:13.565781 28468 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.760s	user 0.009s	sys 0.021s
W20260709 13:09:13.575637 28468 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.796s	user 0.009s	sys 0.021s
I20260709 13:09:13.574867 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3211) took 2139 ms. Trace:
I20260709 13:09:13.576244 21066 rpcz_store.cc:276] 0709 13:09:11.435342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.142310 (+1706968us) service_pool.cc:224] Handling call
0709 13:09:13.142647 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:13.172443 (+ 29796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:13.178665 (+  6222us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:13.181301 (+  2636us) log.cc:844] Serialized 26 byte log entry
0709 13:09:13.324103 (+142802us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:13.324136 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.324221 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.574624 (+250403us) raft_consensus.cc:1685] finished
0709 13:09:13.574647 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.574796 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":142519,"threads_started":1,"wal-append.queue_time_us":188194}
I20260709 13:09:13.577797 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.578415 21066 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:13.579212 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.580206 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.580717 21066 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:13.580940 28500 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.574802 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:13.564642 28350 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.560s	user 0.006s	sys 0.006s
W20260709 13:09:13.587522 28494 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.791s	user 0.006s	sys 0.013s
I20260709 13:09:13.587823 28236 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:13.587656 28350 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.588353 28236 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:13.589247 28236 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.590334 28236 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:13.591572 28535 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:13.592111 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3522) took 2782 ms. Trace:
I20260709 13:09:13.592476 21147 rpcz_store.cc:276] 0709 13:09:10.809601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.920906 (+111305us) service_pool.cc:224] Handling call
0709 13:09:10.957440 (+ 36534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:11.020401 (+ 62961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.020468 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:11.054863 (+ 34395us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.220908 (+166045us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:11.220971 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.221074 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.581604 (+2360530us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:13.591814 (+ 10210us) raft_consensus.cc:1685] finished
0709 13:09:13.591834 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.592049 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17306,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":17940,"spinlock_wait_cycles":71708416,"thread_start_us":120718,"threads_started":1,"wal-append.queue_time_us":466168}
I20260709 13:09:13.595693 21065 raft_consensus.cc:2368] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 16. Current term is 17.
W20260709 13:09:13.572196 28452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.140s	user 0.012s	sys 0.025s
I20260709 13:09:13.559567 28502 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.602360 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.607827 21072 raft_consensus.cc:2410] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 19 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:09:13.609699 28548 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.610193 28412 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.611578 21142 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 24: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:13.595872 28350 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.574072 28499 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.616902 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:13.565820 28572 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.611059 28502 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.624435 28552 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.613092 28468 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:13.616175 28535 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:13.611672 28365 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:13.628353 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.631985 28548 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.632520 28025 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:13.640481 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:13.649868 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:13.650502 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.650560 21072 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:13.638635 28061 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.654763 28482 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:13.655370 28552 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.648102 28025 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:13.656358 28025 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.658041 28499 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.660681 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2422) took 3579 ms. Trace:
I20260709 13:09:13.661105 21218 rpcz_store.cc:276] 0709 13:09:10.080727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.112308 (+ 31581us) service_pool.cc:224] Handling call
0709 13:09:10.112539 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.120665 (+1008126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.660397 (+2539732us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b540015cf54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:13.660586 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2283469568}
I20260709 13:09:13.644220 28468 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.662809 28511 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.661092 28546 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.663774 28419 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:13.674681 28419 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:13.655249 28482 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:13.657694 28572 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:13.665781 28554 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.616s	user 0.000s	sys 0.003s
I20260709 13:09:13.666155 21069 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.664753 28468 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.677995 28482 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.669651 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:13.640163 28437 consensus_queue.cc:260] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.687503 28482 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:13.688238 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:13.689347 21069 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:09:13.689781 21069 pending_rounds.cc:85] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 10
I20260709 13:09:13.690294 21069 pending_rounds.cc:107] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:09:13.651701 28535 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.690635 21069 raft_consensus.cc:2889] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:13.691980 28546 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.693519 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2369) took 2820 ms. Trace:
I20260709 13:09:13.695956 28523 raft_consensus.cc:3055] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:13.701545 28523 raft_consensus.cc:740] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:13.696648 28558 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.700445 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3205) took 2925 ms. Trace:
I20260709 13:09:13.686750 28481 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.701390 21068 rpcz_store.cc:276] 0709 13:09:10.873719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.568659 (+1694940us) service_pool.cc:224] Handling call
0709 13:09:12.569110 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:12.569546 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:12.569588 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:12.570243 (+   655us) log.cc:844] Serialized 26 byte log entry
0709 13:09:13.692929 (+1122686us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b18002aa780. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f420c44bc 00007f7f3f41248b 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1
0709 13:09:13.693091 (+   162us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:13.693127 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.693198 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.693258 (+    60us) raft_consensus.cc:1685] finished
0709 13:09:13.693269 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.693462 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119362,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2983,"spinlock_wait_cycles":3436800,"thread_start_us":119365,"threads_started":1,"wal-append.queue_time_us":294493}
I20260709 13:09:13.700336 28504 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.704576 21069 rpcz_store.cc:276] 0709 13:09:10.775564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.096642 (+321078us) service_pool.cc:224] Handling call
0709 13:09:11.096914 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.665647 (+2568733us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b58009578d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f424780aa 00007f7f424fa447 00007f7f42512fe7 00007f7f425147de 00007f7f4251eb49 00007f7f4251f416 00007f7f4251bd8f 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4
0709 13:09:13.700173 (+ 34526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.700389 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52456192}
I20260709 13:09:13.703706 28523 consensus_queue.cc:260] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.705410 28534 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.705833 28499 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.708401 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.708309 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.699416 28365 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:13.708431 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.712584 21068 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:13.707840 28504 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.711376 28507 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.721117 28507 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.714522 28499 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.716383 21072 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 15.
I20260709 13:09:13.705057 28355 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.718674 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.715262 28534 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.704746 28464 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.722723 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.726338 21068 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:13.726419 21218 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:09:13.724061 21217 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:13.727110 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.727316 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.727790 21220 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:13.727910 21068 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 26.
I20260709 13:09:13.729009 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.729200 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.730721 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.730592 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.731431 21213 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:13.732838 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.732972 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.733704 21213 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:13.736032 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.729802 21068 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:13.738255 28535 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.740208 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:13.741444 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.741207 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.743178 21796 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 24: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:13.740123 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.744261 21217 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:13.745261 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:13.750464 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:13.752914 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.722566 28523 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:13.766338 21216 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 19: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:13.763209 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.768030 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.767328 21072 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:09:13.768079 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.771749 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.772343 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.774009 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.775274 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.775609 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.764989 28464 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.776584 21144 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:13.775949 21217 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:13.777611 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.778518 21145 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:13.778595 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.721691 28419 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.781544 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.783382 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.781661 21217 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 15.
I20260709 13:09:13.784621 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.784732 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:13.785329 21221 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:13.786131 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.785964 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.780851 21213 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 14.
I20260709 13:09:13.787194 21217 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:13.782214 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:13.788463 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.789305 21221 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:13.791424 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:13.792058 21072 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.765674 28481 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.791424 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.794572 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.795513 21141 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:13.795723 21217 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 26.
I20260709 13:09:13.793586 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.796929 21140 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 26.
I20260709 13:09:13.797101 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.800100 21221 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:09:13.799896 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:13.782836 28452 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:13.803151 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.805428 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.794626 28061 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:13.806659 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.807688 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.808790 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.813378 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:13.820974 28546 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:13.821506 28546 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.821975 28546 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:13.823268 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:13.804466 28452 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.823925 21141 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:13.806317 28355 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:13.826347 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.838515 21795 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:13.837743 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:13.839903 21145 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:13.840961 28233 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.841522 28233 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.848460 28529 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.854024 28428 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:13.808495 28507 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.812180 28550 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.856763 21064 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 24: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:13.809890 28541 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:13.838034 21139 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:13.862241 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:13.809476 28061 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.866618 21794 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:13.859458 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.868621 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:13.868777 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:13.874449 21069 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:09:13.860049 28507 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.877035 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.839942 28452 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:13.838063 28355 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.880116 21072 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:13.880189 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.886133 21767 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:13.864619 28558 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:13.859781 28419 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.863102 28541 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.867791 28061 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:13.901155 21141 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:13.892426 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.877969 28529 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:13.890481 28355 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:13.889659 28523 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:13.894459 28431 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:13.906376 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.890610 28532 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:13.894702 28558 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.909448 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:13.894398 28468 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:09:13.914402 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.878938 28507 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:13.905184 28082 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:13.917587 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.904007 28541 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.915551 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:13.910424 28523 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:13.893281 28534 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.920229 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.920759 28082 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.921919 21219 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:13.905598 28529 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.922724 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:13.947502 28347 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.915961 28532 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:13.957280 28532 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:13.960014 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:13.917258 28468 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.961887 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.918715 28552 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:13.916929 28558 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:13.964337 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:13.951612 28082 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:13.921598 28431 raft_consensus.cc:697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:13.964506 28544 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:09:13.967823 28580 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:09:13.968096 28527 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:13.969070 28452 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:13.973232 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2375) took 2198 ms. Trace:
I20260709 13:09:13.959832 28350 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:13.961187 28529 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:13.913199 28280 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:09:13.962703 28504 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:13.952422 28550 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:13.953722 28523 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:13.973966 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:13.979589 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:13.960631 28464 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.965924 28528 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.949855 28576 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:13.964888 28468 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:14.024092 28431 consensus_queue.cc:237] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:14.026396 28541 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.028621 28541 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.012501 28350 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.010161 21066 rpcz_store.cc:276] 0709 13:09:11.774654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.581292 (+1806638us) service_pool.cc:224] Handling call
0709 13:09:13.581591 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:13.581983 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:13.582011 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:13.588532 (+  6521us) log.cc:844] Serialized 26 byte log entry
0709 13:09:13.846531 (+257999us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b1800377260. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f411e07 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f420b620b 00007f7f420bbd23 00007f7f420bc0d1 00007f7f424d7a0f
0709 13:09:13.906900 (+ 60369us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:13.906959 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.907054 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.962353 (+ 55299us) raft_consensus.cc:1685] finished
0709 13:09:13.962378 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.973175 (+ 10797us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":987,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":15140224,"thread_start_us":60110,"threads_started":1,"wal-append.queue_time_us":362464}
I20260709 13:09:13.969157 28499 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:13.972975 28552 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.018302 21219 raft_consensus.cc:2368] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:14.010442 28527 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.020987 28550 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:14.036010 28582 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 13:09:14.038002 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.976787 28421 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:14.018630 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:14.045518 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.031068 28576 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:14.040174 28350 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:14.038447 28583 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:09:14.052119 21066 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 24: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:14.041719 28552 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:14.053704 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3601) took 1265 ms. Trace:
I20260709 13:09:14.048803 28421 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.053082 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:14.043890 28527 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:14.035233 28528 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:14.066697 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:14.058696 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.043465 28550 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.063166 21138 rpcz_store.cc:276] 0709 13:09:12.788102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.788295 (+   193us) service_pool.cc:224] Handling call
0709 13:09:12.788861 (+   566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.502173 (+713312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.053452 (+551279us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b5400424ad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:14.053633 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415102336}
I20260709 13:09:14.041380 28082 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.051940 28135 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:09:14.068181 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.073227 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:14.068012 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.075845 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:14.053941 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.079927 21144 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 24: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:14.078744 28280 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:14.082844 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:14.083763 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.063808 28233 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.056166 28576 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:14.085824 28135 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:14.080780 28504 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:14.078727 28550 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:14.068634 28421 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:14.080993 28452 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.090695 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:14.094218 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.105079 21219 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:14.106699 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:14.107486 21212 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:14.078450 28468 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:09:14.109822 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.085062 28541 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:09:14.102629 28504 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:14.108901 28489 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 2.112s	user 0.007s	sys 0.005s
I20260709 13:09:14.119817 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1346) took 3237 ms. Trace:
I20260709 13:09:14.121883 21063 rpcz_store.cc:276] 0709 13:09:10.882168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.574358 (+1692190us) service_pool.cc:224] Handling call
0709 13:09:12.574867 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:12.976669 (+401802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.119528 (+1142859us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b540041c654. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:14.119749 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634562944}
I20260709 13:09:14.123337 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:14.123917 21063 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:14.100906 28507 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:14.128670 28135 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.131932 28504 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.133235 28082 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.133759 28429 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:14.135521 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:14.135499 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:14.146445 21142 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:14.138165 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.138726 27892 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.139313 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:14.148335 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.127699 28365 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.152446 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:14.120074 28061 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:14.157007 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:14.160398 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.151721 28507 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:14.160539 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.149927 27892 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.164814 21212 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:14.165700 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:14.165910 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:14.166647 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.167497 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.168553 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.169166 21212 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:09:14.169983 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:14.171725 28464 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:14.172186 28464 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.172569 28464 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:14.173103 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.173790 21069 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:14.174768 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:14.156047 28365 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:14.158026 28499 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:14.178478 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:14.179076 21216 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:14.179761 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.180373 21185 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:14.180771 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:14.181906 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.179896 21142 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 27.
I20260709 13:09:14.182448 21069 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:14.184500 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:14.183771 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:14.188297 21063 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:14.160022 28528 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.189898 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.190357 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.187021 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:14.163888 28280 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:14.191646 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.192077 21212 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:09:14.194974 21066 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 16.
I20260709 13:09:14.185463 28334 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:14.195191 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.192713 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:14.197222 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:14.196557 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.198734 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:14.201159 21215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 3397) took 3608 ms (client timeout 1586 ms). Trace:
W20260709 13:09:14.201570 21215 rpcz_store.cc:269] 0709 13:09:10.592654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.668509 (+ 75855us) service_pool.cc:224] Handling call
0709 13:09:14.200992 (+3532483us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54001252d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:14.201093 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3085783808}
I20260709 13:09:14.203068 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.201506 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.182708 28507 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:14.208415 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:14.210772 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.210903 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:14.211532 21794 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:14.191901 28347 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:14.209695 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:14.219012 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:14.190768 28365 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:14.208730 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:14.189587 28504 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:09:14.227063 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.229267 28061 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.231016 21219 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:14.200171 28334 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.225168 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:14.229996 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:14.229338 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:14.231771 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:14.232378 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:14.203698 28388 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:14.200587 28499 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:14.237178 21801 raft_consensus.cc:2410] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 13:09:14.237630 21794 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:14.238973 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:14.239466 21794 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:14.227476 28347 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.210041 28482 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:14.195969 28546 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:14.229295 28408 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:14.237463 28334 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:14.247661 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:14.248466 28570 log.cc:927] Time spent T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.886s	user 0.003s	sys 0.005s
I20260709 13:09:14.249956 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 1353) took 1003 ms. Trace:
I20260709 13:09:14.251621 28437 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:14.210382 28364 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:09:14.244844 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:14.254804 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:14.300118 21219 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:14.300787 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:14.302731 21801 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:14.203543 28534 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:09:14.256842 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:14.257041 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:14.304464 21142 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:14.306778 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:14.308032 28541 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:14.308427 28541 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:14.308404 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:14.309062 28541 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:14.309211 21142 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:14.311583 28401 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:14.312167 21018 auto_rebalancer.cc:893] Removing replace marker failed: Replica 5e5e65640c8b4f5ca7eadc6a14312c55 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:14.260689 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:14.312621 21018 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:09:14.261883 21065 rpcz_store.cc:276] 0709 13:09:13.246723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.333425 (+ 86702us) service_pool.cc:224] Handling call
0709 13:09:14.249751 (+916326us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b540011bcd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:14.249897 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557779200}
I20260709 13:09:14.240197 28499 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:14.312942 21018 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:09:14.329380 28499 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:14.330778 28144 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.312176 28401 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:14.333845 28565 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.336493 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:14.337026 21070 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:09:14.337606 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2380) took 2561 ms. Trace:
I20260709 13:09:14.338230 27892 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:14.339393 21141 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:09:12.769723 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:14.343935 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2377) took 2568 ms. Trace:
I20260709 13:09:14.344431 21065 rpcz_store.cc:276] 0709 13:09:11.775297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.335562 (+2560265us) service_pool.cc:224] Handling call
0709 13:09:14.335920 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.336122 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:14.336149 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:14.336196 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:14.336235 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.343756 (+  7521us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.343878 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:14.239497 28504 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:14.379820 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2425) took 3354 ms. Trace:
I20260709 13:09:14.385430 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:14.401779 21214 rpcz_store.cc:276] 0709 13:09:11.025555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.027945 (+  2390us) service_pool.cc:224] Handling call
0709 13:09:11.028386 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:12.830711 (+1802325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.379532 (+1548821us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400442fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:14.379750 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4127586048}
I20260709 13:09:14.337944 21070 rpcz_store.cc:276] 0709 13:09:11.776172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.336284 (+2560112us) service_pool.cc:224] Handling call
0709 13:09:14.336736 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.337375 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.337551 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:14.386718 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.380950 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3603) took 1592 ms. Trace:
I20260709 13:09:14.404277 21221 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:14.406567 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.345929 28280 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:14.367079 28589 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:14.404258 21143 rpcz_store.cc:276] 0709 13:09:12.788737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.788863 (+   126us) service_pool.cc:224] Handling call
0709 13:09:14.380767 (+1591904us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b54002b43d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:14.380887 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1225453824}
I20260709 13:09:14.417500 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.398530 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3213) took 2430 ms. Trace:
I20260709 13:09:14.431317 21065 rpcz_store.cc:276] 0709 13:09:11.968549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.345642 (+2377093us) service_pool.cc:224] Handling call
0709 13:09:14.397976 (+ 52334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.398163 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:14.398186 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:14.398240 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:14.398274 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.398343 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.398468 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:14.424063 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.254652 28408 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.440891 28408 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:14.246152 28347 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:09:14.293452 28587 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:09:14.441943 28589 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:14.421376 21036 leader_election.cc:336] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.651s (SENT)
I20260709 13:09:14.443236 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:14.444649 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:14.445677 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.446828 28144 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:14.447836 28144 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:14.448300 28144 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:14.448742 28144 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:14.452787 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.408295 21221 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:14.297254 28546 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:14.475544 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.471379 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 2382) took 1118 ms. Trace:
I20260709 13:09:14.476361 21064 rpcz_store.cc:276] 0709 13:09:13.352700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.526876 (+174176us) service_pool.cc:224] Handling call
0709 13:09:14.471123 (+944247us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b5400019fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:14.471274 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748258048}
I20260709 13:09:14.477363 21794 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:14.480475 21220 raft_consensus.cc:1240] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:09:14.295020 28468 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:14.483347 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.484705 21220 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:14.255089 28482 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:14.490163 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1347) took 2065 ms. Trace:
I20260709 13:09:14.498023 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:14.492614 21113 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.193:42843 timed out after 1.746s (SENT) [suppressed 23 similar messages]
I20260709 13:09:14.494109 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:14.498399 21065 rpcz_store.cc:276] 0709 13:09:12.425284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.432810 (+2007526us) service_pool.cc:224] Handling call
0709 13:09:14.489481 (+ 56671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.489723 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:14.489761 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:14.489819 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:14.489872 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.489944 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.490092 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:14.499557 21213 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:14.387573 28557 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.496567 21142 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
W20260709 13:09:14.500144 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:14.500730 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 276039us
I20260709 13:09:14.507051 21214 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:09:14.510989 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:14.513087 28176 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:09:14.522461 21216 raft_consensus.cc:2468] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:14.523905 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.525848 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.528625 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:14.531589 21216 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:14.530941 21219 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:14.539332 21221 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:14.547457 28493 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:14.547870 28176 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:14.548362 28176 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:14.549294 28176 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.549731 28504 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.551484 21214 raft_consensus.cc:2393] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 23: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:14.550585 28176 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:14.366420 28565 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.134s
W20260709 13:09:14.553242 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.548187 21068 raft_consensus.cc:2468] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:14.524896 27378 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:14.560243 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:14.571103 21213 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:14.572568 21214 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:14.572664 27378 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:14.573643 27378 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:14.576818 21213 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:09:14.474704 28527 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.581907 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:14.580981 27378 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.582185 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.548230 28493 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.586428 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:14.588140 21219 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:14.590651 27378 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:14.583477 28523 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.562s	user 0.013s	sys 0.009s
I20260709 13:09:14.608029 21070 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:14.608922 21221 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:14.608278 21187 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:14.612182 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:14.614010 28552 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.539s	user 0.004s	sys 0.009s
I20260709 13:09:14.589318 28493 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:14.590000 27892 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:14.590041 28388 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:14.576505 28534 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:09:14.621133 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:14.584244 27957 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:09:14.618659 28589 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.616475 28417 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:14.655771 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:14.658665 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 166061us
I20260709 13:09:14.665557 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1338) took 3836 ms. Trace:
I20260709 13:09:14.666075 21071 rpcz_store.cc:276] 0709 13:09:10.829187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.890429 (+1061242us) service_pool.cc:224] Handling call
0709 13:09:11.890836 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:12.820494 (+929658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.665252 (+1844758us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b540026e154. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:14.665488 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081310464}
I20260709 13:09:14.667860 21071 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:14.626590 27892 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:14.611363 28523 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.634296 28565 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:14.634996 28320 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:14.634565 28388 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:14.688689 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:09:14.696311 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 116260us
I20260709 13:09:14.699950 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2391) took 1108 ms. Trace:
I20260709 13:09:14.700356 21071 rpcz_store.cc:276] 0709 13:09:13.591200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.667175 (+1075975us) service_pool.cc:224] Handling call
0709 13:09:14.667520 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.668248 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.699895 (+ 31647us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:14.701128 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:14.701857 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:14.703441 21071 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:14.650949 28401 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:14.703660 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:14.675463 28355 consensus_meta.cc:287] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.760s	user 0.017s	sys 0.004s
I20260709 13:09:14.711860 21143 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:14.705360 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:14.690189 28429 consensus_meta.cc:287] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.003s
I20260709 13:09:14.706792 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:14.709458 21110 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:14.704902 21144 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:14.634419 28552 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.715775 21138 raft_consensus.cc:2393] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 24: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:14.712860 21798 raft_consensus.cc:2393] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 23: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:14.714000 21113 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:14.686381 28177 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.777035 21147 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:14.779140 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:14.780016 21146 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:09:14.781438 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:14.783030 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:14.758540 21035 leader_election.cc:336] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Timed out: RequestConsensusVote RPC to 127.19.127.196:33295 timed out after 1.578s (SENT)
I20260709 13:09:14.783746 21035 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:14.689376 27892 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:14.698256 28388 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:14.722671 28529 consensus_meta.cc:287] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.016s
W20260709 13:09:14.821823 21797 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.002s
I20260709 13:09:14.822343 21797 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
W20260709 13:09:14.812435 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53924.7us
W20260709 13:09:14.819995 21792 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.012s
I20260709 13:09:14.823966 21792 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
W20260709 13:09:14.716451 28558 consensus_meta.cc:287] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.695s	user 0.020s	sys 0.000s
I20260709 13:09:14.826150 21796 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:14.811403 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:14.827083 21798 raft_consensus.cc:2393] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:14.697172 28320 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:14.716053 28550 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.619s	user 0.014s	sys 0.000s
W20260709 13:09:14.786108 28532 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.827s	user 0.015s	sys 0.003s
I20260709 13:09:14.847877 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:14.776319 28284 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.777845 28401 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:14.855798 28552 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:14.861312 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 147410us
I20260709 13:09:14.863252 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:14.872071 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:14.826409 27957 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:14.828491 28589 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:14.877539 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:14.830521 28482 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.702494 28565 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:14.877761 21113 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:14.850057 28320 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:14.828676 28529 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.923884 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:14.724301 28355 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:14.910145 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 200791us
I20260709 13:09:14.929075 21110 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:14.929208 21035 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:14.930546 21035 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:14.911875 21798 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:14.805831 28177 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:14.933938 28334 consensus_meta.cc:287] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.631s	user 0.017s	sys 0.014s
I20260709 13:09:14.934401 28334 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:14.938652 28557 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.939218 28557 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.941313 28557 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.941951 28347 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:14.920312 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:14.964103 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.964275 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:14.825188 28558 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:14.922907 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45265.2us
I20260709 13:09:14.965628 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:14.884400 28419 raft_consensus.cc:2764] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:09:14.852461 28401 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:14.932968 28591 log.cc:927] Time spent T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:09:14.935400 28177 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:14.935478 28437 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:14.806866 28429 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 13:09:14.946321 28334 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:14.947513 28529 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:14.852120 28532 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.957726 21035 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:14.914855 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 199550us
I20260709 13:09:15.000489 21212 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 27: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:14.860787 28284 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.860387 28550 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:14.921195 28527 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:14.955250 28541 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.646s	user 0.005s	sys 0.011s
I20260709 13:09:14.928723 28595 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:14.996582 21063 raft_consensus.cc:2393] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:14.967379 28589 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:14.968103 28528 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:14.967851 27957 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:14.996687 28177 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:14.975087 28401 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:14.928500 28552 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
W20260709 13:09:15.003443 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 45796.9us
I20260709 13:09:15.017195 21072 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 24: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:15.006449 28437 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:15.013293 28421 consensus_meta.cc:287] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.904s	user 0.016s	sys 0.005s
I20260709 13:09:15.006103 28603 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:09:15.016425 28576 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.916s	user 0.008s	sys 0.010s
W20260709 13:09:15.029178 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 62287.8us
I20260709 13:09:15.020601 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:15.032043 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 66505.1us
I20260709 13:09:15.012490 28589 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.015132 28541 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:15.071491 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.036223 28515 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:15.077159 21221 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:15.078410 28589 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:15.068183 28600 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:09:15.078905 28350 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.014s	user 0.012s	sys 0.000s
I20260709 13:09:15.041432 28605 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:15.049650 28576 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:15.080061 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.035247 28365 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:09:15.085024 28176 consensus_meta.cc:287] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.002s
I20260709 13:09:15.093384 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.115880 21219 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:09:15.093668 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 73178.8us
I20260709 13:09:15.095592 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2393) took 1497 ms. Trace:
I20260709 13:09:15.116983 21071 rpcz_store.cc:276] 0709 13:09:13.598091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.708186 (+1110095us) service_pool.cc:224] Handling call
0709 13:09:14.708522 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.708773 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:14.876439 (+167666us) log.cc:844] Serialized 14 byte log entry
0709 13:09:15.095104 (+218665us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:15.095176 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:15.095257 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.095364 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.095519 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":219964,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":3214,"raft.queue_time_us":4394,"thread_start_us":318928,"threads_started":2}
I20260709 13:09:15.119206 21071 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:15.119804 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1356) took 1470 ms. Trace:
I20260709 13:09:15.120082 28565 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:15.120102 21071 rpcz_store.cc:276] 0709 13:09:13.649656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.118718 (+1469062us) service_pool.cc:224] Handling call
0709 13:09:15.119023 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.119599 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.119756 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:15.120640 28565 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:15.121289 28565 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.100248 28420 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:15.123812 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:15.123610 28401 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.124656 28347 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:15.125990 28550 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.126101 28401 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.126768 28529 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:15.126886 28550 consensus_queue.cc:1059] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:15.127807 28529 raft_consensus.cc:3055] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:15.128166 28529 raft_consensus.cc:740] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:15.128960 28529 consensus_queue.cc:260] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.129631 27892 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.130144 28529 raft_consensus.cc:3060] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:15.123605 28355 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:15.101008 28408 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.000s
W20260709 13:09:15.117177 28144 consensus_meta.cc:287] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.668s	user 0.016s	sys 0.001s
I20260709 13:09:15.132053 28408 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.088065 28388 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.097240 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:15.064271 28598 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:09:15.036576 28421 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:15.105484 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.139412 28576 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:15.019464 28606 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 13:09:15.103000 28515 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.105805 28350 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.110412 28532 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.104660 28605 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:15.112533 28541 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:15.113664 28464 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.941s	user 0.011s	sys 0.000s
I20260709 13:09:15.140022 21035 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:15.114594 28558 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.132856 28144 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.140385 21064 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:15.143455 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.148564 21035 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:15.054098 28603 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:15.150574 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 75453.1us
I20260709 13:09:15.149724 28541 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:15.155020 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:15.156260 21035 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:09:15.162230 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 65112.4us
I20260709 13:09:15.163048 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:15.153877 28603 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:15.141930 28515 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.139469 28421 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.142465 28350 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.140187 28576 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.165411 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:15.256331 28576 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.257241 28603 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.142894 28605 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:15.148993 28565 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.197883 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:15.168519 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:15.160293 28558 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:15.164309 28177 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:15.166033 28541 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:15.249179 28096 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:15.249364 28144 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:15.251004 28388 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:15.160509 28401 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:15.143564 28464 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:15.165230 28408 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:15.283777 21068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60302 (request call id 1359) took 1572 ms (client timeout 2037 ms). Trace:
I20260709 13:09:15.337667 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:15.403108 28558 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:09:15.402966 21068 rpcz_store.cc:269] 0709 13:09:13.710900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.744787 (+ 33887us) service_pool.cc:224] Handling call
0709 13:09:15.283588 (+1538801us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b54003a2fd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:15.283712 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1729480576}
I20260709 13:09:15.404122 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:15.404336 28558 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.404793 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:15.405143 21067 raft_consensus.cc:2410] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:09:15.405542 21068 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 14.
I20260709 13:09:15.367398 21796 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:15.293845 27378 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:15.275678 28531 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:15.309537 28025 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:15.407490 28401 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.407790 28531 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.341799 28603 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:15.409996 28401 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:15.322131 28284 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.285113 27892 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:15.407974 28541 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:15.295619 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3215) took 3302 ms. Trace:
W20260709 13:09:15.276170 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 110823us
I20260709 13:09:15.327450 28471 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.412405 28541 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.413406 21110 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:15.413241 28471 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:15.342372 28597 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.418s	user 0.001s	sys 0.000s
I20260709 13:09:15.414170 28401 consensus_queue.cc:1059] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:15.415127 28401 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:09:15.293504 28499 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.963s	user 0.013s	sys 0.005s
W20260709 13:09:15.307219 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.416474 28284 raft_consensus.cc:3055] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:15.416227 28499 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.416893 28284 raft_consensus.cc:740] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:15.417430 28233 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.417714 28284 consensus_queue.cc:260] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.418130 28233 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.275672 28611 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.132s
I20260709 13:09:15.418967 28499 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:15.300614 28523 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.420662 21767 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:15.421703 21767 leader_election.cc:400] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 16. Current term is 17.
I20260709 13:09:15.422034 21767 leader_election.cc:403] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:09:15.421793 28572 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:15.422576 28572 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.423033 28572 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:15.424365 28233 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:15.424947 28233 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 16. Current term is 17.
I20260709 13:09:15.426174 28233 consensus_queue.cc:1059] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:15.427158 28233 raft_consensus.cc:3055] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:15.427541 28233 raft_consensus.cc:740] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:15.330317 21795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 1441) took 2495 ms (client timeout 2571 ms). Trace:
I20260709 13:09:15.166175 28595 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:15.437330 28595 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.298852 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39510 (request call id 2392) took 1702 ms. Trace:
I20260709 13:09:15.275532 27957 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:15.419006 28284 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:15.336746 21800 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:09:15.406803 21796 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:15.262665 28606 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:15.288674 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:15.273671 28527 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:15.291693 28082 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:09:15.296283 28493 consensus_meta.cc:287] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.650s	user 0.008s	sys 0.002s
I20260709 13:09:15.289134 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:15.452996 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.305881 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 3435) took 1161 ms. Trace:
I20260709 13:09:15.343845 28365 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:15.344961 28594 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.424s	user 0.002s	sys 0.000s
I20260709 13:09:15.433883 21215 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:15.358446 28552 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:09:15.259202 28576 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:15.435017 28464 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:15.324489 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1351) took 2078 ms. Trace:
I20260709 13:09:15.468804 21064 rpcz_store.cc:276] 0709 13:09:13.246075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.477248 (+1231173us) service_pool.cc:224] Handling call
0709 13:09:14.494688 (+ 17440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.152366 (+657678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.324105 (+171739us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54002ad5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:15.324399 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484423040}
W20260709 13:09:15.297606 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.452457 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1357) took 1800 ms. Trace:
I20260709 13:09:15.470323 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:15.439234 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.300800 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:15.337762 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 3233) took 1395 ms. Trace:
W20260709 13:09:15.475234 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 174515us
I20260709 13:09:15.475127 21063 rpcz_store.cc:276] 0709 13:09:13.942057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.195026 (+252969us) service_pool.cc:224] Handling call
0709 13:09:15.337570 (+1142544us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b54003c9e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:15.337704 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835808000}
W20260709 13:09:15.440958 21795 rpcz_store.cc:269] 0709 13:09:12.834423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.834583 (+   160us) service_pool.cc:224] Handling call
0709 13:09:15.330091 (+2495508us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b540047ead4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:15.330243 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2014791552}
I20260709 13:09:15.476920 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:15.477962 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:15.478278 21765 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:15.438910 21069 rpcz_store.cc:276] 0709 13:09:13.596073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.190906 (+594833us) service_pool.cc:224] Handling call
0709 13:09:15.298617 (+1107711us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b540041d054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:15.298787 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450503552}
I20260709 13:09:15.411972 21070 rpcz_store.cc:276] 0709 13:09:11.993145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.403728 (+2410583us) service_pool.cc:224] Handling call
0709 13:09:14.441654 (+ 37926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.608520 (+166866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.295338 (+686818us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54003cad54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:15.295551 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382121344}
I20260709 13:09:15.288123 21792 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:15.447830 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.406065 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:15.480561 21067 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:15.410415 28531 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.453995 21212 rpcz_store.cc:276] 0709 13:09:14.144091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.210664 (+ 66573us) service_pool.cc:224] Handling call
0709 13:09:15.305555 (+1094891us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54004434d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:15.305761 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785632512}
I20260709 13:09:15.457763 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.487600 21141 raft_consensus.cc:2410] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:09:15.451812 21796 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.473214 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.488294 21796 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:15.480571 21063 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 21. Ops: []
I20260709 13:09:15.450672 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.489236 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3223) took 1802 ms. Trace:
I20260709 13:09:15.489326 21139 raft_consensus.cc:2376] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20. Re-sending same reply.
I20260709 13:09:15.482822 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.491164 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.490485 21063 rpcz_store.cc:276] 0709 13:09:13.686665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.480077 (+1793412us) service_pool.cc:224] Handling call
0709 13:09:15.480411 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.488906 (+  8495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.489169 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:15.448705 28527 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:15.496258 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.452255 28082 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:09:15.498966 21072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 3229) took 1674 ms (client timeout 1811 ms). Trace:
I20260709 13:09:15.499114 21066 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:15.458873 28365 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:15.499486 21072 rpcz_store.cc:269] 0709 13:09:13.824961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.911367 (+ 86406us) service_pool.cc:224] Handling call
0709 13:09:15.498761 (+1587394us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540011b7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:15.498910 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441914240}
I20260709 13:09:15.454654 28595 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:15.459095 28606 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:15.523993 28471 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:15.540203 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:15.542080 21069 raft_consensus.cc:1240] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:15.542958 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2396) took 1883 ms. Trace:
I20260709 13:09:15.543381 21069 rpcz_store.cc:276] 0709 13:09:13.659891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.479260 (+1819369us) service_pool.cc:224] Handling call
0709 13:09:15.541849 (+ 62589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.542661 (+   812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.542891 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:15.532696 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2397) took 1805 ms. Trace:
I20260709 13:09:15.537215 21212 raft_consensus.cc:1240] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:15.547221 21064 rpcz_store.cc:276] 0709 13:09:13.727425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.501784 (+1774359us) service_pool.cc:224] Handling call
0709 13:09:15.531990 (+ 30206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.532235 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.532273 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:15.532337 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:15.532397 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.532472 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.532608 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:15.549715 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 102032us
I20260709 13:09:15.550702 21767 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:15.537833 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.552825 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.538763 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.554134 21219 raft_consensus.cc:2410] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:09:15.545544 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.554836 21146 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:15.492862 28515 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:15.492995 28452 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:15.291806 28320 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.507658 28508 raft_consensus.cc:2804] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:15.552649 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:15.557468 21070 raft_consensus.cc:1240] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 13:09:15.562371 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.562568 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3234) took 1566 ms. Trace:
I20260709 13:09:15.562906 21070 rpcz_store.cc:276] 0709 13:09:13.996408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.520633 (+1524225us) service_pool.cc:224] Handling call
0709 13:09:15.557153 (+ 36520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:15.562299 (+  5146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.562515 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:15.558405 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.563755 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.558780 21141 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 21. Ops: []
I20260709 13:09:15.564502 21213 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:15.564453 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.462703 28493 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.565651 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.566262 21138 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:15.563653 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.570504 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.571437 21147 raft_consensus.cc:2368] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25.
I20260709 13:09:15.576916 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:15.531352 28502 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.007s	sys 0.016s
I20260709 13:09:15.578017 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.579783 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.580137 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3433) took 1501 ms. Trace:
I20260709 13:09:15.580667 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.580515 21220 rpcz_store.cc:276] 0709 13:09:14.078496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.485589 (+407093us) service_pool.cc:224] Handling call
0709 13:09:14.485865 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:14.486223 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:14.486251 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:14.500319 (+ 14068us) log.cc:844] Serialized 26 byte log entry
0709 13:09:14.724048 (+223729us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:14.724108 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.724216 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:15.344326 (+620110us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:15.570199 (+225873us) raft_consensus.cc:1685] finished
0709 13:09:15.570222 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.580076 (+  9854us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38714,"raft.queue_time_us":2776,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":19415,"spinlock_wait_cycles":81058176,"thread_start_us":150441,"threads_started":1,"wal-append.queue_time_us":298426}
I20260709 13:09:15.574906 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.557742 28515 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.593140 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3237) took 1557 ms. Trace:
I20260709 13:09:15.593819 21070 rpcz_store.cc:276] 0709 13:09:14.036084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.563797 (+1527713us) service_pool.cc:224] Handling call
0709 13:09:15.592196 (+ 28399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.592462 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.592529 (+    67us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:15.592788 (+   259us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:15.592843 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.592926 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.593066 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:15.596033 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67459us
I20260709 13:09:15.597260 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.604902 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2376) took 3830 ms. Trace:
I20260709 13:09:15.605463 21066 rpcz_store.cc:276] 0709 13:09:11.774978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.201689 (+2426711us) service_pool.cc:224] Handling call
0709 13:09:15.498901 (+1297212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.503657 (+  4756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.604650 (+100993us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540011b7d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:15.604828 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2982597504}
I20260709 13:09:15.607250 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2398) took 1568 ms. Trace:
I20260709 13:09:15.608323 21066 rpcz_store.cc:276] 0709 13:09:14.038544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.606472 (+1567928us) service_pool.cc:224] Handling call
0709 13:09:15.606776 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.606954 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.606977 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:15.607016 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:15.607048 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.607098 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.607199 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:15.610069 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2399) took 1563 ms. Trace:
I20260709 13:09:15.610388 21066 rpcz_store.cc:276] 0709 13:09:14.046886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.609335 (+1562449us) service_pool.cc:224] Handling call
0709 13:09:15.609624 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.609785 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.609807 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:15.609845 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:15.609878 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.609925 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.610023 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:15.614054 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:15.582551 28502 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.007s	sys 0.016s
I20260709 13:09:15.615208 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:15.616724 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:15.620223 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 453506us
I20260709 13:09:15.647766 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.648561 21219 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 23: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:15.576400 28499 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:15.648343 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:15.653570 21212 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:09:15.611928 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2402) took 1491 ms. Trace:
I20260709 13:09:15.646560 21141 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 11.
I20260709 13:09:15.652727 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.581589 28508 raft_consensus.cc:697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:15.546324 28603 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:09:15.659061 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1481) took 1426 ms. Trace:
I20260709 13:09:15.659531 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.659497 21800 rpcz_store.cc:276] 0709 13:09:14.232319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.238597 (+  6278us) service_pool.cc:224] Handling call
0709 13:09:15.336288 (+1097691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.440834 (+104546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.658875 (+218041us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540047ead0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:15.659006 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2489223040}
I20260709 13:09:15.660425 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:15.529647 28233 consensus_queue.cc:260] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.666471 21065 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:15.607908 28515 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:15.663861 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.666229 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.668232 21220 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:15.665227 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:15.664855 21066 rpcz_store.cc:276] 0709 13:09:14.120536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.611156 (+1490620us) service_pool.cc:224] Handling call
0709 13:09:15.611471 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.611631 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.611654 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:15.611692 (+    38us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:15.611726 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.611772 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.611870 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:15.670658 21066 raft_consensus.cc:1240] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:15.671386 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2403) took 1533 ms. Trace:
W20260709 13:09:15.672166 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:15.672592 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.577219 27892 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:09:15.656476 28499 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:15.671767 21066 rpcz_store.cc:276] 0709 13:09:14.137458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.670087 (+1532629us) service_pool.cc:224] Handling call
0709 13:09:15.670448 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.671065 (+   617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.671319 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:15.690774 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.694662 21066 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:15.667686 28429 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:15.695473 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2404) took 1543 ms. Trace:
I20260709 13:09:15.696861 21139 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 25: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:15.652624 28531 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:09:15.673931 28177 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:15.717731 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.699180 21066 rpcz_store.cc:276] 0709 13:09:14.152471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.694238 (+1541767us) service_pool.cc:224] Handling call
0709 13:09:15.694497 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.695220 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.695426 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:15.701195 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.719040 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.719717 21066 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:15.720398 21139 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:09:15.681915 28408 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:15.699980 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.699714 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.723762 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2406) took 1196 ms. Trace:
W20260709 13:09:15.720139 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 248137us
I20260709 13:09:15.645676 28082 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:15.695230 28499 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:15.674062 27957 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:09:15.698645 28233 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:15.733088 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.734138 21145 raft_consensus.cc:2368] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
W20260709 13:09:15.737340 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 186707us
I20260709 13:09:15.728113 21220 raft_consensus.cc:1240] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:15.724187 21066 rpcz_store.cc:276] 0709 13:09:14.527819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.719307 (+1191488us) service_pool.cc:224] Handling call
0709 13:09:15.719556 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.723117 (+  3561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.723704 (+   587us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:15.717732 28177 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:15.745752 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.758656 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2447) took 1673 ms. Trace:
I20260709 13:09:15.759110 21215 rpcz_store.cc:276] 0709 13:09:14.085004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.202906 (+117902us) service_pool.cc:224] Handling call
0709 13:09:15.758465 (+1555559us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54004420d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:15.758597 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2829057280}
I20260709 13:09:15.761559 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.666441 28606 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.744833 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.768100 21069 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:15.771427 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.766848 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.774854 21220 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:09:15.775786 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.781106 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.781553 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:15.780371 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 290131us
I20260709 13:09:15.787106 21069 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 25.
I20260709 13:09:15.787358 21066 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:15.786543 21217 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:15.768069 28177 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:15.739965 28082 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:09:15.775743 28589 consensus_meta.cc:287] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.671s	user 0.008s	sys 0.002s
W20260709 13:09:15.778683 28616 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:09:15.804071 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.802125 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:15.785858 28614 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:09:15.804826 21147 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:15.805301 21220 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:15.807790 21138 raft_consensus.cc:2468] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:15.808678 21142 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:15.809937 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.810757 21138 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:15.812741 21066 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:15.779028 28320 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:15.829824 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:15.837003 21801 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:15.728761 28508 consensus_queue.cc:237] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.806728 28493 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:15.845052 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.850256 21145 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:09:15.852370 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.779243 28541 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:09:15.866605 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:15.867623 21145 raft_consensus.cc:2393] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:15.470539 21071 rpcz_store.cc:276] 0709 13:09:13.651951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.121290 (+1469339us) service_pool.cc:224] Handling call
0709 13:09:15.121559 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.121748 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:09:15.122058 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 13:09:15.452114 (+330056us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:15.452165 (+    51us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:09:15.452215 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.452278 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.452404 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":693,"prepare.queue_time_us":335,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":2173,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":9273,"spinlock_wait_cycles":20224,"thread_start_us":329720,"threads_started":1}
I20260709 13:09:15.807263 28589 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.827986 28558 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
W20260709 13:09:15.874415 28620 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:09:15.864596 28624 raft_consensus.cc:2804] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:15.887534 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1352) took 2641 ms. Trace:
I20260709 13:09:15.902158 21065 rpcz_store.cc:276] 0709 13:09:13.246422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.618236 (+1371814us) service_pool.cc:224] Handling call
0709 13:09:14.625921 (+  7685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.666948 (+1041027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.887299 (+220351us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400215054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:15.887467 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2392628992}
I20260709 13:09:15.888456 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:15.920563 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 258472us
W20260709 13:09:15.951092 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:15.957098 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.942175 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:15.958356 21145 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:15.883244 28493 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.943315 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.948386 21220 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
W20260709 13:09:15.961252 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 168883us
I20260709 13:09:15.961354 21217 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:15.955538 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.904150 28552 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:15.966440 28482 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:15.975878 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.976749 21215 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:15.980083 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:15.980926 21215 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:15.982596 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:15.988508 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:15.985276 21138 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:15.993628 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:15.996789 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:15.954309 28624 raft_consensus.cc:697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:16.025508 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:15.913117 28572 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:15.912683 28627 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 13:09:15.992623 28320 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:16.055123 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:16.045279 21219 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:16.054031 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:16.054382 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:16.016140 27892 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:16.057874 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:16.054710 21215 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:16.025722 28464 raft_consensus.cc:3060] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:16.064347 21798 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:16.070721 21797 raft_consensus.cc:2368] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:09:16.069800 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:15.971195 28493 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.074157 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:15.971990 28552 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.044358 28625 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:09:16.081529 28622 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:09:16.101866 28625 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:16.113543 28596 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 1.203s	user 0.004s	sys 0.019s
I20260709 13:09:16.114010 28572 leader_election.cc:290] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:16.116477 28599 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.200s	user 0.003s	sys 0.019s
I20260709 13:09:16.121233 28625 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:16.103746 28521 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:16.122404 28521 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.122789 28521 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:16.123553 28502 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:16.123956 28502 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:16.124795 28493 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:16.124745 28502 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.079917 28637 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 13:09:16.127627 28527 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.128916 28527 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:16.111369 28624 consensus_queue.cc:237] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.133440 28528 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:16.133886 28528 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:16.134701 28528 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.141744 28528 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:16.075376 28431 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 13:09:16.144480 28637 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:16.144917 28637 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:16.145242 28499 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.145808 28637 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.146471 28627 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:16.146886 28627 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:16.146974 28637 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:16.125984 28502 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:16.149468 28515 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:16.148106 28630 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:09:16.150478 28464 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:16.151185 28471 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:16.151743 28471 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:16.152742 28605 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.721s	user 0.006s	sys 0.004s
I20260709 13:09:16.152516 28471 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.153158 28605 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.153720 28471 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:16.139487 28589 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:16.148633 28625 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:16.147318 28627 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:16.155364 28552 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:16.149896 28621 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 13:09:16.171703 28499 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:16.155114 28521 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.183408 28521 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:16.249205 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.250969 21140 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:16.253351 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3238) took 2216 ms. Trace:
I20260709 13:09:16.255354 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.429531 28515 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:16.257262 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.261454 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:16.266503 21144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.016s	sys 0.003s
I20260709 13:09:16.266922 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:16.275162 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:16.292695 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 292690us
I20260709 13:09:16.296499 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2448) took 2203 ms. Trace:
W20260709 13:09:16.303952 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.304409 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54648 (request call id 1494) took 1022 ms. Trace:
W20260709 13:09:16.305176 21767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:16.305753 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 235921us
I20260709 13:09:16.306247 21800 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:09:16.308348 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.335245 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:16.337920 20964 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 11 to 18, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:16.352774 21064 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
W20260709 13:09:16.352887 28025 consensus_meta.cc:287] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.944s	user 0.012s	sys 0.000s
I20260709 13:09:16.361045 27957 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:16.365694 28401 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:16.196231 28605 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:16.155494 28625 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:16.437597 28401 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.278806 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:16.433055 21797 rpcz_store.cc:276] 0709 13:09:15.282094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.288375 (+  6281us) service_pool.cc:224] Handling call
0709 13:09:16.303734 (+1015359us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b5400682254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:16.304090 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1443568384}
W20260709 13:09:16.439880 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 161639us
I20260709 13:09:16.440935 28401 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:16.400457 28233 consensus_meta.cc:287] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.000s
I20260709 13:09:16.400488 21068 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:16.442023 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:16.443133 28589 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:16.443661 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 108881us
I20260709 13:09:16.444489 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:16.416838 28464 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:16.445998 28464 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.447258 21145 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 25.
W20260709 13:09:16.447715 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 55989.6us
I20260709 13:09:16.448643 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:16.449035 28617 consensus_queue.cc:1059] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:16.417465 28320 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:16.450333 28499 consensus_queue.cc:1059] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:16.451365 28320 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.451848 28499 raft_consensus.cc:3055] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:09:16.452327 21144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.017s	sys 0.003s
I20260709 13:09:16.452494 28644 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:09:16.454459 28320 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:16.452360 28499 raft_consensus.cc:740] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:16.395431 28284 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.919s	user 0.011s	sys 0.003s
I20260709 13:09:16.417146 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:16.422223 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3247) took 1032 ms. Trace:
I20260709 13:09:16.460752 20964 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 10 to 14, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 14 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:16.470527 28025 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:16.457319 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.460865 21217 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:16.454542 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.450254 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.437626 21218 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:16.461376 27378 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.466284 28521 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:16.456557 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.431784 21216 rpcz_store.cc:276] 0709 13:09:14.093279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.533162 (+439883us) service_pool.cc:224] Handling call
0709 13:09:14.533450 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:14.579840 (+ 46390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:14.579878 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:14.580362 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:09:14.655053 (+ 74691us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:14.655107 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.655251 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:16.071036 (+1415785us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:16.295609 (+224573us) raft_consensus.cc:1685] finished
0709 13:09:16.295659 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:16.296240 (+   581us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49456,"raft.queue_time_us":46778,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5562,"spinlock_wait_cycles":59268480,"thread_start_us":39324,"threads_started":1,"wal-append.queue_time_us":235636}
I20260709 13:09:16.446022 27892 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:16.480742 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3542) took 1031 ms. Trace:
I20260709 13:09:16.511452 27892 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:16.512360 27892 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:16.469853 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.429298 21070 rpcz_store.cc:276] 0709 13:09:14.036414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.595070 (+1558656us) service_pool.cc:224] Handling call
0709 13:09:15.658121 (+ 63051us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:15.658359 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.658385 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:15.664671 (+  6286us) log.cc:844] Serialized 26 byte log entry
0709 13:09:15.841158 (+176487us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:15.841204 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.841272 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:16.252608 (+411336us) raft_consensus.cc:1685] finished
0709 13:09:16.252651 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:16.253011 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":147001,"threads_started":1,"wal-append.queue_time_us":339194}
W20260709 13:09:16.511408 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 94664.6us
W20260709 13:09:16.402922 28177 consensus_meta.cc:287] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.587s	user 0.014s	sys 0.000s
I20260709 13:09:16.462903 28606 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.450104 28617 consensus_queue.cc:1243] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:09:16.517148 21141 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 27: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:16.473976 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.466060 28625 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.507758 28347 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.508154 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:16.510903 21142 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:16.511616 21140 rpcz_store.cc:276] 0709 13:09:15.449671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.449822 (+   151us) service_pool.cc:224] Handling call
0709 13:09:15.457908 (+  8086us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:16.429303 (+971395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:16.431005 (+  1702us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b540030b954. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:16.480659 (+ 49654us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1822142208}
I20260709 13:09:16.519775 21800 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:16.463826 28603 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:16.521271 28625 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:16.521034 28606 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.525466 28515 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:16.527514 28233 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:16.527817 28177 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.528458 28233 consensus_queue.cc:1243] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:09:16.521139 28347 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.530406 28177 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:16.530666 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:16.531436 28502 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:16.532166 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.504702 21069 rpcz_store.cc:276] 0709 13:09:15.389924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.806353 (+416429us) service_pool.cc:224] Handling call
0709 13:09:15.824642 (+ 18289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:15.825112 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.825153 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:15.825723 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:09:16.250619 (+424896us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:16.397161 (+146542us) log.cc:844] Serialized 14 byte log entry
0709 13:09:16.421527 (+ 24366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:16.421687 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:16.421804 (+   117us) raft_consensus.cc:1685] finished
0709 13:09:16.421815 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:16.422010 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22919,"raft.run_cpu_time_us":435,"raft.run_wall_time_us":7545,"thread_start_us":567728,"threads_started":2,"wal-append.queue_time_us":222226}
I20260709 13:09:16.533936 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:16.534893 21064 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:16.534724 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:16.535918 21068 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:16.537123 20964 catalog_manager.cc:5697] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 8 to 15, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 15 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:16.537752 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:16.538759 21065 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:16.539635 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:16.449046 28464 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:16.544064 21796 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:16.541455 28603 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:16.556945 21147 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.749s	user 0.014s	sys 0.010s
I20260709 13:09:16.557557 28347 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:16.557857 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78595.7us
I20260709 13:09:16.560774 21801 raft_consensus.cc:1240] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
W20260709 13:09:16.550470 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 78908.1us
I20260709 13:09:16.557588 21147 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:16.569736 21069 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
W20260709 13:09:16.579396 28365 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.987s	user 0.009s	sys 0.007s
I20260709 13:09:16.586400 28431 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:16.586978 28431 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:16.588191 28431 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.581434 21795 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:09:16.551533 28606 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:16.592895 21036 leader_election.cc:400] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:16.594455 28644 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:16.596705 21218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 3412) took 2858 ms (client timeout 2732 ms). Trace:
W20260709 13:09:16.597225 21218 rpcz_store.cc:269] 0709 13:09:13.737785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.741281 (+  3496us) service_pool.cc:224] Handling call
0709 13:09:16.596421 (+2855140us) spinlock_profiling.cc:243] Waited 2.7 s on lock 0x7b54002ee5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:16.596595 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6198663936}
I20260709 13:09:16.600205 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:16.601172 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.601676 21145 raft_consensus.cc:2410] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:09:16.603917 28611 raft_consensus.cc:3055] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:16.604472 28611 raft_consensus.cc:740] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:16.604612 28464 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:16.605809 28611 consensus_queue.cc:260] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.608246 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.609428 21141 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:16.612085 28603 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.613402 28603 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:16.610888 21141 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:16.617774 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.618458 21146 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:16.619308 28499 consensus_queue.cc:260] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.622037 27378 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.585753 28365 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.625519 21220 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:16.627465 21035 leader_election.cc:400] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:09:16.628018 21035 leader_election.cc:403] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:09:16.628327 28606 consensus_queue.cc:1059] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:16.629606 28177 raft_consensus.cc:3055] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:16.630179 28177 raft_consensus.cc:740] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:16.631629 28177 consensus_queue.cc:260] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.633677 28177 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:16.603080 21140 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
W20260709 13:09:16.635674 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 93985.7us
I20260709 13:09:16.636725 21140 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:16.652148 28611 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:16.585462 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:16.584479 20964 catalog_manager.cc:5697] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 15 to 18, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 18 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:16.662570 28365 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:16.664429 20963 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 169) took 18886 ms (client timeout 15000 ms). Trace:
W20260709 13:09:16.666148 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 149745us
I20260709 13:09:16.667032 21141 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:16.669055 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:16.682211 21184 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:16.682746 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.683736 27378 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:16.684600 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 139531us
I20260709 13:09:16.678025 20964 catalog_manager.cc:5697] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:16.583204 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:16.664233 28082 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.851s	user 0.013s	sys 0.009s
W20260709 13:09:16.685105 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 57093.5us
W20260709 13:09:16.665757 28528 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.523s	user 0.015s	sys 0.001s
I20260709 13:09:16.687403 21064 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:16.638330 21140 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:09:16.687449 21144 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:16.688584 28499 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:16.690852 28025 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:16.660161 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:16.691984 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52868.9us
I20260709 13:09:16.658300 28627 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.695196 21068 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 26.
W20260709 13:09:16.687749 28527 consensus_meta.cc:287] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.010s
W20260709 13:09:16.699090 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 237037us
I20260709 13:09:16.703656 28576 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.713013 21138 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:16.704686 21146 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
W20260709 13:09:16.722818 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.698292 28589 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
W20260709 13:09:16.728155 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.729995 28365 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:16.730582 28365 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:16.731009 28365 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:16.741791 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 59779.6us
I20260709 13:09:16.743338 21184 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:16.741580 28662 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:16.746335 28662 consensus_queue.cc:1243] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 17.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:09:16.745990 28552 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:09:16.746920 28662 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:16.747651 28662 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:16.748152 28637 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:16.748594 28662 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.749766 28637 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.750217 28662 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:16.750398 28637 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.761816 21142 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:16.767484 21147 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:16.765956 28431 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:16.773047 28082 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.768707 28347 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:16.783646 28589 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:09:16.802579 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 142914us
I20260709 13:09:16.806237 28589 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:16.772517 28527 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:09:16.811482 28589 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:16.772281 27892 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:16.823559 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 121198us
I20260709 13:09:16.811004 28667 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.824434 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3565) took 1028 ms. Trace:
I20260709 13:09:16.828315 21147 rpcz_store.cc:276] 0709 13:09:15.793727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.576918 (+783191us) service_pool.cc:224] Handling call
0709 13:09:16.762560 (+185642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:16.767975 (+  5415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:16.807132 (+ 39157us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b5400259ed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:16.821814 (+ 14682us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400259ed0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:16.822005 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398398592}
W20260709 13:09:16.665689 20963 rpcz_store.cc:269] 0709 13:08:57.965458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.003476 (+ 38018us) service_pool.cc:224] Handling call
0709 13:09:16.663470 (+18659994us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:07.540274 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:07.541229 (+   955us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:07.541317 (+    88us) write_op.cc:435] Acquired schema lock
0709 13:09:07.541339 (+    22us) tablet.cc:662] Decoding operations
0709 13:09:07.615098 (+ 73759us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:07.632423 (+ 17325us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:07.656392 (+ 23969us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:09:07.657931 (+  1539us) tablet.cc:701] Row locks acquired
0709 13:09:07.657985 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:09:07.658166 (+   181us) write_op.cc:270] Start()
0709 13:09:07.658325 (+   159us) write_op.cc:276] Timestamp: P: 1783602547658082 usec, L: 0
0709 13:09:07.658362 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:09:07.659196 (+   834us) log.cc:844] Serialized 7226 byte log entry
0709 13:09:08.427666 (+768470us) op_driver.cc:464] REPLICATION: finished
0709 13:09:08.699714 (+272048us) write_op.cc:301] APPLY: starting
0709 13:09:08.715064 (+ 15350us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:09.226014 (+510950us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:09.226139 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:09.702378 (+476239us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:09.703024 (+   646us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:09:09.703080 (+    56us) write_op.cc:312] APPLY: finished
0709 13:09:09.703623 (+   543us) log.cc:844] Serialized 164 byte log entry
0709 13:09:09.859652 (+156029us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:09.875307 (+ 15655us) write_op.cc:454] Released schema lock
0709 13:09:09.896393 (+ 21086us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:09.909032 (+ 12639us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:09.909066 (+    34us) write_op.cc:454] Released schema lock
0709 13:09:10.097773 (+188707us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
Metrics: {"mutex_wait_us":3994699,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":217651,"apply.run_cpu_time_us":50480,"apply.run_wall_time_us":1225011,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":720079,"prepare.run_cpu_time_us":13647,"prepare.run_wall_time_us":399369,"raft.queue_time_us":312562,"raft.run_cpu_time_us":47521,"raft.run_wall_time_us":112913,"replication_time_us":769071,"spinlock_wait_cycles":9714432,"thread_start_us":1014511,"threads_started":5,"wal-append.queue_time_us":462941,"wal-append.run_cpu_time_us":23646,"wal-append.run_wall_time_us":2645615}]]}
I20260709 13:09:16.778733 28576 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.772858 20964 catalog_manager.cc:5697] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 14 to 17, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 17 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:16.833098 28625 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:09:16.849100 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 126973us
I20260709 13:09:16.854743 28552 consensus_queue.cc:1059] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:16.856037 20964 catalog_manager.cc:5697] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:16.824522 28082 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:16.858096 20964 catalog_manager.cc:5697] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 12 to 16, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 16 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } }
W20260709 13:09:16.901968 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.900635 21767 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:16.859825 28637 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:16.921387 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.922613 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:16.923573 21068 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:16.924412 28320 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.924058 20964 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 12 to 14, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 14 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:16.925112 28320 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.926628 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:16.859068 28667 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.929183 28663 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:16.931620 28552 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.932217 28552 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.932605 28625 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.935180 28082 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:16.924058 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:16.936631 21064 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:16.893210 28627 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:16.905371 28527 raft_consensus.cc:3055] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:16.938120 28527 raft_consensus.cc:740] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:16.938402 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:16.939021 21064 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:16.939121 28527 consensus_queue.cc:260] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.945111 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:16.945868 21070 raft_consensus.cc:3060] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:16.951213 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50677.8us
I20260709 13:09:16.952311 21767 leader_election.cc:304] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:16.954201 28515 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:16.954874 28515 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 26 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:16.955493 28627 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:16.956141 28627 raft_consensus.cc:697] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:16.957316 28627 consensus_queue.cc:237] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.963716 28515 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:16.926931 20964 catalog_manager.cc:5697] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 9 to 15, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 15 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:16.979240 28527 raft_consensus.cc:3060] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:16.986874 28177 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:17.001956 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 118912us
I20260709 13:09:16.989372 28625 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:17.019408 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.020257 21067 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
W20260709 13:09:17.023461 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 102545us
W20260709 13:09:17.025499 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.030022 21068 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:16.995615 28611 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.000804 28320 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.028015 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.040215 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:17.051496 28552 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.052232 27892 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:17.054510 28365 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:17.055267 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.059661 21067 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:09:17.060235 21067 pending_rounds.cc:85] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 15
I20260709 13:09:17.063124 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.063784 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:17.064055 21063 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.065707 21072 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:17.065971 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.067443 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.071635 28667 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.071516 20964 catalog_manager.cc:5697] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 13 to 16, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 16 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:09:17.074838 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:17.075304 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.073623 20964 catalog_manager.cc:5697] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 8 to 12, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:17.080250 27378 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.086267 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:17.060554 21067 pending_rounds.cc:107] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:09:17.087808 21067 raft_consensus.cc:2889] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:17.088413 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.022490 28528 consensus_queue.cc:1059] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:09:17.024359 28177 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.089218 27378 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.087200 21065 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:17.096642 28431 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:17.097317 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 57187.2us
I20260709 13:09:17.098145 28637 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:17.098795 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.102020 21070 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:17.106516 21064 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:17.091109 28611 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:17.114543 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:17.115505 28611 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:09:17.118132 21217 raft_consensus.cc:1240] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:17.109490 28637 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.119791 28637 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:17.120028 21216 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:17.119814 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:17.121069 21214 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:17.121737 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:17.122628 21213 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
W20260709 13:09:17.123731 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.123587 21071 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:09:17.124686 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:17.124986 28233 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:17.125735 21219 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:17.125646 28515 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:09:17.088661 28464 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:17.079381 28617 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:17.091041 28606 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.137264 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.130928 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:17.139345 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.123972 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:17.126950 28233 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:17.147917 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:17.148559 21147 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:17.142377 20964 catalog_manager.cc:5697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:17.151441 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:17.152107 21146 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:17.147686 21221 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:17.148536 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:17.153910 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.157239 21071 raft_consensus.cc:1240] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:17.155252 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:17.110749 28677 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:09:17.145164 28233 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:17.197607 21065 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:09:17.198664 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.206692 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.208341 21066 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 23. Current term is 24. Ops: []
W20260709 13:09:17.204584 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 170814us
I20260709 13:09:17.196138 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.147879 28365 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.195669 27378 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.226403 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.199605 20964 catalog_manager.cc:5697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 9 to 16, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 16 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:17.226446 27378 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:17.161947 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:17.199666 21068 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:17.155252 27892 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:17.199987 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:17.227957 21064 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:17.230746 21146 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 26.
I20260709 13:09:17.224970 21147 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
W20260709 13:09:17.232585 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 117010us
W20260709 13:09:17.187565 28664 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.227s	user 0.005s	sys 0.000s
I20260709 13:09:17.227859 27378 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.231739 21800 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:09:17.235704 21138 raft_consensus.cc:2368] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:09:17.148180 28606 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:17.210661 28603 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.597s	user 0.015s	sys 0.004s
W20260709 13:09:17.245360 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.222543 28233 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:17.239775 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.197175 28177 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.197402 28662 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:17.247318 21070 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:17.183890 28605 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:17.189944 28576 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.147795 28617 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.231874 27892 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:17.257637 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.255864 20964 catalog_manager.cc:5697] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 9 to 13, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 13 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:17.257000 27378 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.263317 28605 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.266098 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:17.266935 21217 raft_consensus.cc:2368] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:09:17.267549 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.268334 21066 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:17.269943 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:17.271683 21214 raft_consensus.cc:1275] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:17.272610 28431 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:17.273070 28431 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:17.273272 21215 raft_consensus.cc:1275] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:17.273660 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.274422 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.275405 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.268536 28605 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:17.268534 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.021s	sys 0.064s
W20260709 13:09:17.277268 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.021s	sys 0.064s
I20260709 13:09:17.274096 28431 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.284265 20964 catalog_manager.cc:5697] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 9 to 14, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 14 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:17.292841 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:09:17.297370 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.305480 21212 raft_consensus.cc:1240] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:17.305802 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.307029 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.307950 21072 raft_consensus.cc:2393] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.308753 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.309837 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.310390 21072 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
I20260709 13:09:17.311548 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:17.312295 28679 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:09:17.287783 28677 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:09:17.315590 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.316473 21065 raft_consensus.cc:2393] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.319550 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.256567 28177 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:17.320185 20964 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:17.321662 20964 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 11 to 14, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 14 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:17.305552 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:17.324311 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71181us
I20260709 13:09:17.312435 21063 raft_consensus.cc:2393] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 24: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.311194 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.325191 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.330356 21064 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:17.332032 21216 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:17.335851 21068 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:17.335733 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.337013 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.337682 21064 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.283943 27892 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:17.339039 21071 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.340211 21072 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:09:17.341123 21065 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:17.346961 28233 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.347661 21185 leader_election.cc:400] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25.
I20260709 13:09:17.387549 21185 leader_election.cc:403] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:17.389092 28233 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:17.391585 28365 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25.
I20260709 13:09:17.348130 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.342971 28617 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:17.355448 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:17.356591 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 687689us
I20260709 13:09:17.394474 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:17.396209 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:17.396519 21113 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:17.397004 28177 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:17.397346 28685 consensus_queue.cc:1059] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:17.397881 28177 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:17.398406 28177 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:17.398600 28605 raft_consensus.cc:3055] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:17.399050 28605 raft_consensus.cc:740] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:17.399185 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:17.399611 28667 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:17.400035 28667 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:17.399958 28605 consensus_queue.cc:260] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.400710 28685 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:17.401149 28685 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:17.401525 28605 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:17.402408 28431 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:17.401564 28685 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:17.404289 27378 consensus_queue.cc:1059] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:17.404433 28617 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:17.405083 27378 raft_consensus.cc:3055] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:17.405426 27378 raft_consensus.cc:740] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:17.406275 27378 consensus_queue.cc:260] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.407320 28656 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:17.407425 28177 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.408726 27378 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:17.358958 28606 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:17.359869 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 206961us
I20260709 13:09:17.363709 21063 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:17.364656 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.410665 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:17.365058 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:17.390847 21185 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:17.365583 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:17.366353 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.367455 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.374547 21072 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:17.374958 21765 leader_election.cc:400] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:09:17.415555 21765 leader_election.cc:403] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:17.291911 28515 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.084s
I20260709 13:09:17.277176 28662 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.377036 28611 raft_consensus.cc:3055] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:17.325635 28603 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:17.378736 28637 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.382742 20964 catalog_manager.cc:5697] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:17.408483 27892 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.355396 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.421411 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.403117 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:17.422664 21070 raft_consensus.cc:2410] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 13:09:17.400641 28667 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:17.425658 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:17.426169 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:17.429615 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54779.9us
I20260709 13:09:17.434551 21798 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:17.437278 21801 raft_consensus.cc:2393] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:17.417042 28611 raft_consensus.cc:740] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:17.420707 28603 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.433028 20964 catalog_manager.cc:5697] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 13 to 18, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 18 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:17.433300 28656 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:17.449978 28499 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:17.452840 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 313780us
I20260709 13:09:17.438752 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:17.459101 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 48542.5us
I20260709 13:09:17.460302 21072 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.468369 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:17.470999 21800 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:17.432997 28606 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:17.473541 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 82770us
I20260709 13:09:17.424896 28464 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:09:17.451952 28603 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:17.471890 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:17.488515 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.487859 21063 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 25: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.455305 28656 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:17.456904 28611 consensus_queue.cc:260] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.490986 28656 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:17.493038 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.461694 28499 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:17.478068 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.495824 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:17.501956 21066 raft_consensus.cc:2393] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
W20260709 13:09:17.503855 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 100839us
I20260709 13:09:17.467303 28233 consensus_queue.cc:1048] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:09:17.477617 28617 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:17.510859 21798 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:09:17.503525 20964 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 15 to 16, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 16 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:17.457641 28521 consensus_queue.cc:1048] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:09:17.502463 28499 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:17.523416 21185 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:17.507586 28611 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:17.522377 20964 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 12 to 18, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 18 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:17.514182 28617 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.536388 20964 catalog_manager.cc:5697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 11 to 14, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 14 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:17.534543 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.536082 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:17.516103 28688 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:09:17.543753 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 71973.4us
I20260709 13:09:17.540112 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:17.532718 28365 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:17.543618 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:17.550022 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.548491 21113 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:17.552174 21066 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:17.535779 28617 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:17.553296 21110 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:17.543339 28688 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.555840 20964 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 11 to 13, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 13 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:17.545401 28624 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:09:17.565233 20964 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 13 to 16, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 16 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:17.568974 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.559983 28464 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:09:17.572901 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 65271.9us
I20260709 13:09:17.572415 28696 raft_consensus.cc:2804] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:17.560321 28688 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:09:17.579005 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 48978.1us
I20260709 13:09:17.576505 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:17.560846 28606 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:17.561906 28672 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.582587 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:17.583156 28552 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:17.591142 21067 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:17.591516 21111 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:17.567121 28177 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:17.571868 27892 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:17.573119 28637 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:17.598492 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 45255us
I20260709 13:09:17.576006 28464 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:17.580986 28696 raft_consensus.cc:697] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:17.596338 28677 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.604046 28576 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:17.619196 27378 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:17.603726 28605 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:17.606421 28464 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:17.601522 28606 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:17.653942 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 117921us
I20260709 13:09:17.622474 27378 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:17.632184 28690 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:09:17.634639 21185 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:17.637965 28662 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:17.654772 27378 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:17.674309 21065 raft_consensus.cc:2468] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
W20260709 13:09:17.651182 28694 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.072s	user 0.004s	sys 0.000s
W20260709 13:09:17.683539 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.617578 28677 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:17.684803 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 78631.9us
I20260709 13:09:17.616071 28624 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:09:17.604493 28672 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.686955 21036 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:17.671890 28605 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:17.690443 28677 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:17.690310 28499 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.691129 28662 raft_consensus.cc:2804] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:17.691982 28662 raft_consensus.cc:697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:17.693055 28662 consensus_queue.cc:237] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.693882 28617 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.689350 28605 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.700958 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:17.701648 28365 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:17.701810 27892 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:17.703141 28606 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.704864 28603 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.622879 28177 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:09:17.706000 28617 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:17.704876 21067 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:09:17.708918 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.709453 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:17.713761 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 132728us
W20260709 13:09:17.719017 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:17.737342 28695 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:09:17.737741 28527 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:17.738314 28527 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:17.739042 27892 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:09:17.721947 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 123433us
I20260709 13:09:17.737185 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.738801 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:17.738552 28696 consensus_queue.cc:237] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.729931 27378 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.726703 28499 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:17.746171 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.750345 28703 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:17.750761 28703 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:17.751555 28703 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.752852 28320 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:17.753669 28320 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:17.755226 28528 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:17.755856 28528 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:17.752966 28177 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:17.757718 28177 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.757882 28431 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:17.759848 28177 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:17.762884 28528 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:17.766016 28703 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:17.771256 21187 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:17.783959 28617 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:17.787127 28617 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.789288 28617 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.739977 27892 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:17.790781 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 184496us
I20260709 13:09:17.703009 28656 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.791327 27892 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.792623 27892 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:17.793370 28656 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:17.801229 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.802170 21067 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:09:17.806243 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 70813.3us
I20260709 13:09:17.748106 28365 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:09:17.809056 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.810987 27378 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:17.817436 21765 leader_election.cc:400] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 17 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18.
I20260709 13:09:17.784730 28702 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:17.820034 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 139027us
I20260709 13:09:17.818156 28617 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.823479 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.824105 21798 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:17.824929 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:17.825099 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:17.735946 28685 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.829519 28685 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:17.804962 28499 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:17.830981 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:17.836463 28627 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:17.836697 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.837177 28627 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.838116 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.838704 21063 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:09:17.810595 28365 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:17.839359 28637 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:17.839550 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.840440 21071 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.840482 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:17.843945 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 89115.7us
I20260709 13:09:17.845494 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.846292 21068 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:17.836625 28702 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.854231 28702 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:17.856981 28692 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 13:09:17.754930 28320 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.839046 28626 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:17.841415 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.739255 28527 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.879046 21146 raft_consensus.cc:1275] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:09:17.879778 21071 raft_consensus.cc:2393] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.884193 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.885847 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:17.834811 28431 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.831301 28617 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:17.879958 21146 pending_rounds.cc:85] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 12
I20260709 13:09:17.795868 28688 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.880810 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:17.893786 21146 pending_rounds.cc:107] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:09:17.894124 21146 raft_consensus.cc:2889] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:17.896526 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.897593 28431 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:17.899976 28431 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:17.903412 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:09:17.904309 28698 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:09:17.904944 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:17.905565 21140 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:17.906193 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:17.906530 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:17.906836 21143 raft_consensus.cc:3060] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:09:17.907250 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:17.907939 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 43954.8us
I20260709 13:09:17.908058 21140 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:09:17.908627 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:17.909564 21140 raft_consensus.cc:1275] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:17.912451 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:17.913630 27378 consensus_queue.cc:1059] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:17.920526 21110 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:17.926167 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.827426 28699 raft_consensus.cc:2804] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:17.931362 28656 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:17.933961 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:17.934990 21066 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:17.910772 28528 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:17.858201 28464 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:17.938992 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.941685 21064 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:17.942400 21767 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:17.940070 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.919569 28685 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:17.950337 21067 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
W20260709 13:09:17.961460 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 121067us
I20260709 13:09:17.963156 21217 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:17.963375 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:17.965029 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:17.965667 21066 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:09:17.979936 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:17.981356 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:09:18.063974 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.940012 28528 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:17.925300 28611 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 13:09:17.979189 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:18.069828 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:17.956359 28685 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.071779 21212 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
W20260709 13:09:18.071485 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 146785us
I20260709 13:09:18.081106 21218 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:18.081106 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:18.080289 21064 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:09:18.066977 28233 raft_consensus.cc:2804] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:18.071103 28528 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:18.071960 28672 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:18.077016 28685 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:18.129179 21801 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
W20260709 13:09:18.160766 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 240336us
I20260709 13:09:18.116638 28603 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:18.200728 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:18.206094 21143 raft_consensus.cc:2468] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:18.102661 28320 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:18.207959 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 71108.2us
I20260709 13:09:18.133836 28637 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:09:18.229943 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 136687us
W20260709 13:09:18.250203 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:18.253337 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52731.3us
I20260709 13:09:18.235518 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:18.256742 21064 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:18.161191 28700 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:18.260833 21063 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:18.264909 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:09:18.274436 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 43650.7us
I20260709 13:09:18.261148 28177 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:18.262202 28700 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:18.241058 28320 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:18.348387 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:18.437311 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 159503us
I20260709 13:09:18.256137 28637 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:18.466070 21068 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
W20260709 13:09:18.417429 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 438322us
I20260709 13:09:18.269479 28715 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:18.489598 21064 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
W20260709 13:09:18.535395 21142 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.018s	sys 0.039s
W20260709 13:09:18.535930 21142 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.018s	sys 0.039s
I20260709 13:09:18.396879 28464 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:18.594866 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 148239us
I20260709 13:09:18.432964 28233 raft_consensus.cc:697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:18.460012 28701 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:18.397370 28712 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:09:18.468938 27892 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.676s	user 0.009s	sys 0.003s
I20260709 13:09:18.460572 28688 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:18.400234 28700 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:18.580811 28690 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:09:18.433890 28706 consensus_queue.cc:1059] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.505s
I20260709 13:09:18.610514 28677 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.597615 28672 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:09:18.629663 28464 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:18.672645 28320 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:18.659247 28707 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.692724 28718 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.693503 28718 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:18.695959 28729 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.696713 28729 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:18.697980 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:18.702514 28713 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.418s	user 0.000s	sys 0.001s
W20260709 13:09:18.703456 28720 log.cc:927] Time spent T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:09:18.704082 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:18.704744 21792 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:18.709479 21035 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:18.679491 27892 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.712137 28729 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:18.723616 28711 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.567s	user 0.000s	sys 0.002s
I20260709 13:09:18.725833 28702 raft_consensus.cc:3055] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:09:18.733801 28667 consensus_meta.cc:287] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 1.298s	user 0.010s	sys 0.001s
I20260709 13:09:18.827620 28667 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:18.830435 28667 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:18.831539 21072 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:18.636580 28715 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:18.834793 28715 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:18.716282 28707 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:18.736280 28716 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.520s	user 0.003s	sys 0.000s
I20260709 13:09:18.738279 28656 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:09:18.743078 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:18.655987 28677 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:18.749392 21767 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:18.749908 28734 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.750202 28718 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:18.756518 28365 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:18.764305 21139 consensus_meta.cc:287] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.810s	user 0.009s	sys 0.002s
I20260709 13:09:18.784305 21801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 961) took 1290 ms. Trace:
I20260709 13:09:18.709084 28696 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:18.916038 21801 rpcz_store.cc:276] 0709 13:09:17.470079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.470259 (+   180us) service_pool.cc:224] Handling call
0709 13:09:18.759876 (+1289617us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b5400701a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:18.760000 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537385216}
I20260709 13:09:18.813704 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:18.825053 28521 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:18.708480 28627 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:18.836795 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1389) took 1068 ms. Trace:
W20260709 13:09:18.942508 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 350732us
I20260709 13:09:18.886494 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:18.945400 21068 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:18.892586 21063 raft_consensus.cc:3055] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:18.946653 21063 raft_consensus.cc:740] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:09:18.785746 28703 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.019s	user 0.004s	sys 0.009s
I20260709 13:09:18.887605 21113 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:18.908772 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:18.892554 21187 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:18.912727 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 163396us
I20260709 13:09:18.915999 21139 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:18.950139 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 1090) took 1044 ms. Trace:
I20260709 13:09:18.917362 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:18.923446 21070 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:18.925308 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:18.804373 28702 raft_consensus.cc:740] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:18.748051 28528 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.609s	user 0.010s	sys 0.000s
I20260709 13:09:18.895476 28656 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.951581 21071 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:18.784438 27892 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:18.950589 21139 rpcz_store.cc:276] 0709 13:09:17.905374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.918210 (+ 12836us) service_pool.cc:224] Handling call
0709 13:09:18.950055 (+1031845us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:18.942247 21065 rpcz_store.cc:276] 0709 13:09:17.768871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.769198 (+   327us) service_pool.cc:224] Handling call
0709 13:09:17.797749 (+ 28551us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:17.809810 (+ 12061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:17.809848 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:17.815669 (+  5821us) log.cc:844] Serialized 52 byte log entry
0709 13:09:17.916051 (+100382us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:17.916110 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:17.916210 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:18.836469 (+920259us) raft_consensus.cc:1685] finished
0709 13:09:18.836496 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:18.836697 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6510,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":99911,"threads_started":1,"wal-append.queue_time_us":108839}
I20260709 13:09:18.896394 28667 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:18.937098 28624 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:09:18.953308 21215 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.031s	sys 0.064s
W20260709 13:09:18.965974 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 93374.6us
I20260709 13:09:18.707841 28233 consensus_queue.cc:237] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:18.967298 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 79744us
I20260709 13:09:18.921639 28745 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:18.959882 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67394.4us
W20260709 13:09:18.985252 21215 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.032s	sys 0.065s
W20260709 13:09:18.985056 21220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.006s	sys 0.004s
I20260709 13:09:18.941857 28637 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:18.987972 21766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.000s	sys 0.011s
W20260709 13:09:18.995962 21766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.000s	sys 0.011s
W20260709 13:09:18.992511 21220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.007s	sys 0.004s
I20260709 13:09:18.989722 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:18.915808 28734 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:18.968243 28672 raft_consensus.cc:2764] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:09:19.001262 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1023) took 1053 ms. Trace:
I20260709 13:09:18.999155 21187 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:18.961964 28656 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:18.982761 28365 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:18.999745 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.002s	sys 0.010s
I20260709 13:09:18.986016 28701 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:18.983970 28624 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:18.998342 28685 consensus_meta.cc:287] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.833s	user 0.011s	sys 0.000s
W20260709 13:09:18.988577 28699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.004s	sys 0.038s
I20260709 13:09:18.988169 28606 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.006636 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:19.009490 21067 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.002s	sys 0.011s
I20260709 13:09:19.004503 21063 consensus_queue.cc:260] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:19.012706 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.002s	sys 0.010s
W20260709 13:09:19.037211 21067 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.002s	sys 0.012s
W20260709 13:09:19.000491 28605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.000s	sys 0.011s
I20260709 13:09:18.986055 28736 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:18.941831 28521 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.002019 28729 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:19.034104 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.048241 27378 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.048739 27378 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:18.987154 28667 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.038733 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:19.041394 21184 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.017s	sys 0.018s
I20260709 13:09:19.003214 28700 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:19.034008 28699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.004s	sys 0.039s
W20260709 13:09:19.040746 28605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.000s	sys 0.012s
I20260709 13:09:19.040069 21113 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:19.032600 28624 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:19.014746 28703 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.060559 28703 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.063880 28703 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.034152 28710 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:19.065973 28710 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.066394 28710 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:19.068235 28741 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:09:19.033586 28701 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:19.072221 28701 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:18.962119 28528 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.035900 28732 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.073673 28732 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.053154 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.076475 28732 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.077243 28732 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:09:19.051542 21184 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.017s	sys 0.018s
I20260709 13:09:19.041950 21063 raft_consensus.cc:3060] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:19.079641 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79804.5us
I20260709 13:09:19.080521 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:19.081717 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.082840 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.014564 28603 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:09:19.083508 28527 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:19.084096 28527 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:19.050478 28731 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:19.084951 28527 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:19.085644 28710 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.085760 28731 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.087100 28672 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:19.057637 28699 raft_consensus.cc:697] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:19.056840 28721 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:19.064991 28611 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:19.040373 28718 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:19.056797 28739 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:18.995349 28745 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:19.100695 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 391272us
I20260709 13:09:19.035333 28685 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.118870 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:19.123896 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 114686us
I20260709 13:09:19.129710 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.041219 28606 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.134791 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:19.017768 28702 consensus_queue.cc:260] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.055398 28736 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:19.144042 21214 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:19.146196 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.157392 21217 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:09:19.157727 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.135094 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.143853 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.150612 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.100268 28742 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:19.090067 28611 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.155272 27378 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.079042 28732 consensus_queue.cc:1048] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:09:19.151464 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:19.115974 28729 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:19.170835 21218 raft_consensus.cc:2368] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22.
I20260709 13:09:19.170818 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.158360 21213 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:19.174569 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:19.178526 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 95748.5us
I20260709 13:09:19.180284 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:19.180950 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:19.181625 21213 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:19.182124 21212 raft_consensus.cc:1240] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:19.184089 28736 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.184980 28521 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.187844 28729 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.188292 21212 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:09:19.135912 28718 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:19.202915 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 149858us
I20260709 13:09:19.217036 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:19.177210 28611 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:19.144573 28755 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:19.229552 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.231254 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.179493 28667 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:19.200914 28710 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:19.236169 21214 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
W20260709 13:09:19.231576 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 80898.8us
I20260709 13:09:19.238585 28750 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:09:19.238719 21221 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:19.216794 28742 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:19.216773 28739 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:19.240226 21185 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.249673 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:19.250214 28707 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.250312 28742 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.250504 21185 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.252261 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:19.253031 28742 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:19.253190 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2442) took 1432 ms. Trace:
I20260709 13:09:19.253479 28749 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:19.253700 21072 rpcz_store.cc:276] 0709 13:09:17.820489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.820604 (+   115us) service_pool.cc:224] Handling call
0709 13:09:17.836808 (+ 16204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:18.832078 (+995270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.252934 (+420856us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b54000f9e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:19.253124 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2287082240}
I20260709 13:09:19.254304 28715 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:19.255270 28753 consensus_queue.cc:1048] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:09:19.255843 28719 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.256441 28719 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.256822 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:19.258865 28749 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:19.202392 28701 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:19.259697 28701 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.262015 28701 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:19.217875 27892 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:19.219544 28464 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:19.220008 28677 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:19.248021 21212 raft_consensus.cc:1240] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 14. Current term is 15. Ops: []
W20260709 13:09:19.248523 28744 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:09:19.248713 21214 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:09:19.249032 27378 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.235353 28731 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.120994 28721 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.239516 28718 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:19.263375 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.250918 28707 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:19.231125 28656 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.245174 21213 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:19.264786 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.282418 28746 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.258965 28719 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:19.269114 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.283478 28719 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:19.283694 21221 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
W20260709 13:09:19.273041 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.275640 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.274895 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.285193 21212 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
W20260709 13:09:19.276572 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.285569 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:19.236656 28662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.285s	user 0.018s	sys 0.029s
W20260709 13:09:19.286502 28662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.019s	sys 0.029s
I20260709 13:09:19.283970 28719 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.287047 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.287357 28719 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:19.288616 21036 leader_election.cc:400] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18.
I20260709 13:09:19.289631 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.278489 28464 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.290822 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.269349 27378 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.292815 27378 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:19.294308 28752 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:09:19.299851 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.301568 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.268744 28715 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.302860 28715 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.305013 28715 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:19.283023 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.285131 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1020) took 1342 ms. Trace:
I20260709 13:09:19.313030 21111 leader_election.cc:400] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22.
I20260709 13:09:19.313503 21111 leader_election.cc:403] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:19.274643 28742 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:19.317175 27378 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:09:19.274407 28734 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.334113 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:19.334564 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2451) took 1363 ms. Trace:
I20260709 13:09:19.309994 21219 rpcz_store.cc:276] 0709 13:09:17.942831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.943978 (+  1147us) service_pool.cc:224] Handling call
0709 13:09:18.070028 (+126050us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:18.115322 (+ 45294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:18.115360 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:18.741021 (+625661us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b2c0011d108. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f40e15a11 00007f7f40e25372 00007f7f42557a01 00007f7f4251c43a 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2
0709 13:09:18.741543 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:09:19.036602 (+295059us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:19.036642 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.036729 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:19.284830 (+248101us) raft_consensus.cc:1685] finished
0709 13:09:19.284851 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.285070 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47845,"raft.queue_time_us":556347,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":7474,"spinlock_wait_cycles":2333568,"thread_start_us":308827,"threads_started":2,"wal-append.queue_time_us":428838}
I20260709 13:09:19.335093 21067 rpcz_store.cc:276] 0709 13:09:17.971412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.069980 (+ 98568us) service_pool.cc:224] Handling call
0709 13:09:18.081477 (+ 11497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:18.081686 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:19.038507 (+956821us) log.cc:844] Serialized 14 byte log entry
0709 13:09:19.334111 (+295604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.334193 (+    82us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:19.334255 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.334343 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.334491 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23216,"prepare.queue_time_us":515403,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":4557,"raft.queue_time_us":48641,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":4415,"thread_start_us":1209792,"threads_started":3,"wal-append.queue_time_us":246988}
I20260709 13:09:19.342063 27378 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22.
I20260709 13:09:19.317871 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.318722 28755 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:19.285046 28606 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.314992 28701 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:19.367949 28701 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:19.368520 28701 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:09:19.371876 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54117.4us
I20260709 13:09:19.372983 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:19.374935 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 84172.9us
I20260709 13:09:19.376017 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.377084 28757 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:19.377379 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.377820 28757 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:19.378472 28757 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:19.379215 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:19.305831 28707 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.381274 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.382093 21139 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:19.319620 28719 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:19.320842 28177 raft_consensus.cc:3055] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:19.325359 28611 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.330874 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.359572 28767 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:19.359874 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:19.387454 21147 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:19.363431 21138 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:09:19.365577 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1019) took 1423 ms. Trace:
I20260709 13:09:19.383219 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.319130 28766 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:19.394429 21144 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:19.378584 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:19.384203 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.396625 21141 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 25: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:19.393751 27378 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:19.398192 27378 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:19.399859 28755 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:19.401707 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:19.402894 28667 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:19.403573 28667 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:19.404762 28667 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.415771 21145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1088) took 1482 ms. Trace:
I20260709 13:09:19.418148 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:19.418883 21142 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 25: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:19.388886 21215 rpcz_store.cc:276] 0709 13:09:17.942536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.955523 (+ 12987us) service_pool.cc:224] Handling call
0709 13:09:17.979710 (+ 24187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:18.995897 (+1016187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:18.995932 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:19.014333 (+ 18401us) log.cc:844] Serialized 26 byte log entry
0709 13:09:19.218486 (+204153us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:19.218525 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.218604 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:19.365302 (+146698us) raft_consensus.cc:1685] finished
0709 13:09:19.365331 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.365505 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92617,"raft.queue_time_us":499179,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":99715,"thread_start_us":1067783,"threads_started":2,"wal-append.queue_time_us":230886}
I20260709 13:09:19.419847 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.422175 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:19.283008 28746 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.423367 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.233870 28699 consensus_queue.cc:237] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.426506 21142 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:19.429461 21140 raft_consensus.cc:2368] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21.
I20260709 13:09:19.430505 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.380534 28700 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.431345 21146 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:19.416206 21145 rpcz_store.cc:276] 0709 13:09:17.904751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.918518 (+ 13767us) service_pool.cc:224] Handling call
0709 13:09:17.929066 (+ 10548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:18.302516 (+373450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:18.302564 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:18.580392 (+277828us) log.cc:844] Serialized 26 byte log entry
0709 13:09:18.942112 (+361720us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:18.942155 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:18.942240 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:19.387131 (+444891us) raft_consensus.cc:1685] finished
0709 13:09:19.387154 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.387322 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42568,"raft.queue_time_us":170490,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":17759,"thread_start_us":544236,"threads_started":2,"wal-append.queue_time_us":458881}
I20260709 13:09:19.387791 28177 raft_consensus.cc:740] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:19.433409 21142 raft_consensus.cc:1240] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 19. Current term is 20. Ops: []
W20260709 13:09:19.436910 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 193780us
I20260709 13:09:19.427700 28715 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.420414 28705 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.420081 28528 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.439692 21212 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:19.422214 28710 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:19.435771 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:19.422355 28688 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:19.423600 28685 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.441988 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:19.448194 21146 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:09:19.448573 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.417385 28707 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.452214 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.420586 28606 consensus_queue.cc:1059] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:09:19.450693 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.458041 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.460430 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:19.461237 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:19.463269 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:19.463933 21800 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:19.464116 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.450201 28715 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.472321 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:19.472875 21798 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:19.475399 28726 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.468854 27892 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:19.475999 28726 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.478410 28726 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:19.479970 21792 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:09:19.453655 28700 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.471710 28748 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.482044 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.462563 28527 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:19.469017 28177 consensus_queue.cc:260] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.471210 21142 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:19.486809 28745 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.487660 27892 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:19.488531 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:19.473618 28715 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:19.489708 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 96782.3us
I20260709 13:09:19.493120 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.494132 28707 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:19.494819 28770 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:19.492131 28700 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.506211 28667 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:19.507077 21111 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.506538 21146 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:19.508359 28700 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.496438 21217 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:09:19.497752 28748 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.499899 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:19.497350 28719 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843)
I20260709 13:09:19.509128 28749 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.465318 28729 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:19.505112 21800 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 13:09:19.507262 28667 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:19.463837 28685 raft_consensus.cc:3055] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:19.500406 28746 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:19.510037 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:19.460034 28710 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.446681 28705 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.511198 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:19.513687 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:19.505795 28726 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.521744 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:19.524012 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:19.530297 28611 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.520179 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:19.520673 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:19.490705 28721 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:19.549927 28770 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:19.551244 28750 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:19.594125 28750 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:19.594612 28750 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:19.505388 28732 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:19.594543 28770 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.595535 28732 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:19.551393 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 69449.3us
I20260709 13:09:19.598568 28732 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.551635 28731 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:09:19.600322 21185 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:19.594847 28775 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:09:19.527829 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:19.605943 21796 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
W20260709 13:09:19.591756 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 84737.2us
I20260709 13:09:19.607234 21798 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:19.591156 21792 raft_consensus.cc:2376] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24. Re-sending same reply.
I20260709 13:09:19.591077 21795 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:19.609268 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:19.585731 21794 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:19.614153 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:09:19.588575 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 77460.7us
I20260709 13:09:19.620965 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:19.621945 21111 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:19.591611 28719 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:19.624459 21067 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:19.588413 28700 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:19.591216 28685 raft_consensus.cc:740] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:19.590279 28710 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:19.620126 28721 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:19.631263 28721 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:19.590967 28667 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.611749 28656 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:19.618736 28611 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:19.632694 28611 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.633113 28611 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:19.630667 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.626124 28177 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:19.635720 21215 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:19.631825 28721 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:19.611927 28527 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:19.639605 28527 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.641842 28527 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:19.618904 28499 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:19.585615 28726 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.629457 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:19.646180 27892 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:19.593132 28624 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.627130 28745 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:19.661939 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:19.653059 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:19.664312 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 113519us
I20260709 13:09:19.650111 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.662317 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:19.669736 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:19.670332 21218 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:19.670863 27892 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:19.671242 27892 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.671593 27892 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:19.667109 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:19.673148 21221 raft_consensus.cc:1240] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:19.673110 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.666241 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:19.674383 21215 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:19.674250 21144 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:19.666992 28745 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.623302 28707 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:09:19.669116 21213 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:09:19.675890 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.676149 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.677470 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:19.549687 28749 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:19.715610 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53580.2us
I20260709 13:09:19.716264 21214 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 27.
I20260709 13:09:19.646603 28552 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 13:09:19.720686 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.721390 21142 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:19.723584 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:19.665443 28719 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.648192 28746 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:09:19.728448 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.732270 21142 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:19.675433 28745 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:19.738584 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:19.740370 27892 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.667162 28710 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:19.676340 28606 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:19.733752 28715 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:19.734936 28696 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:19.679663 28499 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:19.730440 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53093.8us
I20260709 13:09:19.742982 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.749955 21142 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 24: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:19.741315 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.751380 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.752036 21219 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:19.741940 28715 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.754885 28715 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:19.741861 28606 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.758950 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:19.747417 28696 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:19.776098 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.779201 21213 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:19.800586 21215 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:19.759145 28606 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:19.800067 21218 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:19.820407 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.834628 21219 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:09:19.738282 28719 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.836035 21146 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:19.846380 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.848299 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:19.848623 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 110121us
I20260709 13:09:19.849409 21215 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 24: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:19.756917 28700 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:19.851071 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.777575 28677 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:19.854897 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:19.834266 28711 log.cc:927] Time spent T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:09:19.774902 28718 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:19.767774 28783 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:19.787559 28727 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:19.761489 28710 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:19.863461 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:19.737844 28685 consensus_queue.cc:260] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:19.852200 28778 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:09:19.859153 28677 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.819468 28748 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:19.855708 28770 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.867076 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:19.870406 21146 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:19.771725 28499 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.882553 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:19.883607 21067 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:19.866639 28783 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:19.871905 28701 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 13:09:19.869563 28605 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:19.873790 28677 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:19.887035 28783 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:19.885939 28790 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:09:19.918380 28780 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:09:19.918917 28499 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:19.787794 28782 consensus_queue.cc:1059] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.259s
I20260709 13:09:19.919978 28782 raft_consensus.cc:3055] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:19.920419 28782 raft_consensus.cc:740] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:19.921211 28789 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:09:19.921335 28782 consensus_queue.cc:260] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.870309 28727 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.923413 28790 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:19.923875 28790 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:19.924211 28727 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:19.926792 28714 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 1.717s	user 0.006s	sys 0.004s
I20260709 13:09:19.877040 28750 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:19.915617 21111 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.602s (SENT) [suppressed 12 similar messages]
I20260709 13:09:19.905133 28624 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:19.890941 28701 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.883600 28770 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.900506 28605 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:19.985177 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 133237us
I20260709 13:09:19.955735 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:19.958865 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.991935 21144 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:19.967060 21145 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
I20260709 13:09:19.986680 21110 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:19.998535 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:19.960955 28784 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:09:19.939738 28793 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.002660 21110 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:20.047593 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:20.057278 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 54702.7us
I20260709 13:09:20.058288 21110 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:19.965517 28605 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.062996 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:20.064633 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 218422us
W20260709 13:09:20.064848 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 149265us
I20260709 13:09:20.069586 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:20.070847 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:20.071924 21144 raft_consensus.cc:2393] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:20.075443 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:20.001696 28718 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:20.081180 21801 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:09:19.963269 28745 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.067399 28703 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:19.978233 28721 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:19.938297 28710 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.084029 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:20.087543 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 232071us
I20260709 13:09:20.079761 28617 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.093276 21144 raft_consensus.cc:2368] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22.
I20260709 13:09:20.084499 28727 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:20.078157 28726 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.082620 28719 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.098289 21218 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:20.087296 28703 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.111913 28795 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:20.112615 28795 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.113059 28795 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:20.116104 28750 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:20.099644 28727 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:20.115903 28801 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:20.067765 28793 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.118680 28776 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:20.126355 28776 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:20.127439 28776 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.131640 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:20.133742 28783 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.136373 28721 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.139890 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.140755 21067 raft_consensus.cc:2410] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 13:09:20.144805 28611 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.001s
I20260709 13:09:20.145416 28611 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.109170 28703 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:20.155747 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.158136 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.160211 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.165495 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.138705 28715 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.167482 21796 raft_consensus.cc:2410] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:09:20.168838 28745 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.124249 28727 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:20.122912 28801 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.196825 28801 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:20.198113 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.198777 21063 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:20.165710 28719 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.199687 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.200424 21066 raft_consensus.cc:2393] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 24: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:20.173048 28797 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:20.202013 28797 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.204453 28797 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.199415 28719 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.205695 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.206475 28719 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.206528 21145 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:20.207999 28793 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.181397 28764 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:20.209076 28764 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:20.209966 28764 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.212533 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.213281 21792 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:20.184460 28696 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.185132 28464 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.185864 28715 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:20.213292 21800 raft_consensus.cc:2393] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 24: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:20.216393 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.182813 28606 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:19.008810 21220 rpcz_store.cc:276] 0709 13:09:17.948135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.955758 (+  7623us) service_pool.cc:224] Handling call
0709 13:09:18.073263 (+117505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:18.073482 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:09:18.273940 (+200458us) log.cc:844] Serialized 14 byte log entry
0709 13:09:19.000791 (+726851us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.000861 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:19.000933 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.001027 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.001193 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33267,"prepare.queue_time_us":918684,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":393539,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":3903,"thread_start_us":896852,"threads_started":2}
I20260709 13:09:20.151677 28701 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.219483 28606 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.220427 28696 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.221647 28677 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.223090 21798 raft_consensus.cc:2368] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19.
I20260709 13:09:20.224803 28677 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.231942 28783 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:20.232582 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.235466 28700 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:20.236796 28700 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.239490 28552 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:20.240201 28552 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.243861 28667 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:20.244369 21063 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:20.244788 28667 consensus_queue.cc:1243] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer b70fc04de37e4239a8336534f0c9c0fb log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:09:20.246213 28667 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:20.246137 28790 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.247625 21794 raft_consensus.cc:2393] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 22: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:20.242766 28552 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.248982 28790 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.250687 28611 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.252583 21063 raft_consensus.cc:1275] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:20.253592 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.253687 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.254467 21069 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:09:20.255424 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.256428 21070 raft_consensus.cc:2393] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 23: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:20.260762 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.261577 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:20.261698 21066 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:20.263608 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.264057 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2494) took 1161 ms. Trace:
I20260709 13:09:20.264446 21067 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
I20260709 13:09:20.264514 21216 rpcz_store.cc:276] 0709 13:09:19.102808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.103444 (+   636us) service_pool.cc:224] Handling call
0709 13:09:20.263823 (+1160379us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54003b60d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:20.263955 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734584576}
I20260709 13:09:20.226903 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.267912 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.269095 21147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ca4319420e54bc19f621da03c811035"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:09:20.269824 21147 raft_consensus.cc:606] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:20.262979 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.279686 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.282006 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.282716 21067 raft_consensus.cc:1275] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:09:20.283006 21070 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
I20260709 13:09:20.293879 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.295367 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.295892 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.298853 21070 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:20.301246 21145 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:20.238637 28801 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.303602 21138 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:20.297950 28626 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:20.304620 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.305588 21138 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:20.247028 28667 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.309576 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.310077 21063 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:20.314826 28667 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.316188 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.319649 28801 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:20.319142 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.321004 21067 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:20.321204 21145 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:20.325672 21065 raft_consensus.cc:1275] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:09:20.329698 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.330453 21067 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:20.314970 28795 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.343871 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.344794 21070 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:20.328317 28732 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.312371 28626 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.351786 21220 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:20.353781 21216 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:20.361465 21216 raft_consensus.cc:1240] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:20.368278 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.329078 28703 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.354074 28626 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:20.354084 28702 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:20.380837 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.389397 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.390483 21216 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 15: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:20.391582 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.392472 21070 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:20.394117 21138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:09:20.396889 21138 raft_consensus.cc:606] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:20.397051 21141 raft_consensus.cc:2368] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24.
I20260709 13:09:20.397177 21138 raft_consensus.cc:614] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:20.402359 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.403849 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.363032 28732 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.403352 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:20.407442 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 332061us
I20260709 13:09:20.410024 28605 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:20.413332 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.414839 21069 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:20.415634 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.418283 21138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6beb666b5d504215a141add24a6d632f"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:09:20.418859 21138 raft_consensus.cc:606] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:20.428571 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.419164 21138 raft_consensus.cc:614] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:20.439509 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.445170 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:20.419135 28749 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.461421 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.463438 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.464181 21138 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:20.465881 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.466796 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:20.451928 28710 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:20.451404 28782 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:20.449744 28736 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.469877 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.474790 21071 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:09:20.445089 28727 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.472458 21070 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:20.489879 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.491317 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:20.495635 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:20.499920 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:20.500555 21144 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:20.502554 21138 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25.
I20260709 13:09:20.505355 21139 raft_consensus.cc:1240] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:20.506826 21139 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:09:20.462785 28527 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.509554 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.510600 21142 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:20.517279 21218 raft_consensus.cc:2410] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
W20260709 13:09:20.517875 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:20.518361 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.473405 28710 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.529542 21144 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:20.532550 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.532642 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.533454 21139 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:20.534924 21142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
W20260709 13:09:20.535369 28809 log.cc:927] Time spent T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:09:20.535580 21142 raft_consensus.cc:606] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:20.536018 21142 raft_consensus.cc:614] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:20.542999 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:20.462883 28801 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:09:20.543742 21142 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:20.544294 21146 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:20.545874 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.552959 21218 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:20.564350 21143 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:20.565368 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.569392 21142 raft_consensus.cc:2468] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:20.570839 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.571813 21142 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:20.532071 28710 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:20.578127 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:20.579326 21139 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:20.480404 28736 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.582857 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:20.560966 28795 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.582978 21142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a473266451804da089da06e9c927e7ee"
dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
mode: GRACEFUL
new_leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:32968
I20260709 13:09:20.588616 21142 raft_consensus.cc:606] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Received request to transfer leadership to TS 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:20.529872 28527 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.590570 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.589056 21142 raft_consensus.cc:614] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:20.591959 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.589080 21147 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 19: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:20.582229 28703 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.600049 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:20.600311 21144 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:20.600860 21216 raft_consensus.cc:2393] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:20.602401 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.602967 21070 raft_consensus.cc:3060] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:20.603055 21139 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:20.607507 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.608032 21066 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:09:20.608230 21071 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:20.608705 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:20.610649 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:20.610908 28527 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.617144 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.618827 21216 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:20.620805 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2497) took 1516 ms. Trace:
I20260709 13:09:20.621153 21792 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:20.621220 21217 rpcz_store.cc:276] 0709 13:09:19.104142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.170465 (+ 66323us) service_pool.cc:224] Handling call
0709 13:09:20.620624 (+1450159us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400101dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:20.620753 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741101952}
I20260709 13:09:20.623076 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:20.624048 21111 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:20.624217 28177 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:09:20.624439 28685 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:20.624655 28177 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:20.624922 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:20.625154 28177 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:20.625800 21111 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:20.626605 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:20.627472 28706 raft_consensus.cc:2804] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:20.627954 28706 raft_consensus.cc:697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:20.628800 28706 consensus_queue.cc:237] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:20.629938 28804 log.cc:927] Time spent T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.242s	user 0.000s	sys 0.002s
I20260709 13:09:20.630414 28722 raft_consensus.cc:2804] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:20.633474 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:20.634703 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.628631 28795 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:20.639467 21800 raft_consensus.cc:1240] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:20.636085 21767 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:20.630986 28722 raft_consensus.cc:697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:20.647197 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.647847 21146 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 25.
I20260709 13:09:20.650880 21070 raft_consensus.cc:2468] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:20.646674 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:20.654915 21795 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:20.654757 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.654806 21072 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:09:20.656006 28617 consensus_queue.cc:1048] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:20.657469 28625 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:20.661772 28771 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:20.662283 28771 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:20.667497 21792 pending_rounds.cc:85] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 13
I20260709 13:09:20.668113 21792 pending_rounds.cc:107] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:09:20.668375 21792 raft_consensus.cc:2889] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:20.637595 28528 raft_consensus.cc:2764] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:09:20.681663 21217 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:20.637615 28732 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:20.646097 28811 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:09:20.689491 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.696071 21800 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:20.652827 28795 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:20.702766 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.694922 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:09:20.708617 21140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 3587) took 1351 ms (client timeout 1527 ms). Trace:
W20260709 13:09:20.711158 21140 rpcz_store.cc:269] 0709 13:09:19.345868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.383913 (+ 38045us) service_pool.cc:224] Handling call
0709 13:09:20.695435 (+1311522us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b54001ec3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:20.697436 (+  2001us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14730496}
I20260709 13:09:20.709604 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 3583) took 1378 ms. Trace:
I20260709 13:09:20.716419 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:20.712427 21143 rpcz_store.cc:276] 0709 13:09:19.330961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.333825 (+  2864us) service_pool.cc:224] Handling call
0709 13:09:20.709377 (+1375552us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540032c3d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:20.709530 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2576321408}
I20260709 13:09:20.719133 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:20.728219 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.729081 28736 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:20.729844 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:20.672676 28810 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:09:20.732589 28810 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:20.732964 28810 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:20.733799 28810 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.743275 28464 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:20.744913 21797 raft_consensus.cc:1240] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:20.735054 28810 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:20.746364 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:20.751405 21767 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:20.754794 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:20.757279 21216 raft_consensus.cc:2393] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 22: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:20.663138 28771 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.765664 28791 consensus_queue.cc:1048] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:20.721060 28727 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.737216 28528 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:20.744207 28464 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:20.772608 28617 raft_consensus.cc:993] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: : Instructing follower 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:09:20.773095 28617 raft_consensus.cc:1081] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Signalling peer 9384358ce43f4e3fac2ae44bdf11c389 to start an election
I20260709 13:09:20.773720 28464 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:20.767534 28611 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:09:20.777180 28742 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:20.764631 28662 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:20.766649 28732 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:20.778877 28732 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:20.778360 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:20.779261 28732 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:20.783635 28734 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:20.784081 28734 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.784451 28734 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:20.701985 28656 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:09:20.749189 28787 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:20.809247 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:20.813727 21216 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:20.770913 21113 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:09:20.847249 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 76403.7us
I20260709 13:09:20.789044 28528 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:20.854913 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.855620 21063 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:20.855543 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:20.857232 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.857976 21063 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 25: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:20.858791 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.859565 21066 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:20.860862 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.861637 21064 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:20.862277 21143 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:20.869534 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:20.871129 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:20.873364 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.824915 28771 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:20.864233 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:20.769512 28746 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.081s
I20260709 13:09:20.888260 21071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.888972 21071 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:20.892815 21072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.893198 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:20.893437 21072 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:20.893740 21220 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:20.894810 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.891320 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:20.898610 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.901772 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.901304 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.908135 21221 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:20.909343 21216 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:20.857910 28706 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:20.913635 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.914039 28706 raft_consensus.cc:697] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:20.914503 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:20.864527 28528 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:20.915130 21215 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:20.816079 28177 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.878744 28527 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:20.878464 28700 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:20.926363 21219 raft_consensus.cc:1240] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:20.928742 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:20.901191 28611 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:20.865227 28789 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:09:20.909442 28787 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:20.879294 28727 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 13:09:20.908810 28705 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.920682 28464 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.939565 21064 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:20.958542 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:20.888636 28757 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:09:20.962991 21064 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25.
I20260709 13:09:20.811995 28722 consensus_queue.cc:237] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:20.841912 28795 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.949658 21066 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:20.949260 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:20.970966 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.959869 21113 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:20.975175 21072 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:20.936285 28611 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.944990 28787 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.979202 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:20.944656 28703 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:20.931614 28700 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.981048 28715 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:20.987197 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:20.994234 21793 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:20.999571 28662 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:20.973984 28464 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:21.004029 28757 consensus_queue.cc:1059] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:20.984061 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.976135 21066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:20.974545 21113 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:20.915086 28706 consensus_queue.cc:237] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:21.412868 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 438392us
I20260709 13:09:21.011125 21220 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
W20260709 13:09:21.418526 21184 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.016s	sys 0.030s
W20260709 13:09:21.796547 21184 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.016s	sys 0.030s
W20260709 13:09:21.796917 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 1.02984e+06us
I20260709 13:09:21.800727 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:21.502795 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.801816 21800 raft_consensus.cc:2410] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:09:21.802421 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54634 (request call id 1474) took 1111 ms. Trace:
I20260709 13:09:21.802733 21800 rpcz_store.cc:276] 0709 13:09:20.691047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.800396 (+1109349us) service_pool.cc:224] Handling call
0709 13:09:21.802337 (+  1941us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:21.745713 28842 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.801759 28844 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.804652 28842 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:21.805008 28844 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:21.753844 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:21.807821 21146 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:21.808073 28844 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:21.810083 28844 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.810575 28844 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:21.757423 21792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1523) took 1093 ms. Trace:
I20260709 13:09:21.811746 21792 rpcz_store.cc:276] 0709 13:09:20.663690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.666864 (+  3174us) service_pool.cc:224] Handling call
0709 13:09:20.667227 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:20.670378 (+  3151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.670495 (+   117us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:20.671528 (+  1033us) log.cc:844] Serialized 52 byte log entry
0709 13:09:20.798452 (+126924us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:20.798513 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.798617 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:21.757090 (+958473us) raft_consensus.cc:1685] finished
0709 13:09:21.757115 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:21.757317 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2753,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8138,"thread_start_us":126370,"threads_started":1,"wal-append.queue_time_us":415150}
W20260709 13:09:21.753845 21217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.014s	sys 0.034s
I20260709 13:09:21.813496 28844 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:21.813805 21217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.014s	sys 0.034s
I20260709 13:09:21.814606 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3472) took 1117 ms. Trace:
I20260709 13:09:21.814931 21217 rpcz_store.cc:276] 0709 13:09:20.697703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.697928 (+   225us) service_pool.cc:224] Handling call
0709 13:09:20.698251 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:20.759181 (+ 60930us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.759263 (+    82us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:20.760172 (+   909us) log.cc:844] Serialized 26 byte log entry
0709 13:09:21.814270 (+1054098us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:21.814317 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:21.814391 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:21.814435 (+    44us) raft_consensus.cc:1685] finished
0709 13:09:21.814448 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:21.814557 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44562,"raft.queue_time_us":45442,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":6031,"thread_start_us":1035157,"threads_started":1,"wal-append.queue_time_us":408035}
I20260709 13:09:21.178035 27892 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:21.241288 28177 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:21.226785 28527 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:21.961428 28527 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:21.961989 28527 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:21.592974 28552 raft_consensus.cc:3055] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:22.002285 28552 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:20.982821 28611 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:21.194912 28528 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:21.037039 28662 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.007982 28552 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.025555 28662 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:20.981060 28688 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:21.016621 28700 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:21.426268 28702 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.029240 28702 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:21.044202 28715 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.032621 28702 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:21.171207 28727 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.198064 28734 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.035804 28727 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.037838 28734 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:22.038717 28727 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.978202 28746 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:21.202147 28685 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.577s	user 0.009s	sys 0.000s
I20260709 13:09:21.239449 28770 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:20.973593 28705 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:22.045694 28705 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:21.217286 28710 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.000s
I20260709 13:09:22.048574 28710 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.050926 28710 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:21.217811 28732 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:21.027781 28782 consensus_meta.cc:287] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.552s	user 0.004s	sys 0.008s
W20260709 13:09:21.428254 28806 log.cc:927] Time spent T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 1.027s	user 0.003s	sys 0.004s
I20260709 13:09:22.062899 28837 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.063650 28837 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:20.985364 28787 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:21.144773 28818 log.cc:927] Time spent T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.381s	user 0.002s	sys 0.000s
W20260709 13:09:21.557348 28821 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.397s	user 0.010s	sys 0.007s
I20260709 13:09:21.203930 28828 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.415650 28829 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:21.545284 28826 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.774318 21065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:21.776410 28843 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:21.778003 21766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.012s	sys 0.032s
W20260709 13:09:21.780652 21070 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.010s	sys 0.018s
I20260709 13:09:21.782794 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:21.807013 28842 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:21.836201 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:21.836490 21765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.013s	sys 0.033s
I20260709 13:09:21.836722 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:21.837646 21212 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:09:21.839020 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:21.843427 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:21.847066 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:21.852419 21036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.978s	user 0.009s	sys 0.018s
W20260709 13:09:21.864514 21068 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.004s	sys 0.017s
W20260709 13:09:21.868872 21143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.012s	sys 0.023s
W20260709 13:09:21.871770 21218 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.017s	sys 0.017s
I20260709 13:09:21.878899 21146 raft_consensus.cc:2468] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
W20260709 13:09:21.887326 21184 connection.cc:256] RPC call timeout handler was delayed by -1.0534728089999135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
W20260709 13:09:21.897819 21071 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.012s	sys 0.009s
I20260709 13:09:21.906193 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1520) took 1272 ms. Trace:
I20260709 13:09:21.911576 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:21.915258 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:21.919803 21072 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ca4319420e54bc19f621da03c811035"
dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
 from {username='slave'} at 127.0.0.1:39510
I20260709 13:09:21.923050 28764 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:21.945909 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:21.949636 21038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.020s	sys 0.040s
I20260709 13:09:21.953639 21215 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:09:21.963792 21063 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:09:21.964614 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 985428us
I20260709 13:09:21.965095 28527 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:21.966992 28846 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:21.968652 21113 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:21.981803 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:21.984138 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53249.6us
I20260709 13:09:21.984879 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50996 (request call id 1117) took 1004 ms. Trace:
I20260709 13:09:21.986444 28850 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:21.989410 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:21.991510 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
W20260709 13:09:21.999313 21185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.006s	user 0.004s	sys 0.021s
I20260709 13:09:21.185608 28464 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:22.019513 28528 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:22.022428 28820 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.859s	user 0.000s	sys 0.007s
I20260709 13:09:22.027230 28688 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:21.750623 28736 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:22.041765 28746 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.043505 28810 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.034469 28715 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:21.425036 28722 raft_consensus.cc:2804] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:22.056295 28705 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:21.016472 28749 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:22.115067 21071 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.183s	user 0.012s	sys 0.014s
I20260709 13:09:22.115767 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3605) took 1652 ms. Trace:
I20260709 13:09:22.116211 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2489) took 1240 ms. Trace:
I20260709 13:09:22.059172 28732 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:22.117650 21185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.009s	sys 0.022s
I20260709 13:09:22.117483 28736 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:22.118749 21765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.228s	user 0.016s	sys 0.033s
W20260709 13:09:22.119122 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 1.23232e+06us
I20260709 13:09:22.119642 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.119326 28736 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.121085 21185 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25.
I20260709 13:09:22.121446 21185 leader_election.cc:403] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:09:22.122067 28749 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.118762 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.134917 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.135496 21212 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:22.116466 21138 rpcz_store.cc:276] 0709 13:09:20.463399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.503236 (+ 39837us) service_pool.cc:224] Handling call
0709 13:09:20.503567 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:20.504349 (+   782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.504399 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:20.505156 (+   757us) log.cc:844] Serialized 26 byte log entry
0709 13:09:20.598551 (+ 93395us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:20.598592 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.598659 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.115565 (+1516906us) raft_consensus.cc:1685] finished
0709 13:09:22.115588 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.115712 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":195,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"thread_start_us":92906,"threads_started":1,"wal-append.queue_time_us":177844}
I20260709 13:09:22.118185 28732 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:22.137091 21067 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:22.137178 28732 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.137705 21067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 3297) took 1219 ms. Trace:
I20260709 13:09:22.138111 21067 rpcz_store.cc:276] 0709 13:09:20.928190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.911087 (+982897us) service_pool.cc:224] Handling call
0709 13:09:22.137645 (+226558us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:22.139839 21218 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.288s	user 0.018s	sys 0.019s
I20260709 13:09:22.140839 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:22.141324 21214 raft_consensus.cc:3060] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:22.141716 21067 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:22.114787 28843 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.122790 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.143564 28662 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:22.144167 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3468) took 1560 ms. Trace:
I20260709 13:09:22.144784 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:22.145128 28842 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:22.145255 28843 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:22.115738 21064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:22.145915 28842 consensus_queue.cc:1048] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:22.127127 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2488) took 1259 ms. Trace:
I20260709 13:09:22.120905 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.147006 28782 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:22.120954 28736 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:22.124768 21797 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:22.147535 28782 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:22.125671 21795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 981) took 1460 ms. Trace:
I20260709 13:09:22.148331 28782 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:22.148597 21795 rpcz_store.cc:276] 0709 13:09:20.664870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.664981 (+   111us) service_pool.cc:224] Handling call
0709 13:09:20.665358 (+   377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:20.665765 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.665798 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:20.666356 (+   558us) log.cc:844] Serialized 52 byte log entry
0709 13:09:20.808396 (+142040us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:20.808458 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.808559 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:21.556636 (+748077us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:22.125379 (+568743us) raft_consensus.cc:1685] finished
0709 13:09:22.125402 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.125613 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2460,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":17750912,"thread_start_us":141630,"threads_started":1,"wal-append.queue_time_us":471929}
I20260709 13:09:22.125228 28746 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:22.127487 21072 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:22.150583 21072 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:22.128408 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.152158 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:22.152957 21215 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:09:22.153700 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:22.122865 28828 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.129952 21065 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:09:22.154704 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:22.123709 28826 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:22.131013 21070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.226s	user 0.011s	sys 0.019s
W20260709 13:09:22.155833 21766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.265s	user 0.013s	sys 0.034s
I20260709 13:09:22.147351 21063 rpcz_store.cc:276] 0709 13:09:20.867670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.878730 (+ 11060us) service_pool.cc:224] Handling call
0709 13:09:20.879034 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.126384 (+1247350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.126928 (+   544us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540019dcd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:22.127073 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2493895424}
W20260709 13:09:22.156620 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:22.131234 21036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.257s	user 0.010s	sys 0.019s
I20260709 13:09:22.157541 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
W20260709 13:09:22.157716 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 1.28384e+06us
I20260709 13:09:22.157847 28770 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.158365 28770 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.158830 28656 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:22.144577 21218 rpcz_store.cc:276] 0709 13:09:20.584288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.584434 (+   146us) service_pool.cc:224] Handling call
0709 13:09:20.584712 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:20.597383 (+ 12671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.597407 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:20.605852 (+  8445us) log.cc:844] Serialized 26 byte log entry
0709 13:09:20.821506 (+215654us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:22.140260 (+1318754us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.143867 (+  3607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.143939 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.143976 (+    37us) raft_consensus.cc:1685] finished
0709 13:09:22.143991 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.144113 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3350,"prepare.queue_time_us":359998,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":4625,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":3749,"thread_start_us":1461014,"threads_started":2,"wal-append.queue_time_us":411240}
I20260709 13:09:22.160575 28770 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.160485 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.160992 28826 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:22.090678 28685 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:22.131902 21216 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 12: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:22.117154 21793 rpcz_store.cc:276] 0709 13:09:20.634179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.654130 (+ 19951us) service_pool.cc:224] Handling call
0709 13:09:20.993816 (+339686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:21.015915 (+ 22099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:21.905917 (+890002us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b5400494b50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:21.906115 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780036352}
I20260709 13:09:22.161870 28552 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:22.161777 28685 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.162454 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:22.132841 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.163405 28742 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.163697 21221 raft_consensus.cc:2368] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 25. Current term is 26.
I20260709 13:09:22.163892 28742 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.164701 21221 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:22.158874 28527 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:09:22.166313 28742 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:22.115940 21217 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:22.166742 28527 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:22.166813 21221 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 13:09:22.133332 21068 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.258s	user 0.004s	sys 0.020s
W20260709 13:09:22.133030 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 164450us
I20260709 13:09:22.158855 21036 leader_election.cc:400] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22.
I20260709 13:09:22.174360 21036 leader_election.cc:403] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:09:22.175048 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.175997 28528 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:22.133728 21213 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:22.133857 28727 raft_consensus.cc:2804] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:22.177312 28757 consensus_queue.cc:1059] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:22.177830 28727 raft_consensus.cc:697] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:22.178535 28757 raft_consensus.cc:3055] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:22.178946 28757 raft_consensus.cc:740] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:22.178814 28727 consensus_queue.cc:237] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.179399 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:22.122552 21038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.219s	user 0.022s	sys 0.042s
I20260709 13:09:22.146453 21064 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:22.180032 28757 consensus_queue.cc:260] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:22.131809 21069 raft_consensus.cc:3060] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:22.134153 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.133570 28705 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:22.155138 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 3299) took 1177 ms. Trace:
I20260709 13:09:22.183884 28705 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.186158 28705 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:22.195853 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:22.196580 28732 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:22.197484 21801 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:09:22.163046 21216 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:22.132478 28829 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:22.128990 21765 connection.cc:256] RPC call timeout handler was delayed by -1.0076078810000126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
I20260709 13:09:22.167168 28527 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:22.199452 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:22.200237 21801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 978) took 1786 ms (client timeout 1517 ms). Trace:
I20260709 13:09:22.199970 28527 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.200632 28734 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.200747 28801 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.201239 28734 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.201499 28801 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.203145 28829 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.203889 28791 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:22.204317 28791 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:22.204672 28791 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:22.204813 28829 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:22.205235 28829 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.205570 28829 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:22.205578 28801 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.206648 28734 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.168519 28828 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.208446 28828 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:22.201264 28527 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:22.169729 28685 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:22.117920 21140 rpcz_store.cc:276] 0709 13:09:20.984660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.986039 (+  1379us) service_pool.cc:224] Handling call
0709 13:09:21.984802 (+998763us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:22.200663 21801 rpcz_store.cc:269] 0709 13:09:20.414279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.415395 (+  1116us) service_pool.cc:224] Handling call
0709 13:09:22.200031 (+1784636us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54003bbd54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:22.200152 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4096318720}
I20260709 13:09:22.175606 28846 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:22.132390 28850 raft_consensus.cc:3055] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:22.175742 28656 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 21. Current term is 22.
I20260709 13:09:22.213330 28850 raft_consensus.cc:740] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:22.178354 28847 consensus_queue.cc:1059] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:22.213874 28611 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.181159 28757 raft_consensus.cc:3060] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:22.217536 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.217909 21069 raft_consensus.cc:2468] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:22.218490 21069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 3295) took 1307 ms. Trace:
I20260709 13:09:22.218868 21069 rpcz_store.cc:276] 0709 13:09:20.911459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.914883 (+1003424us) service_pool.cc:224] Handling call
0709 13:09:22.218432 (+303549us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:22.130779 21143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.158s	user 0.013s	sys 0.025s
I20260709 13:09:22.220093 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3615) took 1300 ms. Trace:
I20260709 13:09:22.220434 21143 rpcz_store.cc:276] 0709 13:09:20.920106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.920301 (+   195us) service_pool.cc:224] Handling call
0709 13:09:20.920734 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:20.921032 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.921073 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:20.927510 (+  6437us) log.cc:844] Serialized 26 byte log entry
0709 13:09:22.219723 (+1292213us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:22.219764 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.219830 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.219884 (+    54us) raft_consensus.cc:1685] finished
0709 13:09:22.219895 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.220006 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1247144,"threads_started":1,"wal-append.queue_time_us":257762,"wal-append.run_cpu_time_us":1702,"wal-append.run_wall_time_us":1001729}
I20260709 13:09:22.187872 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:22.223327 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:22.189168 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.225252 21795 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:22.189826 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.226500 21800 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 12.
I20260709 13:09:22.145962 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.227684 28706 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:09:22.227877 28846 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:22.071862 28700 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:22.228870 28706 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:22.229578 21066 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 24: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:22.230659 28700 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.156499 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60302 (request call id 1418) took 1354 ms. Trace:
W20260709 13:09:22.229017 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83135.6us
I20260709 13:09:22.231545 28706 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:22.232555 28706 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:22.117491 28810 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.158208 21063 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:22.231271 21070 rpcz_store.cc:276] 0709 13:09:20.801979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.802101 (+   122us) service_pool.cc:224] Handling call
0709 13:09:20.821984 (+ 19883us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:20.823609 (+  1625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.823645 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:20.849343 (+ 25698us) log.cc:844] Serialized 26 byte log entry
0709 13:09:22.155697 (+1306354us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:22.156007 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.156216 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.156300 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.156338 (+    38us) raft_consensus.cc:1685] finished
0709 13:09:22.156348 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.156454 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1129,"prepare.queue_time_us":1634,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":3722,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":8049,"thread_start_us":1250493,"threads_started":1,"wal-append.queue_time_us":333894}
I20260709 13:09:22.234992 28787 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:22.170132 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49178.3us
I20260709 13:09:22.235868 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:22.235958 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:22.236510 21063 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
I20260709 13:09:22.236825 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.237143 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:22.237362 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:22.237709 21070 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:09:22.237977 21185 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:22.238150 21063 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:09:22.238636 21070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:22.171388 21072 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.239451 28736 consensus_queue.cc:1059] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:22.241616 28736 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:22.140878 28749 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:22.242084 28736 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:22.242527 28749 raft_consensus.cc:2804] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:22.243235 28746 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:22.239177 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:22.117017 28688 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.244210 21063 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:22.244436 28749 raft_consensus.cc:697] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:22.245254 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:22.245816 21063 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:22.245437 28749 consensus_queue.cc:237] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.131533 28722 raft_consensus.cc:697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:22.243176 28810 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.214524 28850 consensus_queue.cc:260] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.230696 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39498 (request call id 3296) took 1312 ms. Trace:
I20260709 13:09:22.239593 28826 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:09:22.240394 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:22.249087 21066 rpcz_store.cc:276] 0709 13:09:20.917935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.975787 (+ 57852us) service_pool.cc:224] Handling call
0709 13:09:22.230495 (+1254708us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b540011c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:22.230618 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2053561088}
I20260709 13:09:22.250020 21113 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:22.123549 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.250679 21185 leader_election.cc:400] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21.
I20260709 13:09:22.161303 21218 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:22.250824 21113 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:22.239511 28770 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:22.243484 28842 raft_consensus.cc:3055] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:22.251441 28842 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:22.251474 28770 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.252535 28842 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.253904 28770 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.184177 21065 rpcz_store.cc:276] 0709 13:09:20.977650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.774000 (+796350us) service_pool.cc:224] Handling call
0709 13:09:22.155077 (+381077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:22.243049 28736 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.239425 28667 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:22.256610 28843 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:22.256987 28850 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:22.257107 28843 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:22.248399 28715 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:22.257793 28843 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:22.258014 28715 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:22.258283 28727 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:22.251011 21185 leader_election.cc:403] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:22.258728 28727 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.259107 28727 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:22.259591 28846 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21.
I20260709 13:09:22.259737 21185 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:22.253986 28842 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:22.260576 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 72235.9us
I20260709 13:09:22.261258 21185 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:22.262501 28464 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:22.247963 28722 consensus_queue.cc:237] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.265151 21072 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.265663 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:39510 (request call id 2490) took 1389 ms. Trace:
I20260709 13:09:22.116874 21071 rpcz_store.cc:276] 0709 13:09:20.876037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.889600 (+ 13563us) service_pool.cc:224] Handling call
0709 13:09:20.918094 (+ 28494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:20.918376 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:20.932086 (+ 13710us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.115905 (+1183819us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:22.115954 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:22.115995 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.116057 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.116163 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13993,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":272300,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":1183491,"threads_started":1}
I20260709 13:09:22.265847 21218 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:22.067116 28837 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:22.278896 21063 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:22.279775 21214 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
W20260709 13:09:22.267132 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 67764.4us
I20260709 13:09:22.220281 21069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:22.191792 21064 raft_consensus.cc:2468] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 25.
I20260709 13:09:22.303157 21069 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:22.234560 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:22.242097 21111 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:22.269130 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 89821.6us
I20260709 13:09:22.211790 28852 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:22.272495 21767 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:22.318775 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:22.319429 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47058.5us
I20260709 13:09:22.319932 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:22.272977 28746 raft_consensus.cc:2804] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:09:22.273445 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.334031 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:22.334916 21141 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:22.335188 21063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:22.335507 21767 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:22.337555 21036 leader_election.cc:400] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24.
I20260709 13:09:22.338075 21036 leader_election.cc:403] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:09:22.337785 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:22.273623 21143 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:09:22.273401 28464 raft_consensus.cc:697] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:22.259038 28715 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.274716 28662 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25.
I20260709 13:09:22.281355 21067 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:22.365486 21069 raft_consensus.cc:2468] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:22.338598 28852 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:22.239212 21070 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:22.366878 28852 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.367308 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:22.368023 21146 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:22.368173 28852 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:22.368572 21070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2493) took 1429 ms. Trace:
I20260709 13:09:22.266088 21072 rpcz_store.cc:276] 0709 13:09:20.888982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.919651 (+1030669us) service_pool.cc:224] Handling call
0709 13:09:22.265523 (+345872us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:22.368912 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:22.370014 21072 raft_consensus.cc:1275] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:22.370615 21072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3298) took 1393 ms. Trace:
I20260709 13:09:22.338788 28706 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:22.370975 21072 rpcz_store.cc:276] 0709 13:09:20.977280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.369427 (+1392147us) service_pool.cc:224] Handling call
0709 13:09:22.369703 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.370372 (+   669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.370564 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:22.371690 28464 consensus_queue.cc:237] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.292375 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:22.375666 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83410.8us
W20260709 13:09:22.377380 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.370455 21146 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:09:22.305088 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 63056.2us
I20260709 13:09:22.375159 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:22.379104 21111 leader_election.cc:400] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 25. Current term is 26.
I20260709 13:09:22.379453 21111 leader_election.cc:403] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:09:22.317862 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:22.318361 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83872us
I20260709 13:09:22.321157 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:22.322724 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:22.381098 21145 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 22: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:22.308398 21765 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:22.384814 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 76509.7us
I20260709 13:09:22.385826 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:22.368929 21070 rpcz_store.cc:276] 0709 13:09:20.938792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.367754 (+1428962us) service_pool.cc:224] Handling call
0709 13:09:22.368065 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.368269 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:22.368296 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:22.368338 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:22.368371 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.368425 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.368521 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:22.349383 21212 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
W20260709 13:09:22.351115 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 102319us
I20260709 13:09:22.297953 21068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2487) took 1430 ms. Trace:
I20260709 13:09:22.390211 21068 rpcz_store.cc:276] 0709 13:09:20.867379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.874424 (+  7045us) service_pool.cc:224] Handling call
0709 13:09:20.874819 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:22.171789 (+1296970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:22.171820 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:22.172238 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:09:22.297555 (+125317us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:22.297596 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.297678 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.297722 (+    44us) raft_consensus.cc:1685] finished
0709 13:09:22.297733 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.297896 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":328458,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"thread_start_us":1421468,"threads_started":2,"wal-append.queue_time_us":121164}
I20260709 13:09:22.394194 21068 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:09:22.394768 21139 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 18: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:22.400134 21069 raft_consensus.cc:1275] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:09:22.402104 21187 leader_election.cc:400] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Vote denied by peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843) with higher term. Message: Invalid argument: T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25.
I20260709 13:09:22.317870 28705 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:22.403260 21187 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:22.352279 28736 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:22.417804 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.435570 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:22.449283 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 70294.7us
W20260709 13:09:22.435068 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 97846.3us
I20260709 13:09:22.437985 21070 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:09:22.451355 21070 pending_rounds.cc:85] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting all ops after (but not including) 11
I20260709 13:09:22.451709 21070 pending_rounds.cc:107] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:09:22.452024 21070 raft_consensus.cc:2889] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:22.441220 21069 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 13:09:22.442827 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.443449 27892 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:22.446256 21068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389" is_pre_election: true
I20260709 13:09:22.454504 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:22.352141 28764 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:22.484035 21068 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18.
I20260709 13:09:22.482350 21143 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:22.455988 21064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3291) took 1548 ms. Trace:
I20260709 13:09:22.484572 21801 raft_consensus.cc:3060] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:22.486855 21064 rpcz_store.cc:276] 0709 13:09:20.907236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.303414 (+1396178us) service_pool.cc:224] Handling call
0709 13:09:22.375148 (+ 71734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.402093 (+ 26945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:22.402136 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:22.434100 (+ 31964us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:22.434155 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.434221 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.455787 (+ 21566us) spinlock_profiling.cc:243] Waited 71.3 ms on lock 0x7b540041cb50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:22.455919 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35754,"spinlock_wait_cycles":190859392}
I20260709 13:09:22.364388 28770 raft_consensus.cc:2804] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:22.402514 28801 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:22.461508 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:09:22.489302 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 111766us
I20260709 13:09:22.490046 21113 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:22.453778 21795 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:22.491349 27892 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:22.487941 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45203.3us
I20260709 13:09:22.493871 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:22.494510 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:22.495256 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2836134389999643s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 13:09:22.496369 21067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9384358ce43f4e3fac2ae44bdf11c389"
I20260709 13:09:22.496938 21067 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:22.497872 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0676134389999561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
I20260709 13:09:22.497999 27892 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:09:22.498382 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0667134389999546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 13:09:22.498461 27892 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.498533 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:22.498819 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0433134389999168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
I20260709 13:09:22.499007 27892 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:22.501003 21187 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.375159 28746 raft_consensus.cc:697] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:22.412946 28705 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.403900 28624 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:22.453672 28710 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.433945 28742 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.433990 28796 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.434770 28611 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.453504 28662 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.505460 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:22.531225 21067 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:22.442494 28528 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:22.434508 28828 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24.
I20260709 13:09:22.519658 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.492084 28837 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:22.511107 28688 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election won for term 22
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:09:22.499511 28715 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:22.497573 28656 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.486176 28764 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:22.403064 28782 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.454083 28843 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.485548 28787 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.537127 21794 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 12.
I20260709 13:09:22.546655 27892 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:22.641259 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 193471us
I20260709 13:09:22.535302 28528 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.535166 28861 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:09:22.533610 28710 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.551556 28746 consensus_queue.cc:237] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.534868 28742 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:22.553043 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.553184 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.689023 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.514781 28705 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:22.559340 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.502977 28847 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.692675 21792 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 22: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:22.586278 28656 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.507393 28527 consensus_queue.cc:1059] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 13:09:22.514964 28685 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.592872 21801 raft_consensus.cc:2468] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:22.576813 28796 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.542096 28464 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:09:22.646523 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:22.654488 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.673400 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.639240 19967 tablet_server.cc:179] TabletServer@127.19.127.193:0 shutting down...
I20260709 13:09:22.585489 28764 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:22.642933 21767 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:22.648906 28528 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:22.738230 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:22.739114 21799 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 19: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:22.740824 28656 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:22.742687 28801 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:22.742887 28829 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.000s
I20260709 13:09:22.743423 28829 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.744218 21141 raft_consensus.cc:2468] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 26.
I20260709 13:09:22.745736 21071 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 25. Current term is 26. Ops: []
I20260709 13:09:22.683190 27892 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:22.554466 28662 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:22.750864 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 108028us
I20260709 13:09:22.677840 28770 raft_consensus.cc:697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:22.684803 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 202407us
I20260709 13:09:22.676503 28842 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:22.694150 21765 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:22.772413 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 78321.3us
I20260709 13:09:22.573283 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:22.699395 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 281689us
I20260709 13:09:22.683908 28736 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:22.576159 28688 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:22.682950 28700 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:22.718140 28883 consensus_queue.cc:1048] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:22.692914 28727 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:22.718935 28866 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:09:22.700066 28847 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:22.729553 28871 log.cc:927] Time spent T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:09:22.771847 28878 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:22.730584 28843 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:22.812239 28782 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:22.769548 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 279554us
I20260709 13:09:22.813179 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:22.770790 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 309343us
W20260709 13:09:22.771296 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 239288us
I20260709 13:09:22.773329 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:22.786783 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.727499 28464 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.747398 28702 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 25. Current term is 26.
I20260709 13:09:22.767359 28829 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.796912 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:22.801798 28688 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.826030 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:22.826840 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:22.827174 21184 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:22.802888 28847 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.803310 28861 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:22.808938 28757 consensus_meta.cc:287] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.593s	user 0.004s	sys 0.004s
W20260709 13:09:22.818038 28791 consensus_meta.cc:287] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.612s	user 0.006s	sys 0.003s
I20260709 13:09:22.810415 28770 consensus_queue.cc:237] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.714241 28685 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.785135 28667 raft_consensus.cc:3055] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:22.817327 21144 raft_consensus.cc:2393] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 16: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:22.784341 28736 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:22.821305 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 48100.4us
I20260709 13:09:22.879119 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:22.822358 21113 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:22.828439 21219 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
W20260709 13:09:22.801870 21822 debug-util.cc:398] Leaking SignalData structure 0x7b080020ad00 after lost signal to thread 19970
I20260709 13:09:22.882099 28626 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:22.882613 28845 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:22.883018 21822 debug-util.cc:398] Leaking SignalData structure 0x7b08002516a0 after lost signal to thread 21004
I20260709 13:09:22.883782 28177 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.884275 28177 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:22.884838 21822 debug-util.cc:398] Leaking SignalData structure 0x7b08003e1180 after lost signal to thread 21096
W20260709 13:09:22.886231 21822 debug-util.cc:398] Leaking SignalData structure 0x7b0800314480 after lost signal to thread 21171
I20260709 13:09:22.886323 28611 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:22.886968 28611 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.832453 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:22.836505 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.847572 28878 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:22.852272 28710 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:22.856272 21065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2492) took 1966 ms. Trace:
I20260709 13:09:22.829835 28749 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:09:22.860146 28870 log.cc:927] Time spent T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.179s	user 0.003s	sys 0.000s
I20260709 13:09:22.860565 28727 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.860802 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.918197 21220 raft_consensus.cc:2393] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 18: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:22.861439 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.861735 21071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39498 (request call id 3290) took 1955 ms. Trace:
I20260709 13:09:22.919246 21071 rpcz_store.cc:276] 0709 13:09:20.906875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.296717 (+1389842us) service_pool.cc:224] Handling call
0709 13:09:22.363938 (+ 67221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.746149 (+382211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.861507 (+115358us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b540026f054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:22.861679 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877563136}
I20260709 13:09:22.830003 28843 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:09:22.938107 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:22.873237 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.818367 28734 raft_consensus.cc:2804] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:22.818670 28464 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:22.883174 28845 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:22.886590 21822 debug-util.cc:398] Leaking SignalData structure 0x7b0800365020 after lost signal to thread 21172
I20260709 13:09:22.838742 21218 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:22.877863 28837 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:22.795212 28700 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.881253 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:22.890239 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67948.9us
I20260709 13:09:22.876068 28880 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:22.902432 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 346786us
I20260709 13:09:22.891170 28829 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.961607 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:22.909799 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.837872 28872 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:22.911887 21065 rpcz_store.cc:276] 0709 13:09:20.889574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.254696 (+1365122us) service_pool.cc:224] Handling call
0709 13:09:22.255049 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:22.255423 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:22.255453 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:22.255892 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 13:09:22.359601 (+103709us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:22.359628 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.359682 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.856043 (+496361us) raft_consensus.cc:1685] finished
0709 13:09:22.856063 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.856208 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":261,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":103369,"threads_started":1,"wal-append.queue_time_us":254185}
I20260709 13:09:22.889123 28611 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:22.839689 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:22.945225 21822 debug-util.cc:398] Leaking SignalData structure 0x7b080029de80 after lost signal to thread 21245
I20260709 13:09:22.874594 28667 raft_consensus.cc:740] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:22.882874 28626 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:09:22.960693 28734 raft_consensus.cc:697] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:22.972441 28734 consensus_queue.cc:237] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.830137 28782 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.088s
I20260709 13:09:22.888801 28890 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.870559 28886 raft_consensus.cc:2764] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:09:22.890390 28787 raft_consensus.cc:2804] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:22.835219 28883 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:09:22.889839 28873 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.888351 28742 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:22.887274 28177 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:22.963589 21822 debug-util.cc:398] Leaking SignalData structure 0x7b0800205ea0 after lost signal to thread 21246
W20260709 13:09:23.001524 21822 debug-util.cc:398] Leaking SignalData structure 0x7b080035ed60 after lost signal to thread 21825
W20260709 13:09:23.002019 21822 debug-util.cc:398] Leaking SignalData structure 0x7b08003bc020 after lost signal to thread 21826
I20260709 13:09:22.965929 28872 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:22.945870 28528 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:22.970274 28845 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.938301 28843 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.943127 28710 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:22.970715 28796 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:22.872290 28791 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:22.887677 28736 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:22.975244 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 94105.8us
I20260709 13:09:22.977243 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.077417 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:22.937608 28810 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:22.981817 21213 raft_consensus.cc:1240] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:22.988632 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:22.978111 28850 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.721s	user 0.007s	sys 0.004s
I20260709 13:09:22.993846 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.961031 28880 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:22.999712 28873 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.011811 21063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2496) took 1041 ms. Trace:
I20260709 13:09:22.964499 21066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39510 (request call id 2491) took 2075 ms. Trace:
I20260709 13:09:22.977916 28801 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:09:23.061333 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 106186us
I20260709 13:09:23.013346 28872 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:22.978655 28829 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.120721 21066 rpcz_store.cc:276] 0709 13:09:20.889325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.250178 (+1360853us) service_pool.cc:224] Handling call
0709 13:09:22.250479 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.250665 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:09:22.770162 (+519497us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.949173 (+179011us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.964157 (+ 14984us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:22.964209 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:22.964261 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.964324 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.964443 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103386,"prepare.queue_time_us":163753,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":29224,"raft.queue_time_us":19772,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":21007,"thread_start_us":573528,"threads_started":2}
W20260709 13:09:23.127758 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 100624us
I20260709 13:09:23.130405 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.078506 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:23.005630 28847 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:23.001328 28787 raft_consensus.cc:697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:23.090198 21217 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:23.089043 21221 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
W20260709 13:09:23.099905 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.135352 21215 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:23.068265 28710 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:23.106052 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 277749us
W20260709 13:09:23.099713 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 276273us
W20260709 13:09:23.096331 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 102546us
I20260709 13:09:23.144490 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:22.988337 28890 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.066063 28843 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:23.124397 21793 raft_consensus.cc:1240] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:23.057792 28662 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:23.131036 21213 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:23.131618 21218 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:23.093587 28706 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.722s	user 0.003s	sys 0.008s
I20260709 13:09:23.133533 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:23.120697 21063 rpcz_store.cc:276] 0709 13:09:21.970199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.378082 (+407883us) service_pool.cc:224] Handling call
0709 13:09:22.435648 (+ 57566us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:22.485369 (+ 49721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:22.485398 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:22.520392 (+ 34994us) log.cc:844] Serialized 52 byte log entry
0709 13:09:22.682252 (+161860us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:22.682295 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.682365 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:23.011524 (+329159us) raft_consensus.cc:1685] finished
0709 13:09:23.011550 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:23.011733 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17612,"raft.queue_time_us":49545,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":18380,"thread_start_us":104901,"threads_started":1,"wal-append.queue_time_us":467896}
I20260709 13:09:23.139184 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:23.190853 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 57408.1us
I20260709 13:09:23.140401 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:23.190910 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:23.087723 28889 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:23.145738 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.194083 21113 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:23.150771 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.151873 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.195413 21140 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 24.
I20260709 13:09:23.104221 28852 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:23.149415 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:23.154497 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:23.198261 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 48913.9us
I20260709 13:09:23.198939 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.199368 21110 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:23.199621 21213 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:23.200533 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.006239 28886 consensus_queue.cc:1059] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:09:23.095137 28810 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.145315 28177 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:09:23.140631 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:23.191891 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 51563us
I20260709 13:09:23.204947 21141 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:23.192415 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:23.091307 28764 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.066234 28667 consensus_queue.cc:260] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.144680 21185 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:23.146593 28710 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:23.156224 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 91069.8us
I20260709 13:09:23.105270 28880 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:23.141942 28888 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:09:23.156348 28850 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:23.193970 28889 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:23.011708 28879 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.193s
I20260709 13:09:23.070420 28878 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.212314 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:23.203912 28715 consensus_meta.cc:287] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.013s
I20260709 13:09:23.201054 21213 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 15.
I20260709 13:09:23.228701 21218 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:23.194125 28875 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:23.232579 28722 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:23.197319 28527 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.197078 28706 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:23.201438 28727 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:23.197396 28860 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:23.205395 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:23.344971 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 150976us
I20260709 13:09:23.268878 28886 raft_consensus.cc:2804] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:09:23.238499 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 93929.7us
I20260709 13:09:23.246361 21800 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:23.243418 28685 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:23.255050 28896 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:09:23.255616 28889 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:23.259397 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.275190 28722 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:23.199005 28757 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:09:23.201609 28845 consensus_queue.cc:1059] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 13:09:23.235507 28888 consensus_queue.cc:1243] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 9384358ce43f4e3fac2ae44bdf11c389 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:09:23.348328 28850 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.271157 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.275734 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:23.255050 28897 log.cc:927] Time spent T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:09:23.335754 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:23.408764 21217 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:23.249020 28667 raft_consensus.cc:3060] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:23.234571 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.410478 21221 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
W20260709 13:09:23.272220 28853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.014s	sys 0.000s
W20260709 13:09:23.347406 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 142100us
I20260709 13:09:23.416680 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:23.344609 28846 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:23.344676 28727 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.353631 28875 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:23.354157 28898 log.cc:927] Time spent T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:09:23.276510 28852 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:09:23.336860 28705 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.634s	user 0.011s	sys 0.000s
I20260709 13:09:23.362993 21796 raft_consensus.cc:1240] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:23.230343 28656 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:23.449148 21140 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: []
W20260709 13:09:23.367741 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 168461us
I20260709 13:09:23.370746 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.449987 21214 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:23.450951 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.451107 21110 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:23.270684 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 58468.1us
I20260709 13:09:23.399784 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:23.246560 28787 consensus_queue.cc:237] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.462265 28736 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.406637 21213 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:23.402274 21218 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:23.412619 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.417024 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:23.374969 28722 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:23.368768 28860 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.368577 28770 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:23.346213 28527 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.375805 28706 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:23.436335 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:23.414950 28853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.015s	sys 0.000s
I20260709 13:09:23.401218 28528 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:23.445621 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.479733 28829 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:23.401959 28903 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:23.414053 28177 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:23.373262 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:23.452803 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:23.403142 28886 raft_consensus.cc:697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:23.448025 28852 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:23.456597 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:23.456781 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:23.435654 28732 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:09:23.458739 21139 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 27. Current term is 28. Ops: []
I20260709 13:09:23.459088 28705 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.400493 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:23.540014 28688 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:23.468676 28911 raft_consensus.cc:3055] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:23.415758 28850 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.472182 28791 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:23.474890 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:23.435683 28727 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:23.655825 21140 raft_consensus.cc:3055] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:23.656952 21140 raft_consensus.cc:740] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:23.622997 28464 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.001s
W20260709 13:09:23.491458 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 90160.3us
I20260709 13:09:23.604566 28852 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:23.482782 28685 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.478350 21217 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:23.436792 28846 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:23.670156 21219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2535) took 1824 ms (client timeout 1933 ms). Trace:
W20260709 13:09:23.670588 21219 rpcz_store.cc:269] 0709 13:09:21.845644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.123300 (+277656us) service_pool.cc:224] Handling call
0709 13:09:23.669964 (+1546664us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540015ca54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:23.670091 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352062336}
I20260709 13:09:23.498376 28873 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:23.530965 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.683710 21218 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 24.
W20260709 13:09:23.684223 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 68886us
I20260709 13:09:23.532488 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.685014 21213 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 17.
I20260709 13:09:23.512826 21792 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 13:09:23.506060 28810 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:23.568408 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 132142us
I20260709 13:09:23.614210 28911 raft_consensus.cc:740] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:23.445852 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.602959 21142 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:23.603060 21214 raft_consensus.cc:2376] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22. Re-sending same reply.
I20260709 13:09:23.723528 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:23.724372 21214 raft_consensus.cc:2376] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 18. Re-sending same reply.
I20260709 13:09:23.725224 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:23.725868 21214 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:23.727130 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:23.728101 21792 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:23.603266 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 150581us
I20260709 13:09:23.574559 28860 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:09:23.592130 28842 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.801s	user 0.009s	sys 0.002s
I20260709 13:09:23.576251 28888 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:23.585919 28626 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:23.486599 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.749379 21143 raft_consensus.cc:2468] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:23.609361 28829 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:23.661523 21111 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:23.473755 28895 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:23.599865 28764 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:23.668349 28685 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.669441 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:23.759649 21217 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:23.671034 21800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 982) took 1236 ms. Trace:
I20260709 13:09:23.760506 21800 rpcz_store.cc:276] 0709 13:09:22.434864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.435021 (+   157us) service_pool.cc:224] Handling call
0709 13:09:22.435402 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.353797 (+918395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.670814 (+317017us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b540046aad4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:23.670984 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1864541568}
I20260709 13:09:23.608357 28845 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:23.630321 28899 log.cc:927] Time spent T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:09:23.673432 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:23.762044 21219 raft_consensus.cc:2376] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25. Re-sending same reply.
I20260709 13:09:23.575971 28913 consensus_queue.cc:1048] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.100s
W20260709 13:09:23.662756 28861 consensus_meta.cc:287] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.806s	user 0.007s	sys 0.005s
I20260709 13:09:23.707876 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:23.708269 28908 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:09:23.708720 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:23.708935 28907 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
W20260709 13:09:23.709702 28916 log.cc:927] Time spent T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:09:23.710413 28850 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:23.574920 28770 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.570453 21216 raft_consensus.cc:1240] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:23.660080 21140 consensus_queue.cc:260] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:23.720626 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.730962 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:23.468164 28904 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:09:23.730566 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:23.732352 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:23.733896 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:23.608888 28177 raft_consensus.cc:697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:23.740434 21214 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
W20260709 13:09:23.730829 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:23.741140 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:23.749435 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.603169 28903 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.712563 28757 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:23.757330 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 95867.6us
I20260709 13:09:23.765225 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.731976 28853 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:23.898677 21213 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 25. Current term is 26. Ops: []
I20260709 13:09:23.797775 21218 raft_consensus.cc:3060] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:23.742348 28842 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:23.803905 28911 consensus_queue.cc:260] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.753783 28895 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.808240 28527 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:23.920184 28895 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:23.831490 21219 raft_consensus.cc:1275] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:09:23.921514 28685 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:23.749105 28626 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.922395 28626 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:23.821739 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:23.923044 28705 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:23.923832 21800 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 13:09:23.924742 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:23.925581 28853 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:23.888872 19967 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 21083)
I20260709 13:09:23.850996 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.928681 21146 raft_consensus.cc:1275] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:09:23.928848 28853 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:23.893229 21143 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:23.893946 21801 raft_consensus.cc:2376] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24. Re-sending same reply.
I20260709 13:09:23.894181 21111 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:23.931365 28872 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.786s	user 0.001s	sys 0.011s
I20260709 13:09:23.932281 28915 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:23.932772 28915 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:23.933151 28915 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:23.870174 21792 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
I20260709 13:09:23.935840 21212 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:09:23.936965 28727 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:23.894634 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.940044 21800 raft_consensus.cc:1275] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:09:23.894486 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 169418us
I20260709 13:09:23.630106 28688 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.832793 21140 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:23.904435 21217 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:09:23.905861 21142 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:23.842983 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 1041) took 1015 ms. Trace:
I20260709 13:09:23.723606 28464 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.832681 28845 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.800033 28861 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:23.916939 28860 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:23.917593 28912 log.cc:927] Time spent T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:09:23.961025 28911 raft_consensus.cc:3060] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:23.919042 28903 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:23.919504 28770 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:23.904994 28757 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.964725 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.965682 28757 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:23.926347 28662 raft_consensus.cc:3055] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:23.966363 28662 raft_consensus.cc:740] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:23.967438 28662 consensus_queue.cc:260] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.969101 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:23.971058 28893 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:09:23.915162 28847 consensus_meta.cc:287] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.770s	user 0.010s	sys 0.003s
I20260709 13:09:23.900712 21214 raft_consensus.cc:1275] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:09:23.943071 21142 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
W20260709 13:09:23.944900 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 214412us
I20260709 13:09:23.986660 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:23.912988 28810 raft_consensus.cc:2804] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:23.942808 21215 rpcz_store.cc:276] 0709 13:09:22.827255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.830538 (+  3283us) service_pool.cc:224] Handling call
0709 13:09:23.842798 (+1012260us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b54000bc2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:23.842923 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137730432}
I20260709 13:09:23.910974 21218 raft_consensus.cc:2468] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 16.
I20260709 13:09:23.947219 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 3626) took 1662 ms. Trace:
I20260709 13:09:23.947922 21217 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:09:23.833799 28900 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.123s
I20260709 13:09:23.953722 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:23.950616 28527 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.953652 28933 consensus_queue.cc:1048] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:23.939045 28826 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.954941 28846 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:23.955680 21219 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:09:23.956736 28464 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:23.957257 28688 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:23.955340 28880 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.710s	user 0.010s	sys 0.001s
W20260709 13:09:23.959985 28710 consensus_meta.cc:287] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.719s	user 0.007s	sys 0.004s
W20260709 13:09:23.963979 28931 log.cc:927] Time spent T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.253s	user 0.003s	sys 0.000s
W20260709 13:09:23.966033 28667 consensus_meta.cc:287] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.547s	user 0.003s	sys 0.007s
I20260709 13:09:23.950508 28656 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:23.931967 28872 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:23.972937 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:23.971444 21145 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:23.943130 21139 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:23.979614 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:23.981287 21212 raft_consensus.cc:1275] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:09:23.971834 28736 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:23.952775 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:23.945348 21138 raft_consensus.cc:1240] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:09:23.941265 28829 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:23.994024 28903 consensus_queue.cc:1059] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:23.995895 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54660 (request call id 987) took 1509 ms. Trace:
I20260709 13:09:23.999756 28845 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.032146 21796 rpcz_store.cc:276] 0709 13:09:22.486237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.486773 (+   536us) service_pool.cc:224] Handling call
0709 13:09:22.487078 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.473348 (+986270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.995538 (+522190us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b5400701a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:23.995812 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2013931264}
I20260709 13:09:23.948872 28746 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.736281 28886 consensus_queue.cc:237] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:23.999055 21793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1530) took 1725 ms. Trace:
I20260709 13:09:24.003731 21144 rpcz_store.cc:276] 0709 13:09:22.284922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.285063 (+   141us) service_pool.cc:224] Handling call
0709 13:09:23.947011 (+1661948us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400434ed4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:23.947157 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099724544}
I20260709 13:09:23.953127 28853 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.006192 21143 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:24.011452 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:23.983933 28782 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.983721 28847 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:23.971939 28893 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.011416 28810 raft_consensus.cc:697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:24.007045 28757 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:09:24.028380 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 75651.3us
I20260709 13:09:24.026631 21794 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:24.012914 28842 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:09:24.017974 28826 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.036586 21793 rpcz_store.cc:276] 0709 13:09:22.274011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.278956 (+  4945us) service_pool.cc:224] Handling call
0709 13:09:22.323838 (+ 44882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.188619 (+864781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.998776 (+810157us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b54004b7dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:23.998968 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1797599616}
I20260709 13:09:24.026324 28656 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:24.041031 21187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.012219 28527 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.039558 21036 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:24.048550 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:24.024196 28710 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:24.052537 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:24.048735 28727 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:24.064981 21767 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:24.065956 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:24.066257 28700 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:24.066473 21767 leader_election.cc:336] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:24.066903 28700 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:24.055408 28933 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:24.056797 28826 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.053059 28656 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.059496 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.080389 21144 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:24.060845 28893 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:24.060575 28626 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:24.086304 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46826.5us
I20260709 13:09:24.021238 28880 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.041637 28845 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.043203 28860 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:24.072906 21767 leader_election.cc:336] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:24.033417 28736 consensus_queue.cc:1059] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
W20260709 13:09:24.083788 28921 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:09:24.081589 21185 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:24.084419 21220 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 11. Current term is 13. Ops: []
I20260709 13:09:24.050232 28842 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:24.045545 28667 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 13:09:24.110466 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.083806 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.074002 28933 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:24.018432 28177 consensus_queue.cc:237] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:24.067265 28843 consensus_meta.cc:287] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.867s	user 0.005s	sys 0.004s
I20260709 13:09:24.088611 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:24.121503 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.038524 28746 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:24.074163 28894 log.cc:927] Time spent T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.542s	user 0.000s	sys 0.010s
I20260709 13:09:24.068663 28757 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:24.090801 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.128396 21218 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 25.
I20260709 13:09:24.095795 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.043103 28782 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:24.090229 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.163335 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.164476 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.165462 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.165655 28782 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:24.166011 21142 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:24.126825 28903 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:24.074024 28928 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:24.168303 28746 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:24.101444 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.130335 28843 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.169620 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.170848 28843 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.170912 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.171677 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.171933 21218 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:24.172514 21212 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:09:24.173218 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.173333 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.174329 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.166587 28903 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.175611 28903 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:24.177001 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:24.177811 28702 consensus_queue.cc:1059] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:09:24.102114 21797 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:24.080875 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.073701 28935 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:24.178855 28702 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:24.179481 28913 raft_consensus.cc:3055] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:24.079231 28700 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:09:24.180244 28938 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:09:24.180820 28700 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.166867 28928 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.184075 28702 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:24.185029 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:24.185685 21767 leader_election.cc:336] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:24.185406 28702 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:24.186337 21767 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:24.111635 21793 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 25.
I20260709 13:09:24.123075 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:24.193085 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.124264 28710 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.194402 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:24.195199 28842 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.171015 21141 raft_consensus.cc:1240] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:09:24.196480 28842 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:24.197060 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.198288 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.134479 28464 raft_consensus.cc:3055] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:24.198830 28464 raft_consensus.cc:740] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:24.198956 28927 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 13:09:24.141021 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.142711 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.145910 28527 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.208201 21212 raft_consensus.cc:2376] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22. Re-sending same reply.
I20260709 13:09:24.148209 28626 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.148671 28667 consensus_queue.cc:1243] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 20.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:09:24.119096 28933 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:24.104465 28845 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.154585 28889 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:24.156064 28757 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.156806 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.161641 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:24.179226 28935 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.247469 28667 consensus_queue.cc:1059] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:24.248735 28935 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.043244 28853 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.249675 28656 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.251157 28853 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.251149 28746 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.251650 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:24.184310 28928 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.199705 28464 consensus_queue.cc:260] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.252020 28746 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.253074 28464 raft_consensus.cc:3060] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:24.136001 28872 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.111569 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.204128 21142 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:24.204339 28890 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.199028 21145 raft_consensus.cc:2393] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 17: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:24.256713 28746 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.108450 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.206810 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:09:24.259708 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 53005us
I20260709 13:09:24.104055 28860 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:24.262042 28860 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:24.262419 28901 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:24.262863 28901 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:24.264097 28936 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.343s	user 0.000s	sys 0.002s
W20260709 13:09:24.154984 28722 consensus_meta.cc:287] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.000s
I20260709 13:09:24.266382 28846 raft_consensus.cc:3055] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:24.266817 28846 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:24.208012 21219 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:24.268003 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.267906 28846 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.269157 28846 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:24.258301 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:24.257036 28843 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:24.271080 28843 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.225843 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.227941 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.227784 21140 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:24.272543 21215 raft_consensus.cc:2468] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:24.272450 28710 consensus_queue.cc:1059] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:24.273437 28710 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:24.274114 28710 raft_consensus.cc:3055] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:24.274453 28710 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:24.276018 28710 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.277316 28710 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:24.242059 21143 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:24.278393 28845 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.279641 21140 raft_consensus.cc:1275] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
W20260709 13:09:24.242403 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 80848.7us
I20260709 13:09:24.220811 28850 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.280889 28850 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.256820 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:24.283460 21142 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:24.283624 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.284758 21146 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 28: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:24.273597 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.286024 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.286808 21215 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.180085 28913 raft_consensus.cc:740] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:24.271533 28843 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:24.278009 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1043) took 1446 ms. Trace:
I20260709 13:09:24.304975 21216 rpcz_store.cc:276] 0709 13:09:22.831678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.831791 (+   113us) service_pool.cc:224] Handling call
0709 13:09:22.832040 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.832254 (+1000214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.277787 (+445533us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b5400102f54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:24.277951 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697690496}
I20260709 13:09:24.299960 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:24.306380 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.245081 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.307466 21798 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:24.269166 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.308833 21219 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:24.242007 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:24.309731 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 67792.8us
I20260709 13:09:24.310674 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:24.299413 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.313041 21796 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:24.317564 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:24.321485 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.128648 28852 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.190979 28895 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.333362 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.338244 21793 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:24.335865 21797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1533) took 1444 ms. Trace:
I20260709 13:09:24.338651 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:24.263240 28901 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:24.084967 28829 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.339402 21799 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:24.339025 21797 rpcz_store.cc:276] 0709 13:09:22.891896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.892169 (+   273us) service_pool.cc:224] Handling call
0709 13:09:22.892510 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:24.178551 (+1286041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.332247 (+153696us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54006c11d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:24.335803 (+  3556us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2781389056}
I20260709 13:09:24.241855 28847 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.344305 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.348238 21145 raft_consensus.cc:1275] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:09:24.323505 28927 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:09:24.125387 28810 consensus_queue.cc:237] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.266788 28722 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.323082 28801 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:24.334342 28527 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:24.323422 28764 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:24.334108 28826 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:24.367720 21141 raft_consensus.cc:1240] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 26. Current term is 27. Ops: []
I20260709 13:09:24.369078 21215 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 17.
I20260709 13:09:24.386401 28909 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:24.386973 28909 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.387435 28909 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:24.388100 21144 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:24.370000 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.392397 28852 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.396461 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.399004 21144 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:09:24.403570 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.404233 21212 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:24.404299 28895 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.375946 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:24.344390 28935 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:24.351586 28736 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.366416 21142 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:24.323540 28715 consensus_queue.cc:1048] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:09:24.319890 28853 consensus_queue.cc:1059] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.062s
W20260709 13:09:24.436199 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 60327.5us
I20260709 13:09:24.437705 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:24.438784 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 316851us
I20260709 13:09:24.371304 28527 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:24.372982 28850 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.449138 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:24.450407 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:24.453440 21212 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:24.454627 28527 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:24.358397 28847 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.379483 28940 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.384682 28656 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.475899 28702 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:24.440884 28845 raft_consensus.cc:2764] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:09:24.473291 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:24.479662 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.448298 28177 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:24.494024 28933 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.384240 28915 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.411088 28826 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.483867 21111 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:24.466624 28853 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:24.520694 28910 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.810s	user 0.000s	sys 0.003s
I20260709 13:09:24.521237 28852 raft_consensus.cc:3055] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:24.472967 28850 raft_consensus.cc:2804] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:24.457402 28911 consensus_queue.cc:1059] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:09:24.447397 28736 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:24.536051 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 223989us
I20260709 13:09:24.438277 28935 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.460146 28861 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:09:24.469547 28902 consensus_queue.cc:1048] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:09:24.543196 21216 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 25: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:24.498888 28177 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:24.532641 28929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.009s	sys 0.018s
I20260709 13:09:24.379730 28913 consensus_queue.cc:260] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.521253 28845 raft_consensus.cc:2804] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:24.533206 28852 raft_consensus.cc:740] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:24.561174 21212 raft_consensus.cc:2410] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 22 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:09:24.500815 28872 consensus_queue.cc:1048] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:09:24.532931 28883 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:24.571784 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:24.536922 28850 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:24.563019 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:24.552534 28929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.009s	sys 0.018s
I20260709 13:09:24.540686 28688 consensus_queue.cc:1048] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:09:24.551957 28932 consensus_queue.cc:1059] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 13:09:24.568347 28845 raft_consensus.cc:697] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:24.537467 28705 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:09:24.480309 28940 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.573663 21111 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:24.576864 21142 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:24.577301 21765 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:24.554530 28177 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:24.554193 28656 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:09:24.560405 28878 consensus_queue.cc:1059] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 13:09:24.579456 28850 raft_consensus.cc:3060] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:24.564065 28919 consensus_queue.cc:1048] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:09:24.609171 21142 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:24.610435 21214 raft_consensus.cc:2410] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 13:09:24.608181 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:24.548091 28903 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.596698 28656 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:24.623965 28656 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.624233 21218 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:24.576046 28702 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:09:24.576864 28883 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.601570 28918 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:24.613721 28624 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:24.624361 28903 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.634303 28656 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:24.641970 28889 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.642776 28889 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.646006 28889 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:24.647831 28946 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 13:09:24.649521 28942 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:09:24.651773 28944 log.cc:927] Time spent T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:09:24.651968 28915 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.667405 28918 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:24.668591 28918 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.671949 28933 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.672576 28883 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.675621 28935 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.638949 28764 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:24.677948 28764 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:24.678371 28764 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:24.680534 28917 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.970s	user 0.003s	sys 0.004s
I20260709 13:09:24.682238 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1129) took 1534 ms. Trace:
I20260709 13:09:24.682668 21147 rpcz_store.cc:276] 0709 13:09:23.147667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.147804 (+   137us) service_pool.cc:224] Handling call
0709 13:09:23.148133 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:23.148679 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:23.148712 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:23.157053 (+  8341us) log.cc:844] Serialized 26 byte log entry
0709 13:09:23.491813 (+334760us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:23.491854 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.491924 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:24.679846 (+1187922us) spinlock_profiling.cc:243] Waited 4.06 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:24.681880 (+  2034us) raft_consensus.cc:1685] finished
0709 13:09:24.681904 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:24.682159 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1973,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3181,"spinlock_wait_cycles":9343744,"thread_start_us":261210,"threads_started":1,"wal-append.queue_time_us":414006}
I20260709 13:09:24.687903 28826 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.689670 28847 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.690834 28845 consensus_queue.cc:237] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.692173 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1045) took 1858 ms. Trace:
I20260709 13:09:24.692703 21220 rpcz_store.cc:276] 0709 13:09:22.833343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.943692 (+110349us) service_pool.cc:224] Handling call
0709 13:09:22.965248 (+ 21556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:24.105967 (+1140719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.691870 (+585903us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540015c554. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:24.692087 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2573423360}
I20260709 13:09:24.666335 28860 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.694288 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.695685 21799 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 13.
I20260709 13:09:24.695047 21220 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 27.
W20260709 13:09:24.709035 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.696235 28624 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.698853 28801 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:24.716224 21035 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:24.608263 28852 consensus_queue.cc:260] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.718143 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.679126 28909 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:24.793756 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 77666.2us
I20260709 13:09:24.795553 21220 raft_consensus.cc:2368] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13.
I20260709 13:09:24.796470 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:24.787492 21184 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:24.797914 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:24.745404 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.719516 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 269169us
I20260709 13:09:24.800653 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
W20260709 13:09:24.787263 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.711028 28915 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:24.739969 28787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.276s	user 0.017s	sys 0.030s
W20260709 13:09:24.823431 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.826117 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:24.828478 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.830688 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.832962 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:24.833491 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.834730 21795 raft_consensus.cc:2410] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 25 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:09:24.718237 28624 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:24.813721 28749 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:24.824288 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:24.837491 28464 consensus_meta.cc:287] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.001s
I20260709 13:09:24.837821 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.838023 28861 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.838447 21219 raft_consensus.cc:2410] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:09:24.841444 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:24.842937 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:24.843488 28787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.380s	user 0.021s	sys 0.030s
I20260709 13:09:24.838491 28861 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.843957 28853 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:24.845849 28861 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.846479 28861 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:24.846895 28861 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.847421 28749 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:24.848480 28749 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.849797 28624 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.852653 28177 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.719323 28527 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.861639 28527 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.862176 28624 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.862425 28527 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.859580 28177 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.812295 28860 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.853632 28915 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.865474 28915 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.855341 28861 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:24.867928 28940 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.874922 28940 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.875516 28940 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.877697 28940 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:24.879848 28909 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:24.886296 28880 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:24.889153 21794 raft_consensus.cc:1240] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:24.863044 28527 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:24.892459 28947 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:24.893080 28947 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:24.894402 28527 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:24.893993 28947 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:24.898949 28929 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:24.899621 28929 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:24.902168 28770 raft_consensus.cc:3055] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:09:24.902746 28770 raft_consensus.cc:740] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:24.902915 28846 consensus_meta.cc:287] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.633s	user 0.008s	sys 0.003s
I20260709 13:09:24.904058 28770 consensus_queue.cc:260] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.813545 28829 consensus_queue.cc:1059] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.092s
I20260709 13:09:24.814826 28700 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:24.821297 28736 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:24.900733 28929 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:24.906159 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:24.941689 21799 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:24.839164 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:24.929459 21767 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 26 similar messages]
I20260709 13:09:24.944213 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.944999 21142 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 25: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:24.934697 21215 raft_consensus.cc:2410] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:09:24.939970 21146 raft_consensus.cc:1240] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:24.947369 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:24.948046 21801 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:24.944810 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:24.874459 28861 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:24.950286 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:24.947332 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.951112 21221 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:24.957871 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:24.959914 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:24.964110 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:24.964967 21800 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:24.937054 28700 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.981755 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:24.948598 28667 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.948674 28950 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:24.998376 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 381213us
I20260709 13:09:24.948328 28922 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.952268 28826 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:25.113394 28662 raft_consensus.cc:3060] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:24.997094 28700 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:25.114362 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 3648) took 1008 ms. Trace:
I20260709 13:09:24.961249 28929 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:25.003154 21799 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:25.115108 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56804 (request call id 3494) took 1001 ms. Trace:
I20260709 13:09:25.119465 21765 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:25.033375 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.125408 21144 rpcz_store.cc:276] 0709 13:09:24.105644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.122783 (+ 17139us) service_pool.cc:224] Handling call
0709 13:09:25.114139 (+991356us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54002959d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:25.114295 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459538816}
W20260709 13:09:25.128057 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.068063 21214 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:09:25.130136 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:25.120525 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.015s	sys 0.019s
W20260709 13:09:25.131585 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.016s	sys 0.020s
W20260709 13:09:25.136227 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.126253 21212 rpcz_store.cc:276] 0709 13:09:24.113390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.472979 (+359589us) service_pool.cc:224] Handling call
0709 13:09:25.114945 (+641966us) spinlock_profiling.cc:243] Waited 75.8 ms on lock 0x7b5400103454. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:25.115064 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174277504}
I20260709 13:09:25.002988 21800 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 26.
W20260709 13:09:25.012197 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.999590 21801 raft_consensus.cc:2468] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 19.
I20260709 13:09:25.086979 21221 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
W20260709 13:09:25.104161 28954 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:09:24.960979 28770 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:09:25.076051 28710 consensus_meta.cc:287] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.771s	user 0.012s	sys 0.000s
I20260709 13:09:25.144347 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:25.154855 21035 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:25.084438 28843 consensus_meta.cc:287] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.750s	user 0.013s	sys 0.000s
W20260709 13:09:25.124464 28901 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.761s	user 0.013s	sys 0.000s
W20260709 13:09:24.979583 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 291351us
I20260709 13:09:25.172932 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:25.158092 21212 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:09:25.156455 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 313561us
I20260709 13:09:25.181452 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:25.107918 28928 raft_consensus.cc:3055] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 13:09:25.112773 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 353539us
I20260709 13:09:25.156183 28932 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:25.194865 21220 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:09:25.116801 28826 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.201226 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.182466 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:25.118090 28667 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.159595 28955 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:25.187491 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 277856us
I20260709 13:09:25.196519 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:25.202188 21144 raft_consensus.cc:2393] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 26: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:25.210527 21111 leader_election.cc:400] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13.
I20260709 13:09:25.211402 21111 leader_election.cc:403] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:09:25.176199 28829 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:25.125772 28922 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.220252 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:25.183218 28880 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:25.176960 28962 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:25.226135 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.195755 28624 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:09:25.239410 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.239619 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.240015 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.241951 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.242705 21142 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:25.241983 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.243744 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.245435 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:25.246181 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.247157 21144 raft_consensus.cc:2410] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 13:09:25.239452 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:25.248519 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.188735 28928 raft_consensus.cc:740] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:25.094988 28764 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:25.249852 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:25.182678 28734 consensus_queue.cc:1059] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:09:25.175992 28843 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.211898 28939 raft_consensus.cc:3055] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:25.259331 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:25.207394 28847 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:25.215368 28845 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:09:25.266062 21795 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:25.178045 28901 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.269933 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:25.271742 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 69186.6us
I20260709 13:09:25.279740 21796 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:25.286592 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 66426.6us
I20260709 13:09:25.289812 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:25.243548 28727 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13.
I20260709 13:09:25.234174 28829 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:25.261673 28935 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:25.301344 21143 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:25.129027 28950 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.268582 28939 raft_consensus.cc:740] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:25.314623 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2569) took 1065 ms. Trace:
I20260709 13:09:25.273151 28845 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.315045 21215 rpcz_store.cc:276] 0709 13:09:24.248806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.396193 (+147387us) service_pool.cc:224] Handling call
0709 13:09:25.314433 (+918240us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b540015cf54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:25.314568 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235443968}
I20260709 13:09:25.313153 28764 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:25.322950 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:25.328907 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:25.217549 28955 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.331965 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:25.322166 28852 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:25.344430 28901 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:25.346699 28950 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:25.348031 28527 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:25.388464 28527 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.388939 28527 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.348187 28826 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:25.348805 28829 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.391016 28527 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:25.394840 28527 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:25.349325 28860 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:09:25.334903 28935 raft_consensus.cc:2804] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:25.335798 28722 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:25.279155 28847 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.395968 28935 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.396375 28935 raft_consensus.cc:3060] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:25.397666 28847 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:25.317967 28845 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:09:25.330067 28656 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:25.353045 28667 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:25.356159 28939 consensus_queue.cc:260] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:25.359484 28850 consensus_meta.cc:287] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.001s
I20260709 13:09:25.315303 28928 consensus_queue.cc:260] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.365772 28976 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:25.367379 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.376024 28959 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:09:25.335662 28801 raft_consensus.cc:3055] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:09:25.378315 28965 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:09:25.380833 21138 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: [17.8-19.9]
I20260709 13:09:25.351094 28922 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:25.384662 28968 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:25.386693 28955 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:25.405546 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 71989.2us
W20260709 13:09:25.416137 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.395416 28527 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.411530 28705 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.411576 28722 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:25.411716 28656 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.424314 28624 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:25.434227 28939 raft_consensus.cc:3060] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:25.433140 28850 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.490460 28850 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:25.491161 28850 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:09:25.491801 28969 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.181s	user 0.003s	sys 0.000s
I20260709 13:09:25.491729 28850 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.494012 28850 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:25.500168 28850 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.500998 28850 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.504285 28850 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:25.443362 28801 raft_consensus.cc:740] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:25.444211 28928 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:25.455160 28968 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.474900 28847 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:25.438505 28976 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.449647 28667 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:25.481573 21141 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:25.475539 28955 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:25.475530 28929 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:25.480432 28656 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:25.478866 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2570) took 1229 ms. Trace:
I20260709 13:09:25.518595 21219 rpcz_store.cc:276] 0709 13:09:24.249113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.349347 (+100234us) service_pool.cc:224] Handling call
0709 13:09:25.478666 (+1129319us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b540034c1d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:25.478813 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024861440}
I20260709 13:09:25.509393 21216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56816 (request call id 2575) took 1255 ms. Trace:
I20260709 13:09:25.510160 21218 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 24. Current term is 25. Ops: []
W20260709 13:09:25.511250 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 233110us
I20260709 13:09:25.519623 21216 rpcz_store.cc:276] 0709 13:09:24.256799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.306129 (+ 49330us) service_pool.cc:224] Handling call
0709 13:09:25.509077 (+1202948us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540034b2d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:25.509306 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542872448}
I20260709 13:09:25.451081 28860 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.520651 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:25.559741 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:25.560612 21801 raft_consensus.cc:2393] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 23: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:25.561467 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:25.561645 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:25.562934 21797 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 25. Current term is 26. Ops: []
I20260709 13:09:25.576514 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:25.577584 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:25.577515 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:25.512534 28968 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:25.480578 28705 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.480764 28722 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.512482 28847 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.622831 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:25.623661 21801 raft_consensus.cc:2368] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13.
I20260709 13:09:25.624750 21800 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:25.625818 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:25.620437 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.629336 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.620908 21767 leader_election.cc:336] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:25.662241 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.670265 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:25.700711 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 85081.7us
I20260709 13:09:25.701325 21144 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:25.514158 28667 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.570339 28973 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:25.669088 21800 raft_consensus.cc:1240] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
W20260709 13:09:25.655400 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.677472 28974 log.cc:927] Time spent T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 13:09:25.678778 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:25.682960 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.712157 21142 raft_consensus.cc:2393] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 24: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:25.678761 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:25.712626 21146 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:25.683455 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.713459 21141 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:25.672009 21145 raft_consensus.cc:1240] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:25.691548 28881 raft_consensus.cc:3055] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:09:25.694504 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 441189us
I20260709 13:09:25.698396 21147 raft_consensus.cc:1275] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 17. (term mismatch)
I20260709 13:09:25.719146 21147 pending_rounds.cc:85] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 16
I20260709 13:09:25.719456 21147 pending_rounds.cc:107] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 13:09:25.719707 21147 raft_consensus.cc:2889] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:25.698993 21110 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:25.521588 28929 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.716539 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.721129 21110 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:25.567097 28860 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:25.722764 21187 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:25.730433 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:25.732028 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:25.738523 21110 leader_election.cc:400] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 12 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 12. Current term is 13.
I20260709 13:09:25.738399 21144 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 25.
I20260709 13:09:25.745939 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:25.517459 28955 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.741766 20963 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 15 to 17, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 17 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:25.762971 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:25.763778 21141 raft_consensus.cc:2393] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:25.711072 28947 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:25.766021 28940 raft_consensus.cc:3055] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:25.718546 28881 raft_consensus.cc:740] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:25.782588 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:25.740835 28983 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.723042 28911 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:25.733848 28890 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.740990 28727 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:25.757531 21036 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:25.708794 28935 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.710740 28902 consensus_queue.cc:1059] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 13:09:25.754760 28852 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:25.789542 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:25.789985 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.896039 21141 raft_consensus.cc:2410] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:09:25.896107 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.764683 28973 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:25.899739 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 769670us
W20260709 13:09:25.902129 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.902561 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.902935 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.783421 28939 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:25.780970 28940 raft_consensus.cc:740] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:25.745144 28960 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:09:25.869253 28890 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:25.877808 28845 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.791932 28983 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.878749 28929 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:25.880784 28968 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:25.887770 28955 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:25.656966 28801 consensus_queue.cc:260] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:25.894831 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.896970 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.951686 28889 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:25.657703 28624 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.904316 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:25.958709 28981 log.cc:927] Time spent T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:09:25.952535 28850 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:25.875120 28843 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:25.951843 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49579.4us
I20260709 13:09:25.784672 28947 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:25.897692 28527 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:25.891417 28909 consensus_queue.cc:1048] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:09:25.793236 21146 raft_consensus.cc:2468] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:25.949287 28911 raft_consensus.cc:3055] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:09:26.003134 28667 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.004693 28935 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.008185 28656 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:26.010672 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 291701us
I20260709 13:09:26.011423 28886 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:25.906720 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 149269us
W20260709 13:09:25.951305 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.959148 28727 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:09:25.963419 28715 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:25.986898 28982 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.230s	user 0.000s	sys 0.003s
I20260709 13:09:26.017449 28976 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.015926 28850 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.018929 28749 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:26.047911 28947 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:26.018769 28933 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.057317 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:26.060509 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.960976 28852 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.019517 28889 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:25.959686 28705 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:25.960562 28662 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.950537 28881 consensus_queue.cc:260] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.022912 28911 raft_consensus.cc:740] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:26.084895 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:26.051484 28847 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.073463 28850 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:26.072103 28710 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.109656 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:26.110371 21216 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:26.111073 21146 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:09:26.112141 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:26.112879 21146 raft_consensus.cc:2368] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:26.070387 28829 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:26.114317 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:26.115526 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.093063 28722 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:26.116932 21212 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22. Ops: []
W20260709 13:09:26.104861 28700 consensus_meta.cc:287] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.958s	user 0.007s	sys 0.004s
I20260709 13:09:26.122601 21216 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 25.
I20260709 13:09:26.076118 28749 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.089702 28947 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:26.133889 21220 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:26.035215 28940 consensus_queue.cc:260] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.117908 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:26.090399 28860 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.068013 28715 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.101217 28985 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:09:26.140743 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.157116 28889 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.099565 28662 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.164217 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.164453 28976 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.165524 28845 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 29 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.167436 28749 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:26.176471 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:26.178630 28913 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:26.111987 28901 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:26.089881 28922 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.063s
I20260709 13:09:26.180820 21220 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:26.071416 28973 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.130875 28881 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:26.091743 28852 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:26.186066 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.089262 28933 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.155511 28847 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:26.197021 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.189033 28985 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:26.215343 21147 raft_consensus.cc:2393] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 27: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:26.231034 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:26.129626 28710 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.217890 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.225659 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:26.232909 21216 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:09:26.190100 28984 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:09:26.206007 21184 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:26.153553 28829 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.160007 28700 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.244206 21215 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 24.
I20260709 13:09:26.196441 28978 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.234700 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:26.220590 28909 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.236389 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:26.226951 28968 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.234692 21220 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:26.220391 28749 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.226166 28940 raft_consensus.cc:3060] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:26.251461 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.184186 28911 consensus_queue.cc:260] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.233292 28826 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.278214 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:26.279608 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:26.281426 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:26.245396 28890 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.230183 28886 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.195875 28976 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.292819 21216 raft_consensus.cc:2393] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 22: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:09:26.283360 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.304293 28662 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:26.306041 28662 raft_consensus.cc:2804] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:26.306566 28662 raft_consensus.cc:697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:26.307421 28968 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.307950 28968 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.308364 28933 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:26.308954 28933 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:26.233765 28847 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.285125 28749 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.316098 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.316851 21216 raft_consensus.cc:2468] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:26.316766 28886 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.317862 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:26.318516 21216 raft_consensus.cc:2368] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:09:26.318900 28886 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:26.319728 21111 leader_election.cc:400] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:09:26.320021 21111 leader_election.cc:403] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:09:26.320744 28890 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249) with higher term. Message: Invalid argument: T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23.
I20260709 13:09:26.296563 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:26.321812 21147 raft_consensus.cc:2468] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:26.322598 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:26.254705 28983 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:26.316433 28987 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:09:26.228034 28901 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.327515 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:26.328092 21147 raft_consensus.cc:2468] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:26.328454 28901 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:26.330826 28996 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.331523 28996 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.334116 28996 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:26.298743 28911 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:26.295585 28826 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.341984 28826 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:26.364619 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.381915 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.293900 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.406924 21220 raft_consensus.cc:2468] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:26.278244 28909 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:26.426144 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.427099 21216 raft_consensus.cc:2468] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:26.408003 28656 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.437528 28749 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:26.437988 28710 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:26.440891 28989 log.cc:927] Time spent T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:09:26.419219 28985 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:26.448618 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 212292us
I20260709 13:09:26.449512 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:26.280216 28978 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:26.450832 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.452697 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.453511 21219 raft_consensus.cc:2468] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 17.
I20260709 13:09:26.454664 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:26.455912 21216 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:26.471163 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:26.525079 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.528023 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.588253 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:26.589357 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.589708 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2584) took 1993 ms. Trace:
I20260709 13:09:26.452442 28852 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:26.475756 28770 consensus_meta.cc:287] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 1.307s	user 0.014s	sys 0.000s
I20260709 13:09:26.468647 28715 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:26.467964 28749 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:26.468163 28890 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:26.425571 28662 consensus_queue.cc:237] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:26.620023 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.626861 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.644506 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:26.635819 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.664346 28993 log.cc:927] Time spent T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:09:26.666018 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2583) took 2070 ms. Trace:
I20260709 13:09:26.666507 21221 rpcz_store.cc:276] 0709 13:09:24.595526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.154122 (+558596us) service_pool.cc:224] Handling call
0709 13:09:25.154511 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:25.948321 (+793810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:25.948354 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:25.954177 (+  5823us) log.cc:844] Serialized 26 byte log entry
0709 13:09:26.216970 (+262793us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:26.217028 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.217100 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:26.665699 (+448599us) raft_consensus.cc:1685] finished
0709 13:09:26.665734 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:26.665939 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":820912,"raft.queue_time_us":793973,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5794,"thread_start_us":210603,"threads_started":1,"wal-append.queue_time_us":403821}
I20260709 13:09:26.669127 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:26.674685 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.682331 21221 raft_consensus.cc:2410] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:09:26.642732 28829 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:26.701781 21141 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:26.630393 28890 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:26.714588 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.716408 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.730931 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.731767 21219 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:26.680220 28700 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:26.770299 21216 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:26.705300 28727 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.811105 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:26.780893 29000 log.cc:927] Time spent T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:09:26.777120 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:26.812634 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.799431 28962 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.458s	user 0.036s	sys 0.040s
I20260709 13:09:26.794451 28909 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:26.834918 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.835731 21219 raft_consensus.cc:2393] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 26: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
W20260709 13:09:26.830502 28962 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.533s	user 0.036s	sys 0.041s
I20260709 13:09:26.840306 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:26.841634 21216 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:26.828490 28727 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:26.838358 28978 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:26.839751 28962 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:26.894881 29006 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:09:26.892845 29001 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 13:09:26.891916 28928 consensus_meta.cc:287] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 1.379s	user 0.002s	sys 0.012s
I20260709 13:09:26.907245 28890 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:26.952148 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.954916 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 271170us
W20260709 13:09:26.963706 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.964486 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.966210 21146 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 26.
W20260709 13:09:26.967314 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.968833 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.974542 28881 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.750s	user 0.011s	sys 0.004s
W20260709 13:09:26.976967 28911 consensus_meta.cc:287] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.001s
I20260709 13:09:26.978324 28881 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.980288 28881 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:26.982682 28933 consensus_meta.cc:287] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Time spent flushing consensus metadata: real 0.673s	user 0.012s	sys 0.000s
I20260709 13:09:27.006498 28933 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:27.013079 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:27.003537 28940 consensus_meta.cc:287] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.718s	user 0.012s	sys 0.008s
I20260709 13:09:27.004436 28962 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.005602 21140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50996 (request call id 1146) took 1327 ms. Trace:
I20260709 13:09:27.041091 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1057) took 2872 ms. Trace:
I20260709 13:09:26.590092 21214 rpcz_store.cc:276] 0709 13:09:24.596269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.164695 (+568426us) service_pool.cc:224] Handling call
0709 13:09:25.187345 (+ 22650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.187557 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:09:25.211249 (+ 23692us) log.cc:844] Serialized 14 byte log entry
0709 13:09:26.476111 (+1264862us) log.cc:844] Serialized 14 byte log entry
0709 13:09:26.589361 (+113250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:26.589417 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:26.589469 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.589549 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:26.589659 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":699895,"prepare.queue_time_us":224862,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":2908,"raft.queue_time_us":708544,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":13060,"thread_start_us":562710,"threads_started":3,"wal-append.queue_time_us":220964}
W20260709 13:09:27.046238 28850 consensus_meta.cc:287] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.916s	user 0.010s	sys 0.014s
I20260709 13:09:27.057410 28850 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.048442 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1061) took 2497 ms. Trace:
W20260709 13:09:27.048501 29012 log.cc:927] Time spent T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:09:27.051110 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:27.062911 21220 raft_consensus.cc:2393] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 18: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:27.060926 21218 rpcz_store.cc:276] 0709 13:09:24.550628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.115364 (+564736us) service_pool.cc:224] Handling call
0709 13:09:25.509934 (+394570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.519845 (+  9911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.047810 (+1527965us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b540034b2d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:27.048135 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906563328}
I20260709 13:09:27.054804 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:27.063323 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.056015 21140 rpcz_store.cc:276] 0709 13:09:25.677928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.678112 (+   184us) service_pool.cc:224] Handling call
0709 13:09:25.678431 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.708610 (+ 30179us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:09:25.785796 (+ 77186us) log.cc:844] Serialized 14 byte log entry
0709 13:09:26.074504 (+288708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:26.074558 (+    54us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:26.074606 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.004953 (+930347us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b5400423bd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:27.005041 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.005354 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":196712,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":108,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5151,"spinlock_wait_cycles":68836096,"thread_start_us":339323,"threads_started":2,"wal-append.queue_time_us":444719}
W20260709 13:09:27.067518 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.068194 28757 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.069052 28700 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.069144 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:27.093081 21219 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:27.096189 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.093078 28700 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:27.099640 28757 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:27.056548 21213 rpcz_store.cc:276] 0709 13:09:24.168706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.496740 (+328034us) service_pool.cc:224] Handling call
0709 13:09:25.478783 (+982043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:26.001123 (+522340us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:09:26.001155 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:26.001311 (+   156us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:26.001364 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.001436 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.039751 (+1038315us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b18003fd360. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f3fd2a6 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4
0709 13:09:27.040566 (+   815us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b540034c1d0. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:27.040819 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":521929,"raft.queue_time_us":522070,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4242,"spinlock_wait_cycles":2281180288}
I20260709 13:09:27.101063 28700 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.101878 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:27.075515 29013 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.153s	user 0.000s	sys 0.002s
I20260709 13:09:27.102250 28757 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.076093 28852 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:27.104180 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 49647us
I20260709 13:09:27.104480 28852 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.077003 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.087440 28902 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:27.101948 28995 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.092167 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.108374 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:27.112085 28757 raft_consensus.cc:3060] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:27.126556 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:27.121411 21218 raft_consensus.cc:2468] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:27.107658 28902 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:27.129572 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:27.131366 21220 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:27.132812 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.134395 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.134671 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.135764 21213 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:27.122490 28700 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.139180 28700 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:27.140290 28976 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.143164 28902 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.141144 28850 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.144418 21216 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:27.143996 21218 raft_consensus.cc:2410] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 13:09:27.147084 21143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3707) took 1904 ms. Trace:
I20260709 13:09:27.147192 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.147706 21143 rpcz_store.cc:276] 0709 13:09:25.242313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.259193 (+ 16880us) service_pool.cc:224] Handling call
0709 13:09:25.259513 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.306394 (+ 46881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.146581 (+1840187us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b5400279a54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:27.146894 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95715072}
I20260709 13:09:27.144810 28968 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.148284 21218 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
W20260709 13:09:27.152935 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.161032 21798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54660 (request call id 1009) took 1583 ms (client timeout 2046 ms). Trace:
W20260709 13:09:27.161688 21798 rpcz_store.cc:269] 0709 13:09:25.577132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.577251 (+   119us) service_pool.cc:224] Handling call
0709 13:09:27.160542 (+1583291us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400380254. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:27.160745 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3407776512}
I20260709 13:09:27.164137 21796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54648 (request call id 1544) took 2213 ms. Trace:
I20260709 13:09:27.165102 21214 raft_consensus.cc:2368] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:27.166174 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:27.126771 28995 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.164630 21796 rpcz_store.cc:276] 0709 13:09:24.950293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.953893 (+  3600us) service_pool.cc:224] Handling call
0709 13:09:24.954249 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.293116 (+338867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.163524 (+1870408us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54006c1e54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:27.163890 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748078976}
W20260709 13:09:27.238492 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.230572 28624 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.198370 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1070) took 1047 ms. Trace:
I20260709 13:09:27.231974 29005 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:27.236508 28973 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.231984 21145 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:09:27.253017 28973 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.206338 28902 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:27.252890 29005 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.196695 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.205206 20963 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 16 to 17, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 17 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:27.256915 29005 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.197822 21146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3718) took 1091 ms. Trace:
I20260709 13:09:27.258095 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2591) took 1058 ms. Trace:
I20260709 13:09:27.141711 28700 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.227072 28847 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:27.223786 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.174683 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.252460 21212 rpcz_store.cc:276] 0709 13:09:26.150441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.150579 (+   138us) service_pool.cc:224] Handling call
0709 13:09:26.160982 (+ 10403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:26.329781 (+168799us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:09:26.726792 (+397011us) log.cc:844] Serialized 14 byte log entry
0709 13:09:27.124701 (+397909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:27.128956 (+  4255us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:09:27.131809 (+  2853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.197819 (+ 66010us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540015c054. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:27.197896 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.198119 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3366,"prepare.queue_time_us":295112,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":154,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":213,"spinlock_wait_cycles":387511552,"thread_start_us":777853,"threads_started":2,"wal-append.queue_time_us":330466}
I20260709 13:09:27.261610 21212 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:09:27.168229 28861 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.151041 21141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49718 (request call id 3711) took 1433 ms. Trace:
I20260709 13:09:27.257679 21146 rpcz_store.cc:276] 0709 13:09:26.106289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.117664 (+ 11375us) service_pool.cc:224] Handling call
0709 13:09:27.197533 (+1079869us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54004254d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:27.197730 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1802216704}
I20260709 13:09:27.263082 21141 rpcz_store.cc:276] 0709 13:09:25.717130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.900831 (+183701us) service_pool.cc:224] Handling call
0709 13:09:27.150620 (+1249789us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b540032cdd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:27.150827 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1584476928}
I20260709 13:09:27.233278 21144 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:09:27.252185 28624 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.233953 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.267661 21142 raft_consensus.cc:2410] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 13:09:27.239015 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:27.250871 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 122264us
W20260709 13:09:27.258158 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.268896 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.229501 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.262557 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2592) took 1063 ms. Trace:
I20260709 13:09:27.270078 21214 rpcz_store.cc:276] 0709 13:09:26.199184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.256648 (+1057464us) service_pool.cc:224] Handling call
0709 13:09:27.257107 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.257670 (+   563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:27.257740 (+    70us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:27.257808 (+    68us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:27.257858 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.257928 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.258041 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":714,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130}
I20260709 13:09:27.270841 21212 rpcz_store.cc:276] 0709 13:09:26.199446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.260675 (+1061229us) service_pool.cc:224] Handling call
0709 13:09:27.261122 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:27.262175 (+  1053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.262469 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:27.227365 28881 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.244577 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:27.195957 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 1068) took 1081 ms. Trace:
I20260709 13:09:27.187723 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2574) took 2937 ms. Trace:
I20260709 13:09:27.277954 21220 raft_consensus.cc:2468] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
W20260709 13:09:27.278726 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.267593 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.232069 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.277478 21215 rpcz_store.cc:276] 0709 13:09:26.114231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.115259 (+  1028us) service_pool.cc:224] Handling call
0709 13:09:27.195483 (+1080224us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400101dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:27.195714 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253319552}
I20260709 13:09:27.281597 21219 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 26.
W20260709 13:09:27.189328 21767 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:27.253851 28995 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.247988 28727 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.288431 21218 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
W20260709 13:09:27.295213 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.300297 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 1.5887e+06us
I20260709 13:09:27.305632 21143 raft_consensus.cc:2393] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 23: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:27.249738 28852 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.313249 28973 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.314424 21212 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:27.315547 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2596) took 1112 ms. Trace:
I20260709 13:09:27.316030 21212 rpcz_store.cc:276] 0709 13:09:26.203210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.271737 (+1068527us) service_pool.cc:224] Handling call
0709 13:09:27.301731 (+ 29994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.315088 (+ 13357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.315436 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:27.278319 21217 rpcz_store.cc:276] 0709 13:09:24.250209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.107854 (+857645us) service_pool.cc:224] Handling call
0709 13:09:25.118534 (+ 10680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:25.896678 (+778144us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:25.896718 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:25.897154 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:09:26.185330 (+288176us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:26.185371 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.185439 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:26.336464 (+151025us) raft_consensus.cc:1685] finished
0709 13:09:26.336500 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.187089 (+850589us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5400103450. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:27.187490 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":777677,"raft.queue_time_us":780619,"raft.run_cpu_time_us":2153,"raft.run_wall_time_us":48642,"spinlock_wait_cycles":23557376,"thread_start_us":228604,"threads_started":1,"wal-append.queue_time_us":381356}
I20260709 13:09:27.321817 21214 raft_consensus.cc:1240] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 25. Current term is 26. Ops: []
I20260709 13:09:27.322515 21217 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:27.323613 21219 raft_consensus.cc:1240] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:27.324230 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3510) took 1024 ms. Trace:
I20260709 13:09:27.324568 21219 rpcz_store.cc:276] 0709 13:09:26.299446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.322836 (+1023390us) service_pool.cc:224] Handling call
0709 13:09:27.323421 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.324007 (+   586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.324178 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:27.325664 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.326031 21219 raft_consensus.cc:1217] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Deduplicated request from leader. Original: 14.7->[17.8-19.9]   Dedup: 19.9->[]
I20260709 13:09:27.326587 21219 raft_consensus.cc:1240] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20. Ops: [17.8-19.9]
I20260709 13:09:27.323160 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3507) took 1041 ms. Trace:
I20260709 13:09:27.328150 21217 rpcz_store.cc:276] 0709 13:09:26.281351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.321882 (+1040531us) service_pool.cc:224] Handling call
0709 13:09:27.322273 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.322966 (+   693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.323114 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:27.322517 21214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2593) took 1120 ms. Trace:
I20260709 13:09:27.285277 28624 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:27.333735 21221 raft_consensus.cc:2410] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:09:27.330674 21214 rpcz_store.cc:276] 0709 13:09:26.202503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.280215 (+1077712us) service_pool.cc:224] Handling call
0709 13:09:27.321540 (+ 41325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.322273 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.322466 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:27.332562 21218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2590) took 1133 ms. Trace:
I20260709 13:09:27.341782 21216 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:27.342291 21218 rpcz_store.cc:276] 0709 13:09:26.198951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.153103 (+954152us) service_pool.cc:224] Handling call
0709 13:09:27.285211 (+132108us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:27.290981 (+  5770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.332328 (+ 41347us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540015ca50. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251b8ad 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:27.332492 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287632640}
I20260709 13:09:27.257897 28985 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:09:27.350600 21217 raft_consensus.cc:1240] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:27.336536 28727 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.352023 21798 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:09:27.350740 21036 leader_election.cc:400] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:27.353816 21036 leader_election.cc:403] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:27.356964 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:27.360425 28928 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:27.360911 28928 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:27.361346 28928 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:27.365971 28801 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:27.368338 21217 raft_consensus.cc:1275] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:09:27.371395 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.375659 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.352349 28911 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.377003 28911 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.378988 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.379751 21220 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:09:27.380072 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.380470 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2597) took 1174 ms. Trace:
I20260709 13:09:27.389256 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.388860 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.383945 21795 raft_consensus.cc:1240] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:27.380805 21797 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:09:27.380861 21220 rpcz_store.cc:276] 0709 13:09:26.205715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.287078 (+1081363us) service_pool.cc:224] Handling call
0709 13:09:27.345958 (+ 58880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.380186 (+ 34228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.380387 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:27.386883 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:27.388648 21141 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:09:27.389820 21144 raft_consensus.cc:1275] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:09:27.408578 21219 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 13:09:27.384888 21801 raft_consensus.cc:1240] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:27.390286 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:27.378921 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:27.413417 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 112051us
I20260709 13:09:27.382306 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.409164 21219 pending_rounds.cc:85] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 14
W20260709 13:09:27.390815 21187 leader_election.cc:336] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:27.408485 21140 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:27.418960 28985 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:09:27.419431 28973 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.420046 28973 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:27.421656 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.382936 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.411751 21111 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:27.387027 28845 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.385517 28962 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.436905 28973 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.383981 28911 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.440361 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.439158 21792 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:09:27.430096 21212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3504) took 1197 ms. Trace:
I20260709 13:09:27.381665 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.439934 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.440721 21215 raft_consensus.cc:1275] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:09:27.433455 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:27.439761 21221 raft_consensus.cc:1275] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:09:27.440023 21216 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:27.444356 21212 rpcz_store.cc:276] 0709 13:09:26.232920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.317308 (+1084388us) service_pool.cc:224] Handling call
0709 13:09:27.317919 (+   611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.333294 (+ 15375us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:27.353388 (+ 20094us) log.cc:844] Serialized 14 byte log entry
0709 13:09:27.429746 (+ 76358us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:27.429802 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:27.429857 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.429926 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.430038 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6604,"prepare.queue_time_us":13504,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":12038,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":11200,"thread_start_us":72897,"threads_started":1}
I20260709 13:09:27.392822 20963 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 21 to 26, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 26 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:27.439862 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.451695 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.448028 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:27.422272 28624 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.456074 21143 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:27.453114 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.461320 21138 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 22: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:27.462670 21800 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 22: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:27.462456 28845 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.463929 28852 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.464648 28852 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.464883 28962 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.465969 28845 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:27.467456 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 101161us
I20260709 13:09:27.467906 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.415244 21219 pending_rounds.cc:107] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 22.15
I20260709 13:09:27.467025 28852 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.469244 28911 raft_consensus.cc:2804] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:27.469622 21219 raft_consensus.cc:2917] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Aborting config change with OpId 22.15: Aborted: Op aborted by new leader
I20260709 13:09:27.470053 28911 raft_consensus.cc:697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:27.470569 21219 raft_consensus.cc:2889] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:27.471362 28911 consensus_queue.cc:237] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:27.479661 21796 raft_consensus.cc:2393] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 21: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:27.483671 21213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2589) took 1284 ms. Trace:
I20260709 13:09:27.483537 28890 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:27.484174 21213 rpcz_store.cc:276] 0709 13:09:26.198725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.137889 (+939164us) service_pool.cc:224] Handling call
0709 13:09:27.138281 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:27.147367 (+  9086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:27.147636 (+   269us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:27.153087 (+  5451us) log.cc:844] Serialized 26 byte log entry
0709 13:09:27.340801 (+187714us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:27.340847 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.340920 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:27.483067 (+142147us) raft_consensus.cc:1685] finished
0709 13:09:27.483090 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.483258 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5590,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":202,"thread_start_us":187234,"threads_started":1,"wal-append.queue_time_us":291074}
I20260709 13:09:27.486730 29022 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:09:27.391031 28727 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:27.491972 28727 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:27.493294 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71910.1us
I20260709 13:09:27.495954 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:27.497135 28852 raft_consensus.cc:2804] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:27.497615 28852 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:27.497736 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:27.498092 28852 raft_consensus.cc:3060] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:27.499691 28973 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:27.499406 28624 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.459954 28861 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.500744 28861 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.505654 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:27.505899 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.506366 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:27.512763 21797 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:27.505205 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:27.515560 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:27.504247 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.518960 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.519745 21798 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 27.
W20260709 13:09:27.523501 21767 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:27.525799 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.527765 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.528594 21801 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:27.503589 28847 raft_consensus.cc:2804] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:27.503144 28624 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.526950 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.512212 28973 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
W20260709 13:09:27.551774 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.510049 28861 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.553192 21794 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 25: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:27.550498 21799 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:27.513423 28962 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:27.558637 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.559581 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:27.562700 21138 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:09:27.547593 28847 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:27.575466 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.589636 28853 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.589514 28688 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.595798 28688 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.598253 28688 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.589778 28909 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:27.599839 28909 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.600093 28861 consensus_queue.cc:1059] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:27.601433 28861 raft_consensus.cc:3055] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:27.601846 28861 raft_consensus.cc:740] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:27.602221 28886 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:27.602766 28853 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.605311 28853 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.590978 28847 raft_consensus.cc:3060] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:27.592096 21796 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:27.589641 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.615783 20963 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 20 to 21, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 21 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:27.602952 28861 consensus_queue.cc:260] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.632916 28861 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:27.634414 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 57445.4us
I20260709 13:09:27.635913 21143 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:27.635514 21765 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:27.667002 28847 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:27.666373 21797 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:27.556456 28700 consensus_queue.cc:1059] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
W20260709 13:09:27.679953 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.681967 29024 log.cc:927] Time spent T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:09:27.683604 28973 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.685518 28985 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:27.687911 28700 raft_consensus.cc:3055] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:27.688457 28700 raft_consensus.cc:740] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:27.689677 28700 consensus_queue.cc:260] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:27.669480 28847 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:27.731760 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.781440 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.767549 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.714534 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 79155.3us
W20260709 13:09:27.695951 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 227271us
W20260709 13:09:27.780222 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 426028us
W20260709 13:09:27.790550 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.678182 28757 consensus_meta.cc:287] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.007s
W20260709 13:09:27.713203 20964 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 227) took 47305 ms (client timeout 15000 ms). Trace:
I20260709 13:09:27.795038 21799 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 13:09:27.646935 28852 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.708925 20963 catalog_manager.cc:5697] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 10 to 11, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 11 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:27.818639 21799 pending_rounds.cc:85] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 14
I20260709 13:09:27.825309 21799 pending_rounds.cc:107] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 22.15
I20260709 13:09:27.825630 21799 raft_consensus.cc:2917] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Aborting config change with OpId 22.15: Aborted: Op aborted by new leader
W20260709 13:09:27.822894 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 118405us
W20260709 13:09:27.797621 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.822705 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:27.819428 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.714763 28624 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:27.703225 28889 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.784569 28973 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:27.835470 21799 raft_consensus.cc:2889] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:27.839391 28886 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:27.840478 28985 consensus_queue.cc:1059] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:27.842815 28688 consensus_queue.cc:1059] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:27.843581 28890 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:27.844262 28902 consensus_queue.cc:1048] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:27.844277 28847 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:27.845304 28889 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.848440 28845 consensus_queue.cc:1048] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:27.842921 21035 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:27.794418 28700 raft_consensus.cc:3060] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:27.821473 20963 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 16 to 22, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 22 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:27.798067 28757 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:27.888528 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:27.888490 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.784940 28909 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:27.852458 21187 leader_election.cc:336] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:27.799026 29022 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:27.859615 21113 leader_election.cc:336] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:27.896932 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:27.799021 28727 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:27.865204 29029 raft_consensus.cc:3055] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:09:27.868578 28853 raft_consensus.cc:3055] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:27.870687 28889 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:27.872812 28961 consensus_queue.cc:1048] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:27.859616 28624 raft_consensus.cc:697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:27.876472 21138 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:09:27.877208 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.878109 21144 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:27.879710 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.881109 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:27.884687 28928 consensus_meta.cc:287] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.002s
I20260709 13:09:27.882089 21141 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:27.888813 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.900453 21187 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:27.905542 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.908603 21140 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:27.907927 21138 pending_rounds.cc:85] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 14
I20260709 13:09:27.917382 21146 raft_consensus.cc:2410] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:09:27.917608 21138 pending_rounds.cc:107] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:09:27.921916 21138 raft_consensus.cc:2889] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:27.918807 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.919032 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.924366 21216 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:27.918324 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.925877 21219 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:27.918579 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.918082 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.904211 28853 raft_consensus.cc:740] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:27.916831 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.928747 28985 raft_consensus.cc:2804] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:27.929062 21144 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 13.
I20260709 13:09:27.929276 28985 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:27.930219 21214 raft_consensus.cc:2410] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:09:27.919305 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:27.933686 21142 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:27.898074 28727 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:27.899888 29029 raft_consensus.cc:740] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:27.906881 28847 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:27.916450 28902 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 13:09:27.939431 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.940362 21218 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 25: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:27.945204 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:27.947670 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 104800us
I20260709 13:09:27.932001 28886 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:27.933451 28985 raft_consensus.cc:3060] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:27.950217 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:27.915931 29022 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.953243 21144 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:09:27.955541 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 99118.6us
W20260709 13:09:27.957971 21113 leader_election.cc:336] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:27.956717 21141 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 25.
I20260709 13:09:27.959945 21143 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:09:27.930963 28928 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:27.967073 28727 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:27.968786 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:27.969226 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 135027us
I20260709 13:09:27.970419 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:27.973083 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.974438 21767 leader_election.cc:336] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:27.971859 21140 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 27.
I20260709 13:09:27.971392 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:27.917848 28624 consensus_queue.cc:237] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:27.983022 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 152762us
I20260709 13:09:27.929497 28853 consensus_queue.cc:260] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:27.997047 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:27.981320 21799 raft_consensus.cc:1240] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 25. Current term is 26. Ops: []
I20260709 13:09:27.988956 29027 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:09:27.999436 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 139679us
W20260709 13:09:27.989241 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.995719 28847 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:27.987696 21035 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:27.985409 21139 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 26.
I20260709 13:09:27.996732 21110 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:27.999801 20963 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 20, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 20 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:28.018550 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:28.004083 28847 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:28.023463 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.999570 29027 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.006479 28852 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:28.025561 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 67618.9us
W20260709 13:09:28.027055 21767 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:28.028080 21767 leader_election.cc:336] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.011865 28850 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:09:27.796125 20964 rpcz_store.cc:269] 0709 13:08:40.407806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.433611 (+ 25805us) service_pool.cc:224] Handling call
0709 13:09:27.712870 (+47279259us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:17.822495 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:17.931586 (+109091us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:17.931639 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:09:17.931659 (+    20us) tablet.cc:662] Decoding operations
0709 13:09:18.284368 (+352709us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:18.460484 (+176116us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:18.466972 (+  6488us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:09:18.834951 (+367979us) tablet.cc:701] Row locks acquired
0709 13:09:18.835011 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:09:18.835233 (+   222us) write_op.cc:270] Start()
0709 13:09:18.835443 (+   210us) write_op.cc:276] Timestamp: P: 1783602558835153 usec, L: 0
0709 13:09:18.835474 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:09:18.836475 (+  1001us) log.cc:844] Serialized 9910 byte log entry
0709 13:09:19.500904 (+664429us) op_driver.cc:464] REPLICATION: finished
0709 13:09:19.733990 (+233086us) write_op.cc:301] APPLY: starting
0709 13:09:19.734093 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:20.926882 (+1192789us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:20.951429 (+ 24547us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:22.872107 (+1920678us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:23.094846 (+222739us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:09:23.124131 (+ 29285us) write_op.cc:312] APPLY: finished
0709 13:09:23.363693 (+239562us) log.cc:844] Serialized 220 byte log entry
0709 13:09:23.572285 (+208592us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:23.609365 (+ 37080us) write_op.cc:454] Released schema lock
0709 13:09:23.652802 (+ 43437us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:23.716350 (+ 63548us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:23.724965 (+  8615us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1984589,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":179142,"apply.run_cpu_time_us":127038,"apply.run_wall_time_us":3980378,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"num_ops":25,"prepare.queue_time_us":52806,"prepare.run_cpu_time_us":30641,"prepare.run_wall_time_us":1406833,"raft.queue_time_us":246386,"raft.run_cpu_time_us":53871,"raft.run_wall_time_us":115654,"replication_time_us":665238,"thread_start_us":806454,"threads_started":5,"wal-append.queue_time_us":772089,"wal-append.run_cpu_time_us":132285,"wal-append.run_wall_time_us":2612367}]]}
W20260709 13:09:28.027160 21113 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.018450 28845 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:28.012156 28688 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:28.025722 20963 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 18 to 24, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 24 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:28.040501 21187 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.043649 21765 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.039844 21111 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.026477 29027 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:28.024999 28847 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:28.051513 21187 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.044430 28962 raft_consensus.cc:2764] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:09:28.021145 29029 consensus_queue.cc:260] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:28.063357 28852 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:28.063426 21765 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.064177 28852 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.065258 28962 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:28.065965 28962 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:28.066394 28852 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:28.067835 28881 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEARNER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:28.067193 28962 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.070685 28853 raft_consensus.cc:3060] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:28.067239 20963 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 16 to 22, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 22 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:28.074018 20963 catalog_manager.cc:5697] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.076231 29029 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:28.080636 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.079963 21765 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.082103 28881 raft_consensus.cc:2804] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:28.082674 28881 raft_consensus.cc:697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:28.083699 28881 consensus_queue.cc:237] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:28.088379 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.038748 28727 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:28.051178 21113 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:28.101781 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 51696.2us
I20260709 13:09:28.108649 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.041406 28688 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.116014 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:28.121153 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.076231 20963 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 10 to 14, term changed from 19 to 23, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to <none>, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 23 committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:28.121157 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.123659 21799 raft_consensus.cc:1275] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:28.121901 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.128352 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:28.128539 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 61240.5us
I20260709 13:09:28.112450 28688 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:28.122649 21800 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 27.
W20260709 13:09:28.135334 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 113491us
I20260709 13:09:28.122371 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.120676 28845 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:09:28.159440 21112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:28.168308 21766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.168605 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:28.149358 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.175838 21035 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:28.112412 29027 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.126072 28847 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.178243 28757 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:09:28.193406 28852 raft_consensus.cc:2804] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:28.193929 28852 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.194315 28852 raft_consensus.cc:3060] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:28.194509 28757 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:28.194995 28757 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:28.198347 28757 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:28.200759 29027 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:28.201550 29027 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:28.169471 20963 catalog_manager.cc:5697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 23 to 24, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 24 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.143576 28801 consensus_queue.cc:1048] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.041s
W20260709 13:09:28.216079 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 183948us
I20260709 13:09:28.146629 28889 consensus_queue.cc:1048] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
W20260709 13:09:28.219868 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.189929 28928 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:28.225847 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:28.232414 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 211152us
W20260709 13:09:28.233821 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 103792us
W20260709 13:09:28.238943 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 211792us
I20260709 13:09:28.238235 20963 catalog_manager.cc:5697] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:28.240207 21113 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:28.256381 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:28.255848 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:28.261924 21113 leader_election.cc:336] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.260794 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:28.271418 21113 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.272573 28889 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:28.273093 28889 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.273505 28889 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:28.274720 28757 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.275645 29052 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:28.277571 28757 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:28.273109 21113 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:28.284811 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:28.285524 21113 leader_election.cc:336] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.286051 21113 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.287057 29052 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:28.288708 21113 leader_election.cc:336] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.289261 21113 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.290997 29052 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:28.247838 28861 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.291812 28861 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:28.292695 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:28.294706 28861 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:28.293467 28889 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:28.298241 28889 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:28.295126 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:28.299841 21113 leader_election.cc:336] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.300387 21113 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.301674 29036 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:28.301853 21113 leader_election.cc:336] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:28.305001 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.224852 28886 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:09:28.259647 28994 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:09:28.302261 29041 log.cc:927] Time spent T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:09:28.316535 21187 leader_election.cc:336] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.316217 28801 consensus_queue.cc:1059] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 13:09:28.324470 28852 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:28.324934 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:28.236804 28985 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:28.328943 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:28.329197 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 198856us
I20260709 13:09:28.246903 28909 consensus_queue.cc:1048] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:09:28.302284 29036 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.302352 28911 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:28.338857 28850 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:28.341316 28624 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:28.341745 28624 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.342096 28624 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:28.342202 28850 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.345671 28852 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:28.346588 28847 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:28.347635 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:28.347874 29037 log.cc:927] Time spent T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:09:28.339480 20963 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 11 to 17, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 17 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:28.334088 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:28.353825 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 81743.8us
W20260709 13:09:28.359427 21184 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:28.353550 21767 leader_election.cc:336] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:28.364763 21113 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:28.365841 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:28.339406 29036 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:28.370476 28847 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:28.368412 21113 leader_election.cc:336] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.376567 21219 raft_consensus.cc:1240] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 13:09:28.373266 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:28.387831 21214 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:28.389724 21185 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.390903 28886 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:28.391577 21185 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.393297 28994 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:28.383713 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:28.403036 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49514.2us
W20260709 13:09:28.374004 28973 consensus_meta.cc:287] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.514s	user 0.012s	sys 0.001s
W20260709 13:09:28.409754 21186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:28.404443 21767 leader_election.cc:336] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.355934 28801 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 13:09:28.412638 21767 leader_election.cc:336] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.424576 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.448292 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:28.418485 21767 leader_election.cc:336] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:28.454173 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 190887us
W20260709 13:09:28.449870 21767 leader_election.cc:336] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:28.457185 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 63730.9us
I20260709 13:09:28.455175 20963 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 18 to 22, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 22 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
W20260709 13:09:28.457121 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 107154us
I20260709 13:09:28.459673 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.461359 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:28.461457 21767 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:28.465938 28962 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:09:28.466395 28962 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.466744 28962 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:28.469820 28624 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:28.476357 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.474457 20963 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:28.486891 28985 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:28.487406 21187 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.487888 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:28.491575 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 74131us
I20260709 13:09:28.488660 29057 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:28.497491 29049 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.498085 29049 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.500367 29049 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:28.501145 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:28.502051 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:28.503580 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 135351us
W20260709 13:09:28.504379 21113 leader_election.cc:336] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.504428 21221 raft_consensus.cc:1275] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
W20260709 13:09:28.506361 21113 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.506810 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.492117 28890 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:28.463423 28847 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:28.510149 28847 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.510565 28847 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:28.510841 29057 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:28.514108 29022 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:28.518435 28881 raft_consensus.cc:3055] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:28.519068 28881 raft_consensus.cc:740] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:28.520325 28881 consensus_queue.cc:260] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.521576 28881 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:28.523341 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:28.524477 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.510667 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:28.527993 21797 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:28.528034 20963 catalog_manager.cc:5697] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 23, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 23 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.526991 21212 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
I20260709 13:09:28.529726 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:28.559552 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.555173 28890 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:28.563581 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 69713us
I20260709 13:09:28.564121 21146 raft_consensus.cc:1240] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:28.564000 28890 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.564720 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:28.565433 21187 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.565900 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.565950 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:28.566336 28890 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:28.567163 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60829.7us
W20260709 13:09:28.567946 21113 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:28.568516 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:28.568615 21113 leader_election.cc:336] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.568986 28624 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:28.569190 21113 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:28.567204 21187 leader_election.cc:336] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.570552 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.567998 21145 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:09:28.579638 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.580039 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:28.579566 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:28.582406 21767 leader_election.cc:336] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:28.585471 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:28.584267 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:28.590399 21146 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:09:28.598495 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:28.599198 21143 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:28.602983 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.607561 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:28.584685 28909 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:09:28.608860 21113 leader_election.cc:336] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.613648 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.614336 21113 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.585001 29052 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:28.602805 28890 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:09:28.621140 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.576054 28928 consensus_queue.cc:1048] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:09:28.580463 28889 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:09:28.627846 21801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.610909 28985 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:28.630636 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:28.632606 21801 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 26: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:28.613850 28909 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.605532 28994 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:28.638803 29064 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:09:28.639473 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 89082.3us
I20260709 13:09:28.641342 29032 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:28.632263 28911 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:28.630935 29049 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:28.609622 28688 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.660830 21214 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
I20260709 13:09:28.660527 21143 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
I20260709 13:09:28.653457 28909 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:28.695001 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:28.690096 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 84701.1us
W20260709 13:09:28.696311 21113 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.665752 29064 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.699574 29064 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:28.707742 28994 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:28.711609 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 144447us
W20260709 13:09:28.713016 21187 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:28.715513 21187 leader_election.cc:336] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.697578 21794 raft_consensus.cc:2393] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 20: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:28.726261 28889 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:28.661640 29057 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:28.688935 28845 consensus_meta.cc:287] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.002s
I20260709 13:09:28.723955 21797 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
I20260709 13:09:28.671721 28985 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:28.725353 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:28.727108 21139 raft_consensus.cc:2393] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 20: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:28.732872 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:28.697888 28911 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:28.740814 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:28.718742 28886 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:28.749850 21798 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:28.764755 28845 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:28.743110 29036 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:28.805027 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 109888us
I20260709 13:09:28.770519 28853 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:28.819078 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.819353 21793 raft_consensus.cc:2393] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 23: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
W20260709 13:09:28.820961 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:28.822482 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 107054us
W20260709 13:09:28.823661 29050 log.cc:927] Time spent T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:28.810168 29065 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:28.846300 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:28.854014 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:28.864553 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:28.827729 29058 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.829320 29078 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.829425 29069 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:09:28.880815 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:28.884640 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 65654.3us
I20260709 13:09:28.884815 21142 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:28.801941 28985 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:28.847746 28928 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.848562 29065 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.892902 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:28.892804 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:28.895690 21144 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 29: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:28.878760 28688 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:28.878845 29081 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.900732 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:28.902442 29043 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:28.895153 29065 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:28.908469 29036 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:28.899031 28928 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.913919 21142 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:28.914829 21143 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 25: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:28.916944 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:28.919540 21110 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.921365 21035 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:28.921377 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:28.924687 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 94495.6us
I20260709 13:09:28.925246 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:28.832804 28911 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:28.929811 21185 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:28.932747 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:28.938338 29069 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:28.935434 29081 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:28.938858 29069 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.939400 29069 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:28.940845 29081 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:28.941735 29054 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:28.942324 29054 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.944533 29054 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:28.889540 29078 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:28.946039 28909 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.946714 29078 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:28.887404 29058 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:28.948326 29069 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:28.949535 29058 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:28.951390 29069 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:28.937870 28994 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:28.955655 28847 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.956583 28994 raft_consensus.cc:697] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:28.957619 28994 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:28.936825 28727 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:28.914422 28962 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:28.960925 21800 raft_consensus.cc:2368] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 27. Current term is 28.
I20260709 13:09:28.962011 28962 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:28.927048 28853 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:09:28.970148 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 50676.4us
I20260709 13:09:28.971256 21110 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.972060 21141 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:09:28.972992 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:28.974858 28727 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:28.977887 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.978760 28727 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:09:28.982863 29071 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:09:28.983104 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:28.984027 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 54524.1us
I20260709 13:09:28.984916 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:28.989820 21140 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 26: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:09:28.972002 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:28.992009 21187 leader_election.cc:336] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:28.992450 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:28.994848 21793 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:09:28.996249 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:29.006183 21145 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:09:28.974152 28853 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:29.011909 28853 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:29.015144 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:29.015899 21142 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:29.016163 21143 raft_consensus.cc:1275] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:09:29.018034 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.017901 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:29.019022 21145 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:28.969044 28847 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:29.029217 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:29.022946 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 49066.6us
W20260709 13:09:29.027638 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:29.021952 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.022030 21111 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:29.031088 21140 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:29.037655 21144 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:28.973696 29043 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:29.041507 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49523.8us
I20260709 13:09:29.041961 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:28.995680 29053 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:29.044018 29043 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:29.046675 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:29.052933 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 158153us
W20260709 13:09:29.085693 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:29.096688 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 61243.6us
W20260709 13:09:29.092545 21113 consensus_peers.cc:597] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:29.101164 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47114.2us
W20260709 13:09:29.102473 21113 leader_election.cc:336] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.103037 21113 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:29.113147 29029 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:29.113549 29099 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:09:29.115340 21187 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.115839 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:29.041148 28727 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:29.117238 28727 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:29.118229 28727 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:29.118413 29092 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:29.046028 29053 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:29.119457 28624 consensus_queue.cc:1048] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:29.119938 29053 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:29.095700 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3724541309999267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 13:09:29.123255 21036 connection.cc:256] RPC call timeout handler was delayed by -1.3571541310000157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
W20260709 13:09:29.123576 21036 connection.cc:256] RPC call timeout handler was delayed by -1.2339678040000308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 13:09:29.123845 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0868174379999118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
W20260709 13:09:29.124116 21036 connection.cc:256] RPC call timeout handler was delayed by -1.0160541310000326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
I20260709 13:09:29.092859 29043 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:29.092983 28985 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.036243 28852 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.089s
I20260709 13:09:29.153841 21142 raft_consensus.cc:2468] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:29.183869 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:29.222745 21185 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:29.249248 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:29.244284 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:29.249341 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 134063us
I20260709 13:09:29.141310 28850 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.085s
W20260709 13:09:29.257126 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 73328.9us
I20260709 13:09:29.137378 29064 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.293149 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:29.294068 21214 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:29.259155 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:29.204169 28909 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:29.253459 29058 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:29.299559 29058 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:29.299984 29058 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:29.302083 29032 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:29.302781 29032 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:29.308279 21139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49724 (request call id 3682) took 1087 ms. Trace:
I20260709 13:09:29.222059 28928 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:29.288295 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:29.271937 29074 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:29.308450 21147 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:09:29.172914 28985 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:29.296675 29103 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:09:29.308917 21139 rpcz_store.cc:276] 0709 13:09:28.221428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:28.221581 (+   153us) service_pool.cc:224] Handling call
0709 13:09:29.308036 (+1086455us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b540036db54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:29.308217 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140490624}
I20260709 13:09:29.252635 29069 raft_consensus.cc:2804] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:29.217070 29080 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:29.303089 21145 raft_consensus.cc:2468] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 28.
W20260709 13:09:29.326684 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 202700us
I20260709 13:09:29.141582 29097 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.131s
I20260709 13:09:29.338742 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:29.325590 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:29.348209 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 235400us
I20260709 13:09:29.348245 21220 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:29.348970 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:29.346798 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:29.349023 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:29.350313 21217 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
W20260709 13:09:29.344345 29104 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:09:29.359019 21113 consensus_peers.cc:597] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:29.356734 29069 raft_consensus.cc:697] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:29.326018 29065 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.364976 29069 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.365918 28985 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:29.384183 29065 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:29.386032 29064 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:29.386673 28852 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:29.387100 28852 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:29.387506 28852 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:29.390100 21185 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:29.355455 21187 leader_election.cc:336] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.391273 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:29.393200 21140 raft_consensus.cc:2468] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:29.323242 29058 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:29.363802 21113 consensus_peers.cc:597] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:29.412416 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:29.415326 21113 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.422348 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:29.423045 21140 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:29.429473 21113 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.425678 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:29.432330 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 107106us
I20260709 13:09:29.431988 21214 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:29.423231 29074 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:29.426801 29081 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:29.459401 21185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.453109 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:29.462235 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:09:29.466456 21113 leader_election.cc:336] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.466876 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:29.467764 21139 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:09:29.473847 21036 leader_election.cc:336] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.507s (SENT)
W20260709 13:09:29.481859 21113 leader_election.cc:336] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.482421 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:29.450675 29064 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:29.490415 21140 raft_consensus.cc:2468] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:29.465909 28845 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:29.487677 21036 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:29.495452 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 167333us
I20260709 13:09:29.479039 28902 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:29.501173 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:29.481858 28727 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:29.503952 29032 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:29.521986 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 48178us
I20260709 13:09:29.525955 28853 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:09:29.529096 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:29.536247 29058 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:29.537178 21187 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.537783 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:29.540560 29114 log.cc:927] Time spent T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:09:29.541029 29107 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:29.465382 29081 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.495258 28845 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:29.543093 28852 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:29.542937 29043 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.543689 28845 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:29.544940 29081 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:29.496407 29078 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:29.545620 29043 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:29.463827 28985 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
W20260709 13:09:29.539140 21187 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.552685 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:29.566159 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.546967 28727 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:29.546077 28852 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:29.571916 29032 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:29.575415 29107 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:29.576272 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:29.577968 28727 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:29.588093 28852 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.601641 28852 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:29.584707 29065 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.617161 28994 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.617972 28994 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.620507 29053 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:29.622243 21144 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:09:29.604173 29065 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:29.624442 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 101661us
W20260709 13:09:29.625201 21036 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.544s (SENT)
W20260709 13:09:29.625201 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 86092.5us
W20260709 13:09:29.634312 21187 leader_election.cc:336] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.634984 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:29.644204 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 162366us
W20260709 13:09:29.645022 21113 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.645368 21113 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:29.646139 29078 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 13:09:29.646904 29065 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:29.647356 21113 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.647439 28985 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.649120 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:29.649845 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:29.637605 29074 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:29.650921 21146 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
W20260709 13:09:29.650663 21036 leader_election.cc:336] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.555s (SENT)
W20260709 13:09:29.674971 21113 leader_election.cc:336] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:29.674505 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 197401us
I20260709 13:09:29.658823 29078 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:29.677803 21113 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:29.683869 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.687078 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:29.681237 29088 log.cc:927] Time spent T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.701s	user 0.004s	sys 0.001s
I20260709 13:09:29.690876 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:29.691056 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
W20260709 13:09:29.692664 21187 leader_election.cc:336] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.690814 21111 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:29.694376 21187 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:29.678000 28852 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:29.695372 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 44754.3us
W20260709 13:09:29.711100 21036 leader_election.cc:336] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.566s (SENT)
I20260709 13:09:29.711617 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:29.661439 28985 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:29.686498 29032 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.717557 29032 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:29.718353 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3686) took 1146 ms. Trace:
I20260709 13:09:29.718743 21138 rpcz_store.cc:276] 0709 13:09:28.571690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:28.576434 (+  4744us) service_pool.cc:224] Handling call
0709 13:09:28.599145 (+ 22711us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:28.734258 (+135113us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:28.734294 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:28.734797 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:09:28.859822 (+125025us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:28.859881 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:28.859985 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:29.628192 (+768207us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:29.718045 (+ 89853us) raft_consensus.cc:1685] finished
0709 13:09:29.718069 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:29.718296 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12245,"raft.queue_time_us":44244,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":7360,"spinlock_wait_cycles":7684864,"thread_start_us":220411,"threads_started":2,"wal-append.queue_time_us":244733}
I20260709 13:09:29.720086 29032 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:29.711393 29065 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:29.711542 29027 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:29.709761 21184 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:29.724964 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:29.725420 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.727861 21035 leader_election.cc:400] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 27. Current term is 28.
I20260709 13:09:29.728204 21035 leader_election.cc:403] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 13:09:29.729816 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:29.732400 21219 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:29.730636 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:29.730247 21799 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:29.730086 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:29.691030 29126 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:29.735122 21792 raft_consensus.cc:2368] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23.
I20260709 13:09:29.735932 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:29.736652 29043 raft_consensus.cc:3060] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:29.733913 28961 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.729027 29074 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:29.737409 28961 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:29.738310 21036 leader_election.cc:336] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.641s (SENT)
I20260709 13:09:29.738768 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:29.749111 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:29.750372 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:29.750420 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:29.750362 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:29.750607 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
W20260709 13:09:29.754405 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64920.9us
I20260709 13:09:29.754781 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:29.767067 21141 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:09:29.767722 28985 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:29.773062 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:29.774197 29118 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:09:29.753238 28688 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.755255 21139 raft_consensus.cc:2368] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 27. Current term is 28.
W20260709 13:09:29.784518 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46234.9us
I20260709 13:09:29.780968 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:29.771699 28853 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:29.786626 21035 leader_election.cc:400] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23.
I20260709 13:09:29.777145 28700 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:29.787693 21035 leader_election.cc:403] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
W20260709 13:09:29.785887 21187 leader_election.cc:336] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:29.795742 29115 log.cc:927] Time spent T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.260s	user 0.003s	sys 0.003s
W20260709 13:09:29.796061 29098 log.cc:927] Time spent T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.815s	user 0.005s	sys 0.000s
I20260709 13:09:29.786917 28688 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.788496 28961 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:29.798244 21215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56816 (request call id 2624) took 1437 ms. Trace:
I20260709 13:09:29.788229 28847 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:09:29.784888 29053 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:29.799597 21215 rpcz_store.cc:276] 0709 13:09:28.360570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:28.360724 (+   154us) service_pool.cc:224] Handling call
0709 13:09:28.395069 (+ 34345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:28.799331 (+404262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:28.799630 (+   299us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:28.808880 (+  9250us) log.cc:844] Serialized 26 byte log entry
0709 13:09:28.900305 (+ 91425us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:28.900350 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:28.900422 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:29.795479 (+895057us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:29.797928 (+  2449us) raft_consensus.cc:1685] finished
0709 13:09:29.797954 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:29.798185 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72356,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"spinlock_wait_cycles":37927808,"thread_start_us":358318,"threads_started":2,"wal-append.queue_time_us":146354}
I20260709 13:09:29.812657 28850 raft_consensus.cc:3060] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:29.846249 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:29.847077 21142 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:29.849313 21139 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 24. Current term is 25. Ops: []
W20260709 13:09:29.849625 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 69462.2us
W20260709 13:09:29.850380 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:29.850944 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:29.852662 21767 leader_election.cc:336] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:29.854362 21767 consensus_peers.cc:597] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:29.855754 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:29.859694 21765 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:29.857918 21767 leader_election.cc:336] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.849502 29027 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:29.863026 21767 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:29.866595 21187 leader_election.cc:336] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.880617 29107 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:09:29.875663 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:29.882512 21145 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:09:29.883091 21185 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:29.887094 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:29.901070 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 130399us
I20260709 13:09:29.885586 28688 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:29.897281 29043 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 27. Current term is 28.
I20260709 13:09:29.896463 29069 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:29.898938 29057 raft_consensus.cc:2804] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:29.886624 29074 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 13:09:29.900403 29107 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:29.908902 28801 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:29.918165 28845 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.923681 28801 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:29.917623 29107 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:29.943387 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 89575.4us
W20260709 13:09:29.944703 21187 leader_election.cc:336] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.945168 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:29.946529 21187 leader_election.cc:336] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:29.931464 28847 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
W20260709 13:09:29.947906 28847 raft_consensus.cc:1068] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:29.920971 29057 raft_consensus.cc:697] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:29.950637 29057 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.955858 29074 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:29.946941 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:29.938874 21036 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 2.103s (SENT) [suppressed 167 similar messages]
I20260709 13:09:29.975963 29074 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:29.941738 29107 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:29.987138 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 91168.3us
W20260709 13:09:29.989269 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 50439.5us
I20260709 13:09:29.934437 28845 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:29.990949 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44468.4us
I20260709 13:09:29.993918 29134 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:09:29.996992 21187 leader_election.cc:336] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:30.002444 28727 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:29.924417 29027 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:29.941249 28801 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:30.015497 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 155846us
I20260709 13:09:30.015853 28801 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:30.016295 21765 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:30.017254 29092 raft_consensus.cc:2804] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:09:30.017884 29092 raft_consensus.cc:697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 28 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:30.020092 28973 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:30.020911 28973 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.021394 28973 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.022951 21144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3688) took 1032 ms. Trace:
I20260709 13:09:30.018936 29092 consensus_queue.cc:237] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.023389 21144 rpcz_store.cc:276] 0709 13:09:28.990136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.040709 (+ 50573us) service_pool.cc:224] Handling call
0709 13:09:29.041025 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:29.622725 (+581700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:30.022684 (+399959us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b5400296dd4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:30.022887 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1335943424}
I20260709 13:09:30.018952 29062 raft_consensus.cc:2804] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:30.025169 29062 raft_consensus.cc:697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:30.026175 28850 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23.
W20260709 13:09:30.026614 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 245651us
I20260709 13:09:30.026202 29062 consensus_queue.cc:237] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.033299 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:30.033569 21800 raft_consensus.cc:1240] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:30.035804 21798 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEARNER]: Advancing to term 25
I20260709 13:09:30.035678 28973 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:30.041889 28994 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:30.044641 28727 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:30.050293 21798 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 15. (term mismatch)
I20260709 13:09:30.050716 21798 pending_rounds.cc:85] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 14
I20260709 13:09:30.050957 21798 pending_rounds.cc:107] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 13:09:30.051198 21798 raft_consensus.cc:2889] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:30.051653 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:30.052234 21214 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:30.054016 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:30.056185 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:30.060458 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.068848 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:30.077807 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:30.080523 21217 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
I20260709 13:09:30.090718 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:30.078230 21113 leader_election.cc:336] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:30.093541 21113 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:30.095434 21037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.099929 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:30.100782 21221 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
I20260709 13:09:30.077451 29043 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:30.075258 28727 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:30.077157 28847 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:30.123641 21214 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
W20260709 13:09:30.112226 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 94011.7us
I20260709 13:09:30.114152 21220 raft_consensus.cc:1240] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 26. Current term is 27. Ops: []
W20260709 13:09:30.108323 21113 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:30.124979 21113 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:30.126222 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:30.126827 21214 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:30.087944 28850 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
W20260709 13:09:30.131039 21113 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:30.131367 29134 consensus_queue.cc:1059] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:30.110543 29078 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:30.142889 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:09:30.144037 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 286114us
I20260709 13:09:30.116379 28727 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:30.149885 21214 raft_consensus.cc:2468] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 27.
W20260709 13:09:30.153327 21767 leader_election.cc:336] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:30.107021 29057 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:09:30.162235 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:30.138149 28845 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:30.175755 21113 consensus_peers.cc:597] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.176753 21113 leader_election.cc:336] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:30.184053 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51803.1us
I20260709 13:09:30.183401 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:30.188331 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:30.164619 28727 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:30.163349 29130 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:09:30.229420 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52717.3us
I20260709 13:09:30.177831 28853 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.179204 29097 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:30.188602 28973 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:30.164583 29064 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.067s
W20260709 13:09:30.252308 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 61479.5us
I20260709 13:09:30.253605 21111 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:30.254210 29057 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:30.254879 29057 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:30.256314 28845 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:30.257074 28845 raft_consensus.cc:697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:30.259531 29061 consensus_queue.cc:1059] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:09:30.259342 29057 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:30.262522 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:30.252313 29081 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:30.254985 21111 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:30.258373 28845 consensus_queue.cc:237] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:30.232092 28961 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:30.185099 29022 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:09:30.229059 29130 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:30.287535 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:30.286471 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:30.291987 21140 raft_consensus.cc:2376] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20. Re-sending same reply.
I20260709 13:09:30.300976 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:30.270805 28973 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:30.301950 21144 raft_consensus.cc:2368] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 26. Current term is 27.
I20260709 13:09:30.280095 28961 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:30.313246 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:30.336340 21147 raft_consensus.cc:2368] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23.
I20260709 13:09:30.323096 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:30.332262 21221 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 13:09:30.339558 29136 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:09:30.291903 29130 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:30.345664 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:30.252525 28853 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:30.311625 28973 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:30.370355 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:30.310642 29146 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.333750 29129 raft_consensus.cc:2804] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:09:30.341327 28961 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:30.375263 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:30.279259 29097 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.381316 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:30.379196 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 194918us
I20260709 13:09:30.360555 28889 raft_consensus.cc:493] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:30.385474 29129 raft_consensus.cc:697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:30.427678 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:30.382874 28852 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:30.469339 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.411792 28889 raft_consensus.cc:515] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:30.420230 29137 log.cc:927] Time spent T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.193s	user 0.000s	sys 0.002s
I20260709 13:09:30.378539 29107 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.385102 29058 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:30.379446 29146 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:30.473841 29069 raft_consensus.cc:3055] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:09:30.459939 29022 raft_consensus.cc:670] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
W20260709 13:09:30.508481 19967 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 21097)
I20260709 13:09:30.512247 29097 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:30.517323 29107 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:30.489943 29069 raft_consensus.cc:740] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:30.519688 29069 consensus_queue.cc:260] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.521692 29069 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:30.492339 29129 consensus_queue.cc:237] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:30.490733 28852 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:30.560642 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 179420us
I20260709 13:09:30.565382 21036 leader_election.cc:400] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 27. Current term is 28.
I20260709 13:09:30.503691 29058 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:30.568517 21036 leader_election.cc:336] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.501s (SENT)
I20260709 13:09:30.568961 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:30.508517 28889 leader_election.cc:290] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:30.570079 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.604264 28961 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.605794 21147 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 23: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:09:30.647923 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 46287.5us
I20260709 13:09:30.372699 29144 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:09:30.636878 28727 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:30.658470 21145 raft_consensus.cc:2393] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 in current term 24: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:30.658746 28928 raft_consensus.cc:3055] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:09:30.659186 28928 raft_consensus.cc:740] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:30.659982 28928 consensus_queue.cc:260] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:30.661175 28928 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:30.661674 21141 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:30.570858 29058 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:30.666246 29141 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.516s	user 0.001s	sys 0.001s
I20260709 13:09:30.670696 29167 raft_consensus.cc:3060] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:30.572250 28853 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:30.675081 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:30.675801 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:30.701794 28973 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.703898 28961 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:30.704533 29146 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:30.705482 29150 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:09:30.706082 29151 consensus_queue.cc:1059] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:30.707988 21794 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 13:09:30.709805 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 406854us
I20260709 13:09:30.710908 29058 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:30.720167 21798 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
W20260709 13:09:30.732530 21113 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:30.720404 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 151957us
I20260709 13:09:30.720671 21793 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:30.711347 29154 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:30.747764 21143 raft_consensus.cc:1275] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
W20260709 13:09:30.747241 21113 leader_election.cc:336] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:30.757710 21113 leader_election.cc:336] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:30.758317 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:30.759804 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 83350.1us
I20260709 13:09:30.761273 21146 raft_consensus.cc:1275] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 13:09:30.769726 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:30.769834 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:30.770414 21792 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:30.763743 29029 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:30.771793 29146 raft_consensus.cc:3055] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:30.764487 29154 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:30.787674 28727 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:30.809502 28902 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:30.810770 29146 raft_consensus.cc:740] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:09:30.868060 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 110397us
I20260709 13:09:30.873152 21036 leader_election.cc:400] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 26. Current term is 27.
W20260709 13:09:30.830302 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 52918.9us
I20260709 13:09:30.873624 21036 leader_election.cc:403] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 13:09:30.875015 29058 raft_consensus.cc:3060] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:30.876119 21792 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:30.877998 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:30.878600 21214 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:30.880568 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 50083.6us
I20260709 13:09:30.887153 21214 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:30.814760 29029 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.896286 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:30.889184 29130 consensus_meta.cc:287] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.000s
I20260709 13:09:30.900921 28727 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:30.901785 28727 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.871625 29159 consensus_queue.cc:1048] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.094s
I20260709 13:09:30.884384 29058 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 26. Current term is 27.
I20260709 13:09:30.913781 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:30.918301 21036 leader_election.cc:400] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Vote denied by peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) with higher term. Message: Invalid argument: T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23.
I20260709 13:09:30.925451 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:30.926257 21214 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:30.926720 21138 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:09:30.939513 29146 consensus_queue.cc:260] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:30.941742 29029 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:30.946904 21793 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
W20260709 13:09:30.956485 29172 log.cc:927] Time spent T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:09:30.960186 29167 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:30.971962 29074 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:30.972487 29074 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:30.973549 29074 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.978111 28973 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:30.933312 28853 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:30.916342 29133 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.927574 29168 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:30.914723 29156 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.928089 29154 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.911381 29170 consensus_queue.cc:1048] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.133s
I20260709 13:09:30.912633 29130 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:30.988293 29178 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 13:09:30.993631 21216 raft_consensus.cc:1240] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:30.989775 29133 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:30.994712 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.020100 21213 raft_consensus.cc:2410] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:09:31.020768 29133 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:30.997004 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.024099 21212 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:09:31.001004 29156 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.026366 29156 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:31.027439 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 90559.1us
I20260709 13:09:31.030087 29146 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:31.005264 29154 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:31.006242 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:31.006596 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:31.034883 21795 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:31.008838 29173 log.cc:927] Time spent T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:09:31.041437 28727 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:31.014971 29130 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:31.015833 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.016294 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:30.992192 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.056344 21217 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 26: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:30.995035 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.000020 29057 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.058941 29057 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:31.061295 29057 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:31.071259 21767 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:31.073450 21767 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:30.975046 29074 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:31.083098 21215 raft_consensus.cc:1240] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:31.083817 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:31.085601 21795 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 26.
I20260709 13:09:31.087939 21213 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:31.090045 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.081459 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:31.092938 21113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.095166 21138 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:09:31.099489 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:31.091782 21217 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:31.103504 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.103897 21111 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:31.104519 21218 raft_consensus.cc:2393] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 26: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:31.106927 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:31.107661 21035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.107590 21140 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:31.125890 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:31.127170 21795 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:31.131429 21216 raft_consensus.cc:1275] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:31.076362 28973 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:31.133908 21221 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:09:31.136579 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:31.136953 21214 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
W20260709 13:09:31.132328 21113 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:31.130864 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:31.145311 21113 leader_election.cc:336] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:31.146472 21113 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:31.144511 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:31.155419 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:31.158392 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:31.159061 21138 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:31.160144 21113 leader_election.cc:336] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:31.163313 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d51215a492444539c4905a13bd0c12d" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:31.165225 21141 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:09:31.173940 21146 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:09:31.176455 28850 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:31.130059 28853 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.179112 28852 consensus_queue.cc:1059] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:31.179993 28852 consensus_queue.cc:1243] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:09:31.180537 28852 raft_consensus.cc:3055] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:31.180902 28852 raft_consensus.cc:740] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:31.181732 28852 consensus_queue.cc:260] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.182946 28852 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:31.164953 29027 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:31.184087 29027 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.179591 28853 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:31.136233 29069 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:31.129817 29130 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:31.166365 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:31.093631 28700 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 13:09:31.152467 29081 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:09:31.205299 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 105874us
I20260709 13:09:31.195931 29167 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:31.207855 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.208532 21214 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 14.
I20260709 13:09:31.202914 29168 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:31.198038 29069 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:31.213054 21113 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:31.195585 29130 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.218967 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.219705 21212 raft_consensus.cc:3060] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:31.206831 29167 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:31.222152 21113 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.211303 29168 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:31.233862 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.245157 21212 raft_consensus.cc:2468] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 27.
W20260709 13:09:31.248559 21113 leader_election.cc:336] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:31.249262 21147 raft_consensus.cc:2410] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:09:31.249327 21113 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:31.250213 21113 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:31.251672 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.252518 21212 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:31.254565 21217 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:09:31.260205 21221 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
W20260709 13:09:31.268608 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.275336 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.275663 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.276348 21218 raft_consensus.cc:2410] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:09:31.276948 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.279698 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.285157 21794 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEARNER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:31.287794 21217 pending_rounds.cc:85] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Aborting all ops after (but not including) 13
I20260709 13:09:31.288208 21217 pending_rounds.cc:107] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:09:31.288463 21217 raft_consensus.cc:2889] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:31.289377 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.290184 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.291599 21138 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 26.
I20260709 13:09:31.223582 29167 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:31.303429 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.303782 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.304381 21213 raft_consensus.cc:2393] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 23: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
W20260709 13:09:31.297895 21765 leader_election.cc:336] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.637s (SENT)
I20260709 13:09:31.305011 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:31.308893 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.314908 21216 raft_consensus.cc:2468] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14.
I20260709 13:09:31.317374 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.309552 21215 raft_consensus.cc:2376] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23. Re-sending same reply.
I20260709 13:09:31.320287 29032 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:31.322063 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.400374 29130 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:31.404206 29065 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
W20260709 13:09:31.404723 29065 raft_consensus.cc:1068] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:31.342801 29168 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.407858 29168 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:31.444651 20963 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 170) took 11689 ms (client timeout 15000 ms). Trace:
W20260709 13:09:31.451978 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.327788 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.331111 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.453676 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.453733 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.343590 21218 raft_consensus.cc:2368] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23.
I20260709 13:09:31.454452 21212 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 18.
I20260709 13:09:31.454957 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.455363 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.455996 21212 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:09:31.457098 21212 raft_consensus.cc:1275] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:09:31.458132 21218 raft_consensus.cc:1275] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:09:31.343900 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:31.356716 21113 leader_election.cc:336] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:31.459710 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 103022us
I20260709 13:09:31.459481 29069 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:31.357267 29177 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.463650 29177 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:31.340057 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:31.359040 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 201869us
I20260709 13:09:31.467633 21221 raft_consensus.cc:2376] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 14. Re-sending same reply.
I20260709 13:09:31.472721 21110 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:31.475579 21185 leader_election.cc:304] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:31.475739 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.359676 28845 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:09:31.363088 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:31.370184 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.383183 28853 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:31.385583 29043 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:31.321877 29081 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:31.452114 29168 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.342648 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.361708 29027 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:31.454962 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:31.516523 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 153525us
I20260709 13:09:31.517288 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:31.509182 19967 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 21097)
I20260709 13:09:31.490545 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.519503 21794 raft_consensus.cc:1240] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:31.476464 20970 catalog_manager.cc:5697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:31.524622 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:31.527626 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.528432 21216 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 27: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:31.531219 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.531908 21221 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 28.
W20260709 13:09:31.534919 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:31.538193 20970 catalog_manager.cc:5697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:31.544200 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:31.493288 29130 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:09:31.556128 29154 consensus_meta.cc:287] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Time spent flushing consensus metadata: real 0.524s	user 0.006s	sys 0.003s
I20260709 13:09:31.556289 29130 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:31.557271 29130 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:31.516371 28853 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.559610 28853 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:31.561156 21800 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:09:31.561565 21800 pending_rounds.cc:85] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 12
I20260709 13:09:31.561808 21800 pending_rounds.cc:107] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:09:31.561905 28853 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:31.562297 21800 raft_consensus.cc:2889] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:31.551051 29081 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:31.492487 28994 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.538894 29151 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.554236 21220 raft_consensus.cc:2410] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:09:31.527007 29133 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:09:31.534515 29043 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:31.534024 29129 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:31.561667 20970 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:31.545728 21111 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:31.544871 21792 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:31.526466 28928 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:31.602281 28928 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:31.601869 20970 catalog_manager.cc:5697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 12 to 13, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 13 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:31.603140 28928 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.589677 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:31.596400 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.607759 21216 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 25: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:31.602587 28688 raft_consensus.cc:2804] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:09:31.608596 28688 raft_consensus.cc:697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:31.609520 28688 consensus_queue.cc:237] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:31.601676 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56013.5us
I20260709 13:09:31.612862 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:31.614058 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:31.615375 29131 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:31.617452 29131 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:31.617866 29131 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.618216 29131 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:31.619452 29049 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:31.622439 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:31.535751 29168 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:31.642932 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.594305 29151 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.590673 28994 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:31.598994 29129 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:31.649273 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.689795 21216 raft_consensus.cc:1240] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:31.645056 29057 raft_consensus.cc:2749] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:31.692150 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:31.697523 21036 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.678s (SENT)
I20260709 13:09:31.651854 29129 raft_consensus.cc:3060] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:31.648787 28850 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
W20260709 13:09:31.699344 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 102611us
I20260709 13:09:31.691303 28847 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.712886 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:31.716997 21036 leader_election.cc:336] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.696s (SENT)
I20260709 13:09:31.719887 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:31.724341 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.729128 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:31.744290 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:31.744946 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 209400us
W20260709 13:09:31.748440 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:31.750660 21036 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.677s (SENT)
W20260709 13:09:31.753103 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 210754us
W20260709 13:09:31.749583 21767 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:31.827665 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.874974 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:31.876572 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 112011us
I20260709 13:09:31.711458 28850 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:31.885741 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.886629 21218 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:09:31.753252 28853 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:09:31.882622 21035 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:31.894744 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:31.877323 28852 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.001s
I20260709 13:09:31.732174 28847 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.902505 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:31.884796 29146 consensus_meta.cc:287] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.854s	user 0.013s	sys 0.003s
I20260709 13:09:31.876901 29196 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.908176 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.914456 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.915555 21221 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:31.877682 29185 consensus_queue.cc:1059] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 13:09:31.900133 29197 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.926765 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.911548 29146 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:31.934970 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.936645 21221 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:31.937981 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:31.939648 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:31.877928 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:31.943145 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 65252.8us
I20260709 13:09:31.946453 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.947242 21212 raft_consensus.cc:2368] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 19. Current term is 20.
I20260709 13:09:31.933981 29146 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:31.941037 29167 consensus_meta.cc:287] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.620s	user 0.007s	sys 0.004s
W20260709 13:09:31.950901 29191 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 13:09:31.955737 29189 log.cc:927] Time spent T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
W20260709 13:09:31.959614 21219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.020s	sys 0.036s
I20260709 13:09:31.962515 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:31.966398 21219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.020s	sys 0.036s
I20260709 13:09:31.966789 21212 raft_consensus.cc:2368] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 26. Current term is 27.
W20260709 13:09:31.947047 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:31.967651 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:31.964824 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 82318us
I20260709 13:09:31.940799 29185 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:31.918939 29196 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.971717 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.945289 29151 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:31.963268 29146 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:31.975701 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:31.975821 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:31.980075 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.971251 29200 raft_consensus.cc:3055] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:31.984486 21212 raft_consensus.cc:2393] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 27: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:31.938366 29197 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.965796 29167 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.991386 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:31.999321 21793 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 20.
I20260709 13:09:31.976330 29205 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:32.027341 29205 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:31.997783 29196 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:32.028443 21215 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:32.032586 29198 log.cc:927] Time spent T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.101s	user 0.001s	sys 0.001s
I20260709 13:09:32.001858 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:31.984544 28994 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.008038 21213 raft_consensus.cc:2368] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 26. Current term is 27.
I20260709 13:09:32.009326 29197 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:32.011861 28847 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:32.012964 28850 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.014204 21798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:32.017549 29177 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:31.985643 29200 raft_consensus.cc:740] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:32.021569 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:31.976986 29185 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:32.039371 21215 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:09:32.043674 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.050988 29200 consensus_queue.cc:260] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.052373 29200 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:32.052078 29131 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.053946 29167 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:32.048893 29197 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:32.054203 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.054687 29197 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.055228 29197 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:32.055531 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.056161 21214 raft_consensus.cc:2376] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23. Re-sending same reply.
I20260709 13:09:32.057899 21212 raft_consensus.cc:2393] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 25: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:32.060480 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.045136 21221 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:09:32.030964 21218 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22.
W20260709 13:09:31.445232 20963 rpcz_store.cc:269] 0709 13:09:19.755254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.755497 (+   243us) service_pool.cc:224] Handling call
0709 13:09:31.444550 (+11689053us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:28.907407 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:28.908105 (+   698us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:28.908162 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:09:28.908181 (+    19us) tablet.cc:662] Decoding operations
0709 13:09:28.909771 (+  1590us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:28.909876 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:28.909910 (+    34us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:09:28.911257 (+  1347us) tablet.cc:701] Row locks acquired
0709 13:09:28.916983 (+  5726us) write_op.cc:260] PREPARE: finished
0709 13:09:28.934423 (+ 17440us) write_op.cc:270] Start()
0709 13:09:28.959274 (+ 24851us) write_op.cc:276] Timestamp: P: 1783602568930388 usec, L: 0
0709 13:09:28.959345 (+    71us) op_driver.cc:348] REPLICATION: starting
0709 13:09:28.960360 (+  1015us) log.cc:844] Serialized 6806 byte log entry
0709 13:09:29.603691 (+643331us) op_driver.cc:464] REPLICATION: finished
0709 13:09:29.766226 (+162535us) write_op.cc:301] APPLY: starting
0709 13:09:29.777055 (+ 10829us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:30.020121 (+243066us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:30.020292 (+   171us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:31.003120 (+982828us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:31.051183 (+ 48063us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:09:31.051262 (+    79us) write_op.cc:312] APPLY: finished
0709 13:09:31.051852 (+   590us) log.cc:844] Serialized 156 byte log entry
0709 13:09:31.213940 (+162088us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:31.231802 (+ 17862us) write_op.cc:454] Released schema lock
0709 13:09:31.260656 (+ 28854us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":1604767,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":162132,"apply.run_cpu_time_us":42593,"apply.run_wall_time_us":1560158,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75221,"num_ops":17,"prepare.queue_time_us":208696,"prepare.run_cpu_time_us":89103,"prepare.run_wall_time_us":240330,"raft.queue_time_us":221274,"raft.run_cpu_time_us":21751,"raft.run_wall_time_us":79406,"replication_time_us":644109,"thread_start_us":634508,"threads_started":5,"wal-append.queue_time_us":454560,"wal-append.run_cpu_time_us":25099,"wal-append.run_wall_time_us":1276884}]]}
I20260709 13:09:32.064478 21216 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:32.062897 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.066028 21221 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 28.
I20260709 13:09:32.064183 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.200031 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.202766 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.203647 21212 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 20.
I20260709 13:09:32.205468 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.206580 21212 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 23. Current term is 24. Ops: []
W20260709 13:09:32.208019 21765 leader_election.cc:336] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.922s (SENT)
I20260709 13:09:32.208669 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.054250 29131 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:32.212234 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.216460 21214 raft_consensus.cc:1240] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:09:32.198026 28850 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.207641 28994 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:32.222751 29157 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.031s	sys 0.097s
W20260709 13:09:32.227355 29129 consensus_meta.cc:287] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.003s
W20260709 13:09:32.227571 29157 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.031s	sys 0.097s
I20260709 13:09:32.227902 29129 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.210510 28847 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.229619 28847 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.230091 29129 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.232043 28847 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.227478 28994 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.235443 28994 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.226389 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.236064 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:32.236720 21794 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:09:32.238348 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.241014 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:32.246147 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:32.247371 29195 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:32.247804 29195 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.248142 29195 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:32.264958 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.281085 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:32.282203 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:32.286455 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 59209.4us
I20260709 13:09:32.225024 28850 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.291700 29205 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:32.302700 29185 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:32.312722 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.317448 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:32.319546 21038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:32.321025 21036 leader_election.cc:336] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.830s (SENT)
I20260709 13:09:32.321918 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.325922 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.326797 21218 raft_consensus.cc:2376] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 22. Re-sending same reply.
I20260709 13:09:32.327477 28850 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.328075 29167 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:32.330967 21036 leader_election.cc:336] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Timed out: RequestConsensusVote RPC to 127.19.127.195:46249 timed out after 1.894s (SENT)
I20260709 13:09:32.332772 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:32.331655 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.334125 29146 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.336516 29146 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.337740 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:32.338352 21799 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:32.339625 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.340257 21214 raft_consensus.cc:2468] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:32.341074 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.341598 21214 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:32.341701 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.343261 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.343631 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.344640 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.344769 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.345659 21212 raft_consensus.cc:2410] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:09:32.345760 21218 raft_consensus.cc:2410] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 13:09:32.346419 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:32.349406 29184 log.cc:927] Time spent T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: Append to log took a long time: real 1.020s	user 0.007s	sys 0.007s
I20260709 13:09:32.349731 29208 consensus_queue.cc:1059] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:32.350992 29208 raft_consensus.cc:3055] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:32.352008 29208 raft_consensus.cc:740] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:32.353185 29208 consensus_queue.cc:260] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.353742 29195 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.354854 29208 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:32.346899 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.369392 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.369969 21213 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 18.
I20260709 13:09:32.373443 29092 consensus_queue.cc:1059] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:09:32.374540 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.374468 29092 raft_consensus.cc:3055] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:32.375254 29092 raft_consensus.cc:740] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:32.375487 29043 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.376178 29092 consensus_queue.cc:260] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.377463 29092 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:32.378221 29192 log.cc:927] Time spent T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.644s	user 0.000s	sys 0.012s
I20260709 13:09:32.380064 29197 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.372161 21214 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 21.
I20260709 13:09:32.384544 29043 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:32.385661 29193 log.cc:927] Time spent T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.641s	user 0.000s	sys 0.011s
W20260709 13:09:32.389746 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 58826.3us
I20260709 13:09:32.389655 29032 consensus_queue.cc:1048] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:32.395584 29197 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.391440 29133 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.396430 29133 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.396883 21217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1096) took 1110 ms. Trace:
I20260709 13:09:32.391454 29170 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:32.397269 21217 rpcz_store.cc:276] 0709 13:09:31.286852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.287241 (+   389us) service_pool.cc:224] Handling call
0709 13:09:31.287612 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:31.289057 (+  1445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:31.289089 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:31.289503 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 13:09:31.623315 (+333812us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:31.643789 (+ 20474us) log.cc:844] Serialized 14 byte log entry
0709 13:09:31.643996 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.644071 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:32.337723 (+693652us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b18004d8000. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f522 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:09:32.377601 (+ 39878us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:32.396676 (+ 19075us) raft_consensus.cc:1685] finished
0709 13:09:32.396698 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:32.396828 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21472,"prepare.run_cpu_time_us":250,"prepare.run_wall_time_us":1544,"raft.queue_time_us":66052,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":8241,"spinlock_wait_cycles":30732416,"thread_start_us":284543,"threads_started":1,"wal-append.queue_time_us":391409}
W20260709 13:09:32.392238 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 767749us
I20260709 13:09:32.400041 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:32.400604 21800 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:32.401580 29029 consensus_queue.cc:1048] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:32.392892 21220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54620 (request call id 1073) took 1402 ms. Trace:
I20260709 13:09:32.404364 21220 rpcz_store.cc:276] 0709 13:09:30.990818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.015550 (+ 24732us) service_pool.cc:224] Handling call
0709 13:09:32.392719 (+1377169us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54003fa7d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f425202fa 00007f7f474e1b17 00007f7f40a5c6d4 00007f7f40a5c662 00007f7f40a5c591 00007f7f40a5c50c 00007f7f40a5b7c2 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:32.392841 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149232384}
I20260709 13:09:32.392999 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.393400 29078 consensus_queue.cc:1048] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:32.410378 21219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54620 (request call id 1072) took 1420 ms. Trace:
I20260709 13:09:32.411433 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.412218 21215 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 21: Already voted for candidate 9384358ce43f4e3fac2ae44bdf11c389 in this term.
I20260709 13:09:32.410931 21219 rpcz_store.cc:276] 0709 13:09:30.990466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.003462 (+ 12996us) service_pool.cc:224] Handling call
0709 13:09:31.003875 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:31.079903 (+ 76028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:31.079941 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:31.098578 (+ 18637us) log.cc:844] Serialized 26 byte log entry
0709 13:09:31.966873 (+868295us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:31.966918 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.403621 (+436703us) spinlock_profiling.cc:243] Waited 58.2 ms on lock 0x7b54002ebb54. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f4251ce01 00007f7f4251b83e 00007f7f474e0419 00007f7f40a57bd4 00007f7f40a57b62 00007f7f40a57a91 00007f7f40a57a0c 00007f7f40a56cc2 00007f7f4085748c 00007f7f408568c6
0709 13:09:32.403668 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:32.403715 (+    47us) raft_consensus.cc:1685] finished
0709 13:09:32.403725 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:32.410325 (+  6600us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13139,"raft.queue_time_us":14119,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":19236,"spinlock_wait_cycles":133792000,"thread_start_us":803908,"threads_started":1,"wal-append.queue_time_us":363716}
W20260709 13:09:32.413321 21036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.414417 21110 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.412676 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.416440 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.417086 21212 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:09:32.417294 21215 raft_consensus.cc:2393] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 14: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:32.417255 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.417819 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.415694 29170 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:09:32.420008 21799 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:32.424275 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.420681 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.426517 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.427874 21218 raft_consensus.cc:1240] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:32.428014 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:32.434252 21187 leader_election.cc:336] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.435212 21187 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.407702 29027 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.431511 29157 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:32.447916 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.431954 29154 raft_consensus.cc:2764] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:09:32.449148 29126 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.431308 29167 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.440280 21800 raft_consensus.cc:2468] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:32.441865 29032 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:32.444352 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.444983 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.415264 29205 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.422308 29139 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:32.451032 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.452370 21138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49718 (request call id 3764) took 1097 ms. Trace:
I20260709 13:09:32.439808 29195 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.435727 28850 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:09:32.451930 29074 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:09:32.457325 21138 rpcz_store.cc:276] 0709 13:09:31.355090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.355377 (+   287us) service_pool.cc:224] Handling call
0709 13:09:31.355721 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:31.356092 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:31.356117 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:31.356697 (+   580us) log.cc:844] Serialized 52 byte log entry
0709 13:09:31.681177 (+324480us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:09:31.756625 (+ 75448us) log.cc:844] Serialized 14 byte log entry
0709 13:09:32.383663 (+627038us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f420b786e 00007f7f420b6bb6 00007f7f420b67d8 00007f7f420c0221 00007f7f420c01d9 00007f7f420c0169
0709 13:09:32.409207 (+ 25544us) log.cc:844] Serialized 14 byte log entry
0709 13:09:32.451945 (+ 42738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.452029 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:32.452138 (+   109us) raft_consensus.cc:1685] finished
0709 13:09:32.452153 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:32.452304 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":693658,"prepare.queue_time_us":217713,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":2847,"raft.queue_time_us":708131,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":7331,"spinlock_wait_cycles":10586880,"thread_start_us":368558,"threads_started":2,"wal-append.queue_time_us":387003}
I20260709 13:09:32.449716 29126 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.462033 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:32.463925 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.464164 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:32.466921 21113 consensus_peers.cc:597] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:32.461911 29205 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.471925 29043 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:32.483060 21212 raft_consensus.cc:1275] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
W20260709 13:09:32.483820 21113 leader_election.cc:336] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.465377 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.485371 21146 raft_consensus.cc:2393] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 21: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:32.485664 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.488451 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:32.489876 21187 leader_election.cc:336] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.490310 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.491393 21765 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.465674 29185 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:32.488819 29074 raft_consensus.cc:1068] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:32.504968 21216 raft_consensus.cc:1240] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:32.482358 29133 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.487185 29157 raft_consensus.cc:2764] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:09:32.463688 29167 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.516156 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.517059 28850 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:32.518335 29167 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:32.493568 29208 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:32.493248 29146 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:32.512212 29205 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:32.512682 29032 raft_consensus.cc:2804] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:32.520092 29032 raft_consensus.cc:697] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:32.520999 29032 consensus_queue.cc:237] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.508762 29197 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:32.452026 29027 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.518373 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.526473 28850 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:32.527872 21036 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.528684 28850 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:32.529209 28850 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:32.530294 28850 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:32.509660 21187 leader_election.cc:336] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:32.531644 19967 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 21097)
I20260709 13:09:32.531919 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.514276 29151 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:32.538703 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:32.543226 21187 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.543977 29151 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:09:32.544418 29151 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.544831 29151 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:09:32.544879 21187 leader_election.cc:336] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.545378 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.546147 29065 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:32.547470 21187 leader_election.cc:336] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.548316 21187 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.549126 29065 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:32.550505 21187 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.551187 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.552068 29065 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:32.553936 21187 leader_election.cc:336] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:32.555428 21187 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:32.556624 29194 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.813s	user 0.000s	sys 0.003s
W20260709 13:09:32.556834 21187 leader_election.cc:336] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.557454 21187 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.544420 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.558434 29065 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:32.559324 29043 raft_consensus.cc:3060] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:32.560802 21187 leader_election.cc:336] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.564523 29027 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:32.565037 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.565990 21215 raft_consensus.cc:2410] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:09:32.567559 29043 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:32.569455 29043 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.554888 29151 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.570420 29167 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.571396 29167 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.573547 29167 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.569993 29043 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.578141 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.579960 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.580165 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.582968 29167 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:32.583463 21036 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.583809 21219 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:32.585089 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.586213 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.587371 28847 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:32.588151 28847 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:32.589488 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.591421 28847 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:32.592592 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.593575 21219 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
W20260709 13:09:32.594210 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 140157us
I20260709 13:09:32.580453 29151 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:32.588182 29144 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:32.594611 28847 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:32.595626 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.596323 29151 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:32.596789 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.597301 29127 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:32.597544 29151 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:32.598116 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.598809 29151 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:32.599748 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.600728 29151 raft_consensus.cc:2804] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:32.601085 29151 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.601133 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.601768 29126 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:32.602077 28847 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:32.602496 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.601704 29151 raft_consensus.cc:3060] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:32.603310 28847 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:32.603770 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.604470 28847 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:32.605607 21036 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.606348 28847 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:32.607239 21036 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:32.608314 29144 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:32.608469 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.609856 29144 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:32.608003 28847 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:32.614897 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.616904 29043 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:32.625792 29167 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.634768 29167 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.540305 29133 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:32.637995 29167 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.638804 29167 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:32.641371 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.641991 21218 raft_consensus.cc:2468] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 23.
I20260709 13:09:32.642431 29167 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.642949 29167 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.645459 29167 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:32.646142 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.646406 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.646737 21218 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:09:32.647147 29167 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:32.649273 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:32.650017 21765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.651094 29167 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:32.652518 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.654395 28889 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:09:32.654425 29167 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:32.654844 21218 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 29.
I20260709 13:09:32.581276 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:32.679381 21212 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:32.669332 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.670097 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:32.596815 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 182465us
I20260709 13:09:32.659766 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.657227 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.666039 21184 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:32.686872 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.688448 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
W20260709 13:09:32.688998 21764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.691018 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:32.681524 21213 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 24.
I20260709 13:09:32.694449 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:32.695513 21138 raft_consensus.cc:2393] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 29: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:32.575021 29168 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:32.699607 21146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:32.656050 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:32.701870 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45874.6us
I20260709 13:09:32.703526 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:32.704325 21800 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:32.711846 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:32.655264 28889 raft_consensus.cc:1068] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:32.721655 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.721899 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.655764 29197 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:32.727617 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:32.729265 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:32.730428 21792 raft_consensus.cc:2410] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46c1676121b64b7d9bcf2227cc639dd3 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 13:09:32.731274 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.696233 29065 raft_consensus.cc:2804] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:32.724982 21111 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.698562 29144 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:32.701683 29043 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.710112 29146 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:09:32.747663 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 51886.6us
I20260709 13:09:32.752202 21035 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:32.724754 29168 raft_consensus.cc:2804] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:32.752719 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.745352 29197 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.723872 21800 raft_consensus.cc:2468] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:32.721587 29133 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:32.762373 21212 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 19.
I20260709 13:09:32.763875 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.773432 21216 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 27.
I20260709 13:09:32.765372 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.766517 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:32.775820 21213 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 29.
I20260709 13:09:32.759649 29043 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:32.783241 21220 raft_consensus.cc:1240] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:32.785456 29043 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:32.784123 29061 consensus_queue.cc:1059] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:09:32.710295 29078 consensus_queue.cc:1059] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:09:32.790697 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.792061 21138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:32.794538 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f0de1289c4c98ae7ce302ca584a30" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:32.795792 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:32.802465 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 50405.9us
I20260709 13:09:32.804800 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.808516 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:32.811550 21187 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:32.812849 21187 leader_election.cc:336] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:32.814196 21187 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.814693 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.815893 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:32.752676 29177 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:32.821673 21143 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:09:32.827767 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:32.751804 29065 raft_consensus.cc:697] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:32.756152 29168 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.738168 29049 consensus_queue.cc:1059] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:09:32.842170 29065 consensus_queue.cc:237] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:32.843524 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:32.835789 29195 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:09:32.764947 29205 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:32.820144 28688 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:32.830859 28928 raft_consensus.cc:3055] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:32.852262 29168 raft_consensus.cc:3060] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:32.769447 29027 consensus_queue.cc:1048] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:09:32.820469 28850 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:32.802244 29144 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.801499 29133 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.838934 21184 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:32.846004 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:32.852694 29146 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:32.873601 21113 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.879091 21113 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:32.877745 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:32.882973 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64558.7us
W20260709 13:09:32.839505 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 151157us
I20260709 13:09:32.887470 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.887485 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:09:32.885033 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:32.862742 28928 raft_consensus.cc:740] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:32.872404 28847 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:32.861107 29205 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.922370 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.896852 29156 raft_consensus.cc:3055] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:09:32.903591 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 89436.2us
I20260709 13:09:32.904433 21110 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.875051 29130 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 13:09:32.908165 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:32.897356 28850 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:32.862357 28688 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:32.925892 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52327.4us
W20260709 13:09:32.927639 21187 leader_election.cc:336] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.941485 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.917153 29167 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:32.921667 29209 log.cc:927] Time spent T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 13:09:32.942857 21113 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:32.886397 29146 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.881495 29144 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:32.943779 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:32.921810 29049 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 13:09:32.945741 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 93546us
I20260709 13:09:32.926128 29156 raft_consensus.cc:740] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:32.928889 29061 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:32.928210 29130 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.927606 29227 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:32.945724 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.981388 29049 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:32.994459 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 48937.2us
I20260709 13:09:32.997946 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:32.932408 29131 consensus_queue.cc:1059] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:09:32.980935 28845 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:32.982849 29228 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:33.013957 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 71136.3us
I20260709 13:09:33.011302 29078 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:32.981560 29236 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:32.989696 29130 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:09:33.038559 29224 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:09:33.039975 29230 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:09:32.984663 29081 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:09:33.008349 29049 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:09:33.010528 29043 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:09:32.997174 29197 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.014896 29167 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:33.048458 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:32.976699 28928 consensus_queue.cc:260] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.026111 29237 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
W20260709 13:09:33.035355 29219 log.cc:927] Time spent T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:09:33.034902 29131 consensus_queue.cc:1243] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer 46c1676121b64b7d9bcf2227cc639dd3 log is divergent from this leader: its last log entry 22.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:09:33.056840 29167 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:33.069157 29222 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:09:33.027235 28845 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.081270 28928 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:33.097285 21221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56804 (request call id 3576) took 1040 ms. Trace:
I20260709 13:09:33.008764 28847 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.068414 29043 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 29 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
W20260709 13:09:33.084743 29229 log.cc:927] Time spent T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:09:33.087571 29133 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:33.106173 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:33.088192 29227 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:33.091843 29205 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:33.092350 28845 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.096186 28688 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.098495 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:33.083434 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:33.097726 21221 rpcz_store.cc:276] 0709 13:09:32.057370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.210037 (+152667us) service_pool.cc:224] Handling call
0709 13:09:32.220931 (+ 10894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:32.221278 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.221316 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:32.221874 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 13:09:32.275629 (+ 53755us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:32.312563 (+ 36934us) log.cc:844] Serialized 14 byte log entry
0709 13:09:32.312816 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.312893 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:32.740051 (+427158us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b1800200260. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:09:33.081257 (+341206us) raft_consensus.cc:1685] finished
0709 13:09:33.097072 (+ 15815us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.097227 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50063,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":5888,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":2340,"spinlock_wait_cycles":4240256,"thread_start_us":87962,"threads_started":2,"wal-append.queue_time_us":99176}
W20260709 13:09:33.104653 21111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:33.082209 29167 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:33.113065 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:33.107905 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:33.144611 29027 raft_consensus.cc:3055] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:33.145144 29027 raft_consensus.cc:740] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:33.146159 29027 consensus_queue.cc:260] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.035157 29156 consensus_queue.cc:260] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.135820 21147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49724 (request call id 3716) took 1426 ms. Trace:
I20260709 13:09:33.133898 21141 raft_consensus.cc:2468] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 25.
I20260709 13:09:33.134582 21794 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 23: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:33.153842 29092 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.159963 29092 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.162099 29092 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:33.162585 29146 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:33.164412 29221 log.cc:927] Time spent T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.313s	user 0.000s	sys 0.006s
W20260709 13:09:33.165257 29151 consensus_meta.cc:287] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.561s	user 0.004s	sys 0.008s
I20260709 13:09:33.165660 29151 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.071254 29237 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.172387 29242 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.172158 29043 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.172873 29242 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.162020 29144 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:33.176007 21110 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:33.176553 29191 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.200s	user 0.000s	sys 0.002s
W20260709 13:09:33.177619 29235 log.cc:927] Time spent T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:09:33.068260 28850 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.170140 28911 raft_consensus.cc:3055] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:33.182641 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:33.183825 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:33.184345 21800 raft_consensus.cc:1240] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:09:33.184470 21794 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
W20260709 13:09:33.186308 21113 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:33.187657 21113 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.183403 21144 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 26.
I20260709 13:09:33.149806 21147 rpcz_store.cc:276] 0709 13:09:31.709138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.709396 (+   258us) service_pool.cc:224] Handling call
0709 13:09:31.727542 (+ 18146us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:32.446730 (+719188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.446778 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:32.452637 (+  5859us) log.cc:844] Serialized 52 byte log entry
0709 13:09:32.542621 (+ 89984us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:32.542674 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.542750 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.135570 (+592820us) raft_consensus.cc:1685] finished
0709 13:09:33.135599 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.135760 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":718654,"raft.queue_time_us":721670,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":49019,"threads_started":1,"wal-append.queue_time_us":231058}
I20260709 13:09:33.175053 29242 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.194058 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:33.199790 29250 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.193024 29027 raft_consensus.cc:3060] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:33.191103 28911 raft_consensus.cc:740] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:33.200555 29250 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.217490 28911 consensus_queue.cc:260] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.218935 28911 raft_consensus.cc:3060] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:33.226493 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 245777us
I20260709 13:09:33.226627 29237 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.226871 29195 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:33.228515 21185 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:33.226132 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:33.229300 29133 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:33.226112 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 387213us
I20260709 13:09:33.234756 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 
I20260709 13:09:33.248924 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:33.254454 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 60511.9us
I20260709 13:09:33.257820 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:33.259882 29242 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:33.260435 29242 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.260798 29242 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:09:33.272154 29241 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:09:33.274588 21144 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:09:33.276614 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:33.270885 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:33.270954 21794 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:09:33.282843 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:33.286545 21217 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:09:33.288010 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:33.288713 21215 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:33.288919 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:33.270857 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:33.226521 29233 raft_consensus.cc:2804] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:33.281375 29151 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.242739 29227 raft_consensus.cc:2804] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:33.236284 29250 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.307981 29074 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:09:33.316987 21794 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:09:33.340715 21794 pending_rounds.cc:85] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Aborting all ops after (but not including) 15
I20260709 13:09:33.341081 21794 pending_rounds.cc:107] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:09:33.341423 21794 raft_consensus.cc:2889] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:33.356355 21217 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:09:33.360476 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:33.378239 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:33.354952 29233 raft_consensus.cc:697] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:33.388289 29233 consensus_queue.cc:237] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:33.391036 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:33.391516 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 449209us
W20260709 13:09:33.392047 21187 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.392434 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:33.351025 29074 raft_consensus.cc:1068] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Unable to promote non-voter b70fc04de37e4239a8336534f0c9c0fb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:09:33.408413 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:33.411358 29232 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:33.353960 29227 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:33.418267 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 52027.3us
W20260709 13:09:33.419432 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 59037.5us
W20260709 13:09:33.403540 29168 consensus_meta.cc:287] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.004s
I20260709 13:09:33.312367 29154 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.107s
W20260709 13:09:33.447211 29247 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:09:33.430703 29151 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:33.351814 29127 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.072s
I20260709 13:09:33.446285 29155 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:33.449492 29155 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.449894 29155 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:33.446133 29168 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.465948 29168 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:33.431236 28847 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:09:33.479549 29167 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.481827 29167 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:33.502018 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:33.503201 29245 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:33.503635 29245 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:33.504005 29245 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:33.505609 29131 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:33.485083 29130 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.508693 29131 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:33.509832 29130 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:33.485920 29156 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:33.511845 29245 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.515398 21792 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:09:33.413918 29049 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.532380 21219 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:33.443128 29227 raft_consensus.cc:3060] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:33.525627 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:33.498642 29242 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.527981 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:33.532119 19967 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 21097)
I20260709 13:09:33.539669 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:33.540283 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:33.524151 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:33.533933 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:33.535768 21140 raft_consensus.cc:1275] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:09:33.541584 21140 pending_rounds.cc:85] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 12
I20260709 13:09:33.541826 21140 pending_rounds.cc:107] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:09:33.541796 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:33.542271 21140 raft_consensus.cc:2889] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:33.541319 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:33.544330 21212 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:33.547693 21792 raft_consensus.cc:2393] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 23: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:33.555365 21142 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 26. Current term is 27. Ops: []
I20260709 13:09:33.565966 21216 raft_consensus.cc:1240] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Rejecting Update request from peer b70fc04de37e4239a8336534f0c9c0fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:33.539940 29245 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.577066 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:33.579874 21213 raft_consensus.cc:1275] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:09:33.581506 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:33.584851 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:33.587726 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:33.589027 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:33.589568 21113 consensus_peers.cc:597] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:33.595249 21212 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9384358ce43f4e3fac2ae44bdf11c389 in term 25.
I20260709 13:09:33.598639 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:33.598430 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
W20260709 13:09:33.596103 21187 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.606925 21221 raft_consensus.cc:3060] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:33.598605 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:33.598122 29146 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:33.543263 29232 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:09:33.608875 21217 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:09:33.598196 29168 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:33.598688 29226 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:09:33.595875 29133 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.606973 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:33.595515 29155 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:33.646690 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49295.3us
I20260709 13:09:33.595649 29249 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
W20260709 13:09:33.664218 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 68132.6us
W20260709 13:09:33.665176 21187 leader_election.cc:336] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.665850 21187 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:33.653023 21113 leader_election.cc:336] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.660998 21220 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:33.668439 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:33.671876 21221 raft_consensus.cc:2468] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 28.
I20260709 13:09:33.686170 21798 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 26. Current term is 27. Ops: []
I20260709 13:09:33.627377 29197 consensus_queue.cc:1048] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:09:33.712806 21794 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:09:33.689518 29178 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:33.721045 29242 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.686770 29205 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:33.723589 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:33.650352 29133 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:33.736012 21113 leader_election.cc:336] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.695840 29057 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:33.740617 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:33.751730 29049 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 30 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:33.754841 29049 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:33.765110 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:33.765799 21221 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 29 FOLLOWER]: Advancing to term 30
W20260709 13:09:33.765142 21113 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:33.738852 29168 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:09:33.780471 29133 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:33.781397 29133 raft_consensus.cc:1064] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:09:33.782366 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:33.784739 29155 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:33.784184 29133 consensus_queue.cc:237] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:33.789578 21111 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:33.790807 21111 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:33.793649 29061 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:33.794268 29227 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:33.796023 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:33.801075 21184 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:33.803256 29232 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 13:09:33.803232 29197 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:33.829196 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:33.843461 21221 raft_consensus.cc:2468] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 30.
I20260709 13:09:33.836942 29167 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:33.845537 21111 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:09:33.848769 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 183612us
I20260709 13:09:33.828467 28911 raft_consensus.cc:2804] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:09:33.806272 29061 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:33.863847 21113 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:33.864886 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 121038us
I20260709 13:09:33.881568 21144 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:09:33.855752 29027 raft_consensus.cc:2804] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 13:09:33.892861 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:33.895407 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:33.898576 21800 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 27: Already voted for candidate b70fc04de37e4239a8336534f0c9c0fb in this term.
I20260709 13:09:33.906585 21139 raft_consensus.cc:2393] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb in current term 27: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:33.887734 28911 raft_consensus.cc:697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:33.926141 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 75937.8us
W20260709 13:09:33.927348 21187 leader_election.cc:336] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:33.917824 29268 log.cc:927] Time spent T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:09:33.931785 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:33.932363 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:33.925807 29027 raft_consensus.cc:697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 30 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:33.934855 29027 consensus_queue.cc:237] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.892861 29232 consensus_queue.cc:1059] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 13:09:33.928886 21187 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:33.938879 29233 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:33.940452 21187 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.940227 28911 consensus_queue.cc:237] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:33.940990 21187 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:33.941720 29270 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:33.942790 21187 leader_election.cc:336] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.943241 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:33.921339 29144 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
W20260709 13:09:33.944998 21187 leader_election.cc:336] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.948321 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:33.953142 21110 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:33.947913 29232 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:09:33.966737 29049 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 13:09:33.976635 21187 leader_election.cc:336] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.977311 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:33.956236 29227 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:33.956045 29127 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:33.971242 29074 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:33.930869 29061 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:34.003049 21113 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:33.987268 29185 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:33.994282 29127 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.019539 29127 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:34.026505 29271 log.cc:927] Time spent T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.344s	user 0.004s	sys 0.000s
I20260709 13:09:34.051105 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.051947 21214 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 24: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:34.058588 21140 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 30 LEADER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 29. Current term is 30. Ops: []
W20260709 13:09:34.045420 29276 log.cc:927] Time spent T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:09:34.073448 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 89792.9us
W20260709 13:09:34.077688 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:34.078267 21187 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:34.078609 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:34.097016 21036 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:34.097414 29275 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:09:34.052587 29127 consensus_queue.cc:1059] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:09:34.098519 29167 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:34.099046 29167 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 LEADER]: Becoming Leader. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:34.099895 29167 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.113946 29273 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:34.115705 29197 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:34.147871 29074 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:34.148600 29074 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.153677 29227 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.154412 29227 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.133088 28847 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:34.159086 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.169600 28889 raft_consensus.cc:3055] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:09:34.176316 28889 raft_consensus.cc:740] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:34.155272 29261 log.cc:927] Time spent T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.469s	user 0.002s	sys 0.002s
I20260709 13:09:34.192162 21141 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:09:34.177301 28889 consensus_queue.cc:260] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.193194 28889 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:34.141499 29197 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:34.197160 29280 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:09:34.161661 29260 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.198021 29260 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.159662 28847 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.153731 29168 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.164013 29151 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.203454 29168 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.203819 29151 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.177687 29240 consensus_queue.cc:1059] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:09:34.157032 29227 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.210500 21147 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 13:09:34.218397 21139 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:09:34.219331 29185 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:34.220000 29185 consensus_queue.cc:1243] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 23.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:09:34.221654 29185 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:34.205832 29240 raft_consensus.cc:3055] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:34.229955 21139 pending_rounds.cc:85] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 14
I20260709 13:09:34.230355 21139 pending_rounds.cc:107] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 13:09:34.173445 29157 consensus_queue.cc:1048] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:09:34.230631 21139 raft_consensus.cc:2889] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:34.231562 21217 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:09:34.226411 29249 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.236289 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:34.240177 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75007.7us
I20260709 13:09:34.237342 21220 raft_consensus.cc:1240] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Rejecting Update request from peer 9384358ce43f4e3fac2ae44bdf11c389 for earlier term 29. Current term is 30. Ops: []
I20260709 13:09:34.241281 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:34.241873 21792 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:34.237790 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:34.239102 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.248859 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.249679 21145 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:34.251375 21110 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3
I20260709 13:09:34.257237 29168 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.252882 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:34.269547 29061 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:34.269759 21218 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:09:34.270123 29061 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:34.270586 29061 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:34.230675 29240 raft_consensus.cc:740] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:34.273038 29240 consensus_queue.cc:260] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.274622 29074 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:34.275424 29151 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.248183 29249 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:34.285403 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 104321us
I20260709 13:09:34.291363 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:34.292272 21767 consensus_peers.cc:597] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:34.293989 28847 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:34.295130 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.295941 29197 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:34.279466 29260 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.304502 29081 raft_consensus.cc:3055] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:34.304986 29081 raft_consensus.cc:740] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:34.306066 29249 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.305846 29081 consensus_queue.cc:260] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.306748 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.285104 29240 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:34.312825 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:34.306888 29081 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:34.319442 29227 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.316586 21799 raft_consensus.cc:1275] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:09:34.317281 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.323521 21794 raft_consensus.cc:1275] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:09:34.323670 21799 raft_consensus.cc:1240] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Rejecting Update request from peer 46c1676121b64b7d9bcf2227cc639dd3 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 13:09:34.325531 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.326283 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:34.381639 29296 log.cc:927] Time spent T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:09:34.353287 29288 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.414652 29288 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.320381 29208 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:34.418535 29208 raft_consensus.cc:493] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.418972 29208 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:34.421521 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:34.422943 20970 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 244) took 64271 ms (client timeout 15000 ms). Trace:
I20260709 13:09:34.329016 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:34.420259 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:34.427480 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:34.391341 21215 raft_consensus.cc:1240] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Rejecting Update request from peer 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
W20260709 13:09:34.428804 21113 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:34.359488 29195 consensus_queue.cc:1048] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 13:09:34.328559 29151 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.104s
I20260709 13:09:34.432739 29249 consensus_queue.cc:1059] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:34.433172 29254 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.433457 29249 consensus_queue.cc:1243] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 30.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:09:34.433612 21184 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.433732 29254 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.436231 29254 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.436547 29226 raft_consensus.cc:3055] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 13:09:34.437129 29226 raft_consensus.cc:740] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 9384358ce43f4e3fac2ae44bdf11c389, State: Running, Role: LEADER
I20260709 13:09:34.438086 29226 consensus_queue.cc:260] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.439307 29226 raft_consensus.cc:3060] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:09:34.439818 29144 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.440366 29144 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.442473 29144 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:34.349367 21220 raft_consensus.cc:2410] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b70fc04de37e4239a8336534f0c9c0fb for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 13:09:34.444149 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:34.359714 28889 consensus_queue.cc:1048] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.089s
I20260709 13:09:34.324391 29227 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.344743 29146 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:09:34.423430 20970 rpcz_store.cc:269] 0709 13:08:30.158831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.172581 (+ 13750us) service_pool.cc:224] Handling call
0709 13:09:34.422869 (+64250288us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:31.940220 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:32.010025 (+ 69805us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:32.010081 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:09:32.010108 (+    27us) tablet.cc:662] Decoding operations
0709 13:09:32.011004 (+   896us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:32.011087 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:32.011128 (+    41us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:09:32.011741 (+   613us) tablet.cc:701] Row locks acquired
0709 13:09:32.011770 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:09:32.011996 (+   226us) write_op.cc:270] Start()
0709 13:09:32.012162 (+   166us) write_op.cc:276] Timestamp: P: 1783602572011930 usec, L: 0
0709 13:09:32.012196 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:09:32.013060 (+   864us) log.cc:844] Serialized 1718 byte log entry
0709 13:09:32.516234 (+503174us) op_driver.cc:464] REPLICATION: finished
0709 13:09:32.617733 (+101499us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b4400022b4c. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f355c85 00007f7f3f3566a2 00007f7f420c81db 00007f7f420c3aaf 00007f7f3f4120ac 00007f7f4826cab9 00007f7f3f42e69b 00007f7f3f42c93f 00007f7f3f42ec02 00007f7f45f52ef0
0709 13:09:32.870945 (+253212us) write_op.cc:301] APPLY: starting
0709 13:09:32.873588 (+  2643us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:33.087571 (+213983us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:33.087598 (+    27us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:33.090192 (+  2594us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:33.090385 (+   193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:09:33.090432 (+    47us) write_op.cc:312] APPLY: finished
0709 13:09:33.090681 (+   249us) log.cc:844] Serialized 49 byte log entry
0709 13:09:33.106800 (+ 16119us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:33.136046 (+ 29246us) write_op.cc:454] Released schema lock
0709 13:09:33.154896 (+ 18850us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:33.211939 (+ 57043us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:33.211970 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13014,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":335822,"apply.run_cpu_time_us":9238,"apply.run_wall_time_us":339689,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"num_ops":4,"prepare.queue_time_us":310457,"prepare.run_cpu_time_us":4538,"prepare.run_wall_time_us":81737,"replication_time_us":503848,"spinlock_wait_cycles":38623616,"thread_start_us":370030,"threads_started":2}]]}
I20260709 13:09:34.395716 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:34.455540 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 59903.7us
I20260709 13:09:34.456424 29146 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:34.356784 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:34.464789 29227 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:34.465917 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:34.466869 21792 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:09:34.328398 28847 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.141s
W20260709 13:09:34.469190 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:34.379983 29061 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.473027 29061 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Requested vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:34.327454 21187 leader_election.cc:336] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:34.473745 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:34.473929 21187 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.474591 21220 raft_consensus.cc:2393] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in current term 24: Already voted for candidate 46c1676121b64b7d9bcf2227cc639dd3 in this term.
I20260709 13:09:34.474814 28889 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:34.475801 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 148392us
W20260709 13:09:34.476498 21187 leader_election.cc:336] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:34.476898 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:34.486258 29208 raft_consensus.cc:515] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.488538 29208 leader_election.cc:290] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.489534 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:34.491758 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:34.492399 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:34.498340 21035 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.499402 29151 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:09:34.499415 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:34.491382 20965 catalog_manager.cc:5697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 14 to 15, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 15 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:09:34.505298 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 91876.3us
I20260709 13:09:34.527123 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:34.417140 29288 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:34.509402 29270 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:34.527931 21799 raft_consensus.cc:2368] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 24. Current term is 25.
I20260709 13:09:34.512199 21220 raft_consensus.cc:1275] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 13:09:34.529984 29049 consensus_queue.cc:1048] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:34.514219 29057 raft_consensus.cc:2955] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Committing config change with OpId 25.16: config changed from index 11 to 16, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:34.516465 21800 raft_consensus.cc:2955] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Committing config change with OpId 25.16: config changed from index 11 to 16, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:34.517405 29195 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:34.538846 19967 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 21097)
W20260709 13:09:34.549398 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:34.510315 21187 leader_election.cc:336] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:34.558702 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 61274.5us
I20260709 13:09:34.528935 20965 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 14 to -1, term changed from 23 to 12, leader changed from <none> to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195), VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) evicted. New cstate: current_term: 12 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:34.552096 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:34.564654 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 54366.3us
I20260709 13:09:34.567231 29288 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:34.570111 21220 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:09:34.571738 29049 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:09:34.590387 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:34.591074 21139 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:34.598431 29049 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.599107 29049 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.613831 21139 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 25.
I20260709 13:09:34.625020 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.625733 21215 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 21.
W20260709 13:09:34.626067 21113 leader_election.cc:336] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:34.627490 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:34.628358 29049 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.629338 29049 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:34.629810 29049 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.630221 29049 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:34.642889 29226 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.643693 29226 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.646374 29226 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:34.648319 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:34.649324 29267 log.cc:927] Time spent T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.012s
W20260709 13:09:34.651733 29299 log.cc:927] Time spent T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.050s	user 0.002s	sys 0.009s
I20260709 13:09:34.657894 21765 leader_election.cc:304] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:34.664351 21110 leader_election.cc:400] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 24. Current term is 25.
I20260709 13:09:34.664701 29208 raft_consensus.cc:2804] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:34.668112 28847 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.686264 28847 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.688475 28847 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:34.669536 29227 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.689126 29227 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.690228 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.691924 29260 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:09:34.694197 28847 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.694837 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:34.694792 28847 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.694082 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.691152 29227 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.697731 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.698163 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.700438 21036 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.702525 29227 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:34.685655 29146 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:09:34.685145 21110 leader_election.cc:403] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
W20260709 13:09:34.704241 21113 consensus_peers.cc:597] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:34.701112 28847 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.704748 28727 raft_consensus.cc:2749] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295) with higher term. Message: Invalid argument: T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e5e65640c8b4f5ca7eadc6a14312c55 for earlier term 24. Current term is 25.
I20260709 13:09:34.710717 21139 raft_consensus.cc:2955] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Committing config change with OpId 25.16: config changed from index 11 to 16, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:34.718236 29260 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.718904 29260 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.721385 29260 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.707777 29208 raft_consensus.cc:697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:34.723615 29208 consensus_queue.cc:237] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.735857 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.737942 28847 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:34.738751 28847 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:34.740856 28847 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:34.740993 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e3a58d24d64816bce2752c6adb7e33" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:34.741760 21212 raft_consensus.cc:3060] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:34.742897 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:34.746299 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.746929 29226 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.747494 29226 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.750079 21147 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:09:34.751366 21036 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.752776 29151 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:34.752921 29146 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:34.753729 29146 consensus_queue.cc:1243] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Peer 5e5e65640c8b4f5ca7eadc6a14312c55 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:09:34.753741 29226 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:34.757545 21113 consensus_peers.cc:597] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:34.759634 21212 raft_consensus.cc:2468] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 25.
I20260709 13:09:34.762195 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.764617 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.764797 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.766587 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.766978 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.768136 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.768489 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.768999 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.769269 21221 raft_consensus.cc:2410] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:09:34.769680 21214 raft_consensus.cc:2410] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:09:34.770074 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.766294 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.773684 21794 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:09:34.777143 21217 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Refusing update from remote peer 9384358ce43f4e3fac2ae44bdf11c389: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:09:34.777587 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.778187 21139 pending_rounds.cc:85] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting all ops after (but not including) 14
I20260709 13:09:34.778568 21139 pending_rounds.cc:107] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 13:09:34.778820 21139 raft_consensus.cc:2889] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:34.781867 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.793744 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.795343 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.798768 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.797293 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.803710 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.805943 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.789144 29238 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.811133 21036 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.811640 21220 raft_consensus.cc:1275] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 13:09:34.812739 29227 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:34.813593 29227 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.814137 29227 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.800876 29146 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:34.797363 28847 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:34.816416 29227 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.817910 29237 consensus_queue.cc:1048] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:34.825139 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.806967 29151 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:34.808447 29292 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:34.838788 29254 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.834115 29144 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:34.836525 28847 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:34.834122 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.844468 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.845541 29146 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:34.841336 29254 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.848733 29049 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.849665 29254 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.850576 21145 raft_consensus.cc:2393] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 22: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:34.846045 29227 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:34.807518 29236 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 13:09:34.821975 29238 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.861304 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:34.862218 21217 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:34.851344 29049 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:34.862792 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 74730.9us
W20260709 13:09:34.863446 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 61552.6us
W20260709 13:09:34.896032 29301 log.cc:927] Time spent T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:09:34.896806 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.897480 21220 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 23.
I20260709 13:09:34.898670 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:34.902175 28911 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:34.902653 28911 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.903007 28911 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:34.895215 29238 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:34.906467 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:34.911695 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:34.924803 29302 log.cc:927] Time spent T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.116s	user 0.003s	sys 0.000s
W20260709 13:09:34.925338 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:34.930333 21217 raft_consensus.cc:2468] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 22.
W20260709 13:09:34.935529 21113 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:34.936213 21113 consensus_peers.cc:597] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:34.934623 21187 leader_election.cc:336] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:34.941075 21113 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 244 similar messages]
I20260709 13:09:34.940305 29146 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.948227 29146 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.940552 29249 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.950574 29146 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:34.950472 29249 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.952771 29249 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:34.958837 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:34.940943 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.960052 29232 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:34.961301 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.941861 21111 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:34.964478 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.967199 29238 raft_consensus.cc:2804] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:34.967828 29238 raft_consensus.cc:697] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:34.968719 29238 consensus_queue.cc:237] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:34.969982 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.970928 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.971832 21220 raft_consensus.cc:2410] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:09:34.972725 29144 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:34.973230 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.974351 29144 raft_consensus.cc:3060] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:34.975255 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.974081 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:34.980187 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.982019 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.982903 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:34.985131 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:34.985162 21187 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:34.988895 21187 leader_election.cc:336] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:34.989305 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:34.947682 28911 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:34.998008 21113 leader_election.cc:336] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:34.996891 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:34.989825 29146 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:35.003518 29146 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:34.992182 29270 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:35.005470 21113 consensus_peers.cc:597] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:35.006143 21113 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:35.007467 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:35.017228 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:35.020116 21147 raft_consensus.cc:2393] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 in current term 24: Already voted for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in this term.
I20260709 13:09:35.019570 29292 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:09:35.020288 29155 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:35.039635 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:35.050153 29144 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:35.078850 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 49050.5us
I20260709 13:09:35.095865 29144 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.096444 29144 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:35.100543 29144 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:35.100744 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.102715 29144 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.103262 29144 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:35.104744 29292 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.105372 29144 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:35.105309 29292 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:35.107602 29292 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:35.109145 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:35.109509 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.112221 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:35.112982 21217 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:35.118657 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:35.120029 29292 raft_consensus.cc:3060] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:35.129382 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.131158 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.131892 21221 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:09:35.117807 28911 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:35.132862 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.135417 21217 raft_consensus.cc:2468] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 24.
I20260709 13:09:35.137676 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:35.138661 21111 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
W20260709 13:09:35.139379 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 63780.9us
I20260709 13:09:35.137353 29260 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:09:35.140733 29061 raft_consensus.cc:2804] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:35.141085 29146 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:35.141274 29061 raft_consensus.cc:697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:35.142259 29061 consensus_queue.cc:237] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:35.149662 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:35.152072 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:35.153985 29144 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:35.153805 29146 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:09:35.165733 29146 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:35.165887 29292 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:35.166493 29146 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:35.169468 29146 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:35.173321 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.194180 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.196136 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:35.197072 29144 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.197682 29144 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:35.199888 29144 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:35.200403 29144 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:35.201649 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:35.211809 21113 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:09:35.213128 21113 consensus_peers.cc:597] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:35.219600 21147 raft_consensus.cc:1275] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:09:35.224843 29305 consensus_queue.cc:1048] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:35.233155 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.240772 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:35.241991 29249 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:35.287762 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:35.355453 29144 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.356228 29144 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:35.359699 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.361994 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.374275 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:35.375229 29146 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:35.379370 29144 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:35.381352 21113 consensus_peers.cc:597] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:35.399750 29144 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.400372 29144 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:35.401688 20965 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 259) took 94449 ms (client timeout 15000 ms). Trace:
I20260709 13:09:35.405589 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:35.409077 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.409191 20966 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 13 to 15, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 15 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:35.412218 21217 raft_consensus.cc:1275] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:09:35.415061 29061 consensus_queue.cc:1048] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:35.420819 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:35.421859 29146 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:35.423804 29144 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:35.439860 29249 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.440445 29249 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:35.402164 20965 rpcz_store.cc:269] 0709 13:08:00.951857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.952817 (+   960us) service_pool.cc:224] Handling call
0709 13:09:35.401606 (+94448789us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:34.937716 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:34.938286 (+   570us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:34.938320 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:09:34.938340 (+    20us) tablet.cc:662] Decoding operations
0709 13:09:34.938761 (+   421us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:34.938838 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:34.938865 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:09:34.939191 (+   326us) tablet.cc:701] Row locks acquired
0709 13:09:34.939209 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:09:34.939407 (+   198us) write_op.cc:270] Start()
0709 13:09:34.939534 (+   127us) write_op.cc:276] Timestamp: P: 1783602574939358 usec, L: 0
0709 13:09:34.939561 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:09:34.987188 (+ 47627us) log.cc:844] Serialized 935 byte log entry
0709 13:09:35.191664 (+204476us) op_driver.cc:464] REPLICATION: finished
0709 13:09:35.243712 (+ 52048us) spinlock_profiling.cc:243] Waited 6.82 ms on lock 0x7b180017a520. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f483045b1 00007f7f3f42f3f3 00007f7f3f4317c1 00007f7f3f431779 00007f7f3f431709 00007f7f3f4316d1 00007f7f3f4309cd 000000000038f354 000000000038f199 00007f7f3f41283a
0709 13:09:35.243849 (+   137us) write_op.cc:301] APPLY: starting
0709 13:09:35.243930 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:35.245596 (+  1666us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:35.245627 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:35.254300 (+  8673us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:35.254475 (+   175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:09:35.254523 (+    48us) write_op.cc:312] APPLY: finished
0709 13:09:35.254736 (+   213us) log.cc:844] Serialized 33 byte log entry
0709 13:09:35.254903 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:35.254992 (+    89us) write_op.cc:454] Released schema lock
0709 13:09:35.255030 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:35.255499 (+   469us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:35.255531 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1534708,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":44358,"apply.run_cpu_time_us":4554,"apply.run_wall_time_us":11594,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":163710,"prepare.run_cpu_time_us":10684,"prepare.run_wall_time_us":109327,"raft.queue_time_us":101621,"raft.run_cpu_time_us":11496,"raft.run_wall_time_us":76774,"replication_time_us":251901,"spinlock_wait_cycles":15675264,"thread_start_us":288500,"threads_started":4,"wal-append.queue_time_us":62362}]]}
I20260709 13:09:35.442641 29249 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:35.447570 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.448925 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.467916 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:35.469249 29249 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:35.506664 29144 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.506789 20970 catalog_manager.cc:5697] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 18, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 18 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:35.509490 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:35.507360 29144 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:35.508724 20970 catalog_manager.cc:5697] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.513707 29144 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:35.515069 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.515069 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.516582 20970 catalog_manager.cc:5697] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.520424 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:35.527206 21113 consensus_peers.cc:597] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:35.532357 29146 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.531818 20970 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 21 to 22, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 22 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.532873 29146 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:35.533797 20970 catalog_manager.cc:5697] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 12 to 19, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 19 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.534951 29146 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:35.522974 29144 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:35.536613 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.537035 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.538314 20970 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 20 to 26, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 26 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:35.539337 19967 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 21097)
I20260709 13:09:35.540102 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:35.539901 20970 catalog_manager.cc:5697] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 22 to 23, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 23 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.541899 20970 catalog_manager.cc:5697] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 14 to 18, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:35.545879 20966 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 226) took 73501 ms (client timeout 15000 ms). Trace:
I20260709 13:09:35.543138 29144 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:35.546370 20966 rpcz_store.cc:269] 0709 13:08:22.044697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.061632 (+ 16935us) service_pool.cc:224] Handling call
0709 13:09:35.545766 (+73484134us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:35.453648 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:35.454223 (+   575us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:35.454258 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:09:35.454275 (+    17us) tablet.cc:662] Decoding operations
0709 13:09:35.454654 (+   379us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:35.454717 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:35.454748 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:09:35.454953 (+   205us) tablet.cc:701] Row locks acquired
0709 13:09:35.454968 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:09:35.455123 (+   155us) write_op.cc:270] Start()
0709 13:09:35.455244 (+   121us) write_op.cc:276] Timestamp: P: 1783602575455073 usec, L: 0
0709 13:09:35.455263 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:09:35.456028 (+   765us) log.cc:844] Serialized 544 byte log entry
0709 13:09:35.458225 (+  2197us) op_driver.cc:464] REPLICATION: finished
0709 13:09:35.458713 (+   488us) write_op.cc:301] APPLY: starting
0709 13:09:35.458793 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:35.459703 (+   910us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:35.459802 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:35.460539 (+   737us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:35.460702 (+   163us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:09:35.460754 (+    52us) write_op.cc:312] APPLY: finished
0709 13:09:35.460955 (+   201us) log.cc:844] Serialized 25 byte log entry
0709 13:09:35.461127 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:35.461209 (+    82us) write_op.cc:454] Released schema lock
0709 13:09:35.461249 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:35.461650 (+   401us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:35.461684 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9873,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":279,"apply.run_cpu_time_us":2884,"apply.run_wall_time_us":2884,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":117,"prepare.run_cpu_time_us":2671,"prepare.run_wall_time_us":2672,"replication_time_us":2795}]]}
I20260709 13:09:35.561695 21791 tablet_service.cc:1558] Processing DeleteTablet for tablet b7a0bb40f8e3428899935fdfa915dfd0 with delete_type TABLET_DATA_TOMBSTONED (TS b70fc04de37e4239a8336534f0c9c0fb not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:33646
I20260709 13:09:35.543547 20970 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 22, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 22 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:35.571161 20970 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 20 to 21, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 21 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.584010 20970 catalog_manager.cc:5697] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 19 to 21, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 21 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:35.587827 20970 catalog_manager.cc:5697] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.590373 20970 catalog_manager.cc:5697] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 14 to 21, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 21 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.598279 20970 catalog_manager.cc:5697] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:35.605469 20970 catalog_manager.cc:5697] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 13 to 19, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 19 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:35.618858 20970 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index -1 to 14, term changed from 12 to 25, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196), VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 25 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.621280 20970 catalog_manager.cc:5697] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 16 to 18, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 18 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.622481 29057 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.623067 29057 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:35.622949 20970 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 21 to 23, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 23 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.624562 20970 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:35.625910 20970 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:35.627125 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:35.627761 21794 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 27.
I20260709 13:09:35.627904 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.628603 21147 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 27.
I20260709 13:09:35.629066 21184 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:35.629876 29315 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:09:35.630251 29315 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.630597 29315 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:35.633342 29323 tablet_replica.cc:333] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:09:35.634403 29323 raft_consensus.cc:2243] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:09:35.626132 29057 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:35.646557 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:35.658809 29323 raft_consensus.cc:2272] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:35.658633 29315 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:35.665632 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:35.666249 21147 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:35.673193 21147 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 28.
I20260709 13:09:35.676700 21185 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:35.682751 29315 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:35.685068 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce55364f10c348819e7c63c08633b184" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:35.685724 21794 raft_consensus.cc:3060] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:35.691422 29057 raft_consensus.cc:2804] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 13:09:35.691947 21113 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:35.692011 29057 raft_consensus.cc:697] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:35.693030 29057 consensus_queue.cc:237] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:35.703896 29315 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.713361 29315 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:35.718916 21187 leader_election.cc:336] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:35.719453 29323 ts_tablet_manager.cc:1916] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:35.724313 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:35.725081 21800 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:35.726990 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:35.729138 29195 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:35.729535 29195 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.729874 29195 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:35.731431 21794 raft_consensus.cc:2468] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 28.
I20260709 13:09:35.744979 29315 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:35.751919 21217 raft_consensus.cc:1275] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:09:35.754254 29249 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:35.754858 29249 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:35.757133 29249 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:35.760771 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:35.762837 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.764449 29146 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.765069 29146 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:35.772387 28911 consensus_queue.cc:1048] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:09:35.775494 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:35.776741 29146 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:35.778365 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:35.832566 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.836328 21036 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:35.839447 29144 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.839850 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:35.839953 29144 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:35.842422 29144 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:35.819007 29195 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:35.838965 29146 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:35.876442 29195 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:35.882151 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:35.860255 29144 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:35.899523 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 59748.1us
I20260709 13:09:35.910017 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:35.913147 21794 raft_consensus.cc:3060] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:35.923456 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:35.923461 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:35.978681 21794 raft_consensus.cc:2468] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:09:35.981963 21184 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:35.995713 29146 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:36.001037 29144 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:36.027612 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 51470.5us
I20260709 13:09:36.018626 29327 raft_consensus.cc:2804] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:36.034072 29332 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.048134 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:36.028339 29146 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:36.058965 21113 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:36.032101 29144 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:36.059384 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:36.042764 29327 raft_consensus.cc:697] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:36.072551 29334 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:36.104341 29327 consensus_queue.cc:237] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:36.113965 21794 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 13:09:36.115803 29324 log.cc:927] Time spent T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: Append to log took a long time: real 0.239s	user 0.002s	sys 0.004s
I20260709 13:09:36.049474 29332 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:36.129945 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 72427.3us
W20260709 13:09:36.133531 21187 leader_election.cc:336] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:36.134531 29332 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:36.138338 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:36.147183 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:36.152719 29144 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:36.128719 29326 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.019s
I20260709 13:09:36.164953 29146 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:36.170065 21035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 43915us
I20260709 13:09:36.173853 20970 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 171) took 3644 ms. Trace:
I20260709 13:09:36.178721 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:36.180032 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:36.174770 20970 rpcz_store.cc:276] 0709 13:09:32.527779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.456393 (+1928614us) service_pool.cc:224] Handling call
0709 13:09:36.172268 (+1715875us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:35.663248 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:35.663795 (+   547us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:35.663829 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:09:35.663848 (+    19us) tablet.cc:662] Decoding operations
0709 13:09:35.666362 (+  2514us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:35.666426 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:35.666457 (+    31us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:09:35.667976 (+  1519us) tablet.cc:701] Row locks acquired
0709 13:09:35.667992 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:09:35.668147 (+   155us) write_op.cc:270] Start()
0709 13:09:35.668270 (+   123us) write_op.cc:276] Timestamp: P: 1783602575668098 usec, L: 0
0709 13:09:35.668293 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:09:35.668974 (+   681us) log.cc:844] Serialized 8062 byte log entry
0709 13:09:35.685362 (+ 16388us) op_driver.cc:464] REPLICATION: finished
0709 13:09:35.685816 (+   454us) write_op.cc:301] APPLY: starting
0709 13:09:35.685892 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:35.730239 (+ 44347us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:35.730333 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:35.873845 (+143512us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:35.874181 (+   336us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:09:35.874234 (+    53us) write_op.cc:312] APPLY: finished
0709 13:09:35.874655 (+   421us) log.cc:844] Serialized 180 byte log entry
0709 13:09:35.874823 (+   168us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:35.874991 (+   168us) write_op.cc:454] Released schema lock
0709 13:09:35.875048 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:35.896913 (+ 21865us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:35.909519 (+ 12606us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46214,"reported_tablets":28,"spinlock_wait_cycles":4608,"child_traces":[["op",{"apply.queue_time_us":255,"apply.run_cpu_time_us":25785,"apply.run_wall_time_us":201391,"cfile_cache_hit":80,"cfile_cache_hit_bytes":83640,"num_ops":20,"prepare.queue_time_us":154,"prepare.run_cpu_time_us":5431,"prepare.run_wall_time_us":6043,"replication_time_us":16930}]]}
I20260709 13:09:36.184901 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:36.187557 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:36.190086 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:36.191391 20968 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 16 to 24. New cstate: current_term: 24 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.195120 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:36.196902 29344 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:36.183746 29331 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.205021 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.216391 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.209399 20968 catalog_manager.cc:5697] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 12 to 14, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 14 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:36.223728 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:36.225462 29146 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:36.213424 29331 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:36.228798 29323 ts_tablet_manager.cc:1929] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 25.16
I20260709 13:09:36.229410 29323 log.cc:1199] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Deleting WAL directory at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0/minicluster-data/ts-3-root/wals/b7a0bb40f8e3428899935fdfa915dfd0
I20260709 13:09:36.230072 29331 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:36.236140 29344 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.236680 29344 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:36.238881 29344 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:36.241434 21036 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:36.243188 29342 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:36.244691 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.245163 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.253162 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:36.254011 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:36.254386 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:36.254675 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:36.261636 29331 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.264343 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:36.266068 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:36.268064 21113 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:36.275977 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 71973.6us
I20260709 13:09:36.274469 20968 catalog_manager.cc:5697] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 15 to 16, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 16 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.289835 20954 catalog_manager.cc:5028] TS b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): tablet b7a0bb40f8e3428899935fdfa915dfd0 (table test-workload [id=707c24a2500c4eab9b729d6de4e01f10]) successfully deleted
I20260709 13:09:36.273097 29342 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:36.277393 29146 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:09:36.301302 21145 raft_consensus.cc:1275] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 13:09:36.291510 20968 catalog_manager.cc:5697] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 15 to 25, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 25 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.310359 20968 catalog_manager.cc:5697] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 14 to 20, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 20 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.265673 29331 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:36.327409 29328 consensus_queue.cc:1048] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 13:09:36.379127 29146 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.379822 29146 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:36.382644 29146 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:36.389156 29331 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:36.397905 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:36.411188 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 90186us
I20260709 13:09:36.418375 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:36.424939 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.425287 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:36.425792 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 87090.5us
I20260709 13:09:36.329895 20968 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 15 to 20, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 20 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:36.439517 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:36.453433 21036 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:36.457820 29332 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:36.439877 29249 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.464174 29144 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.472728 29330 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:09:36.500195 20968 catalog_manager.cc:5697] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.502724 29340 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.502369 20968 catalog_manager.cc:5697] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 18 to 22, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 22 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:36.495837 29144 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:36.508478 29144 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:36.503515 29340 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:36.513273 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:36.479542 29249 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:36.539830 19967 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 21097)
W20260709 13:09:36.549949 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:36.557231 29249 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:36.559338 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:36.559980 21794 raft_consensus.cc:1803] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: voting while tombstoned based on last-logged opid 25.16
I20260709 13:09:36.560566 21794 raft_consensus.cc:2410] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9384358ce43f4e3fac2ae44bdf11c389 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
W20260709 13:09:36.567436 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:36.581379 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:36.587189 21036 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:36.601737 29344 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:36.603511 29340 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:36.587492 20968 catalog_manager.cc:5697] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:36.634294 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:36.640966 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.641326 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5d4dd9fad84e018bc7168755b0ba6b" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.642633 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.649230 29333 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:36.649818 29333 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:36.652848 29333 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:36.656090 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:36.656672 21794 raft_consensus.cc:1803] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: voting while tombstoned based on last-logged opid 25.16
I20260709 13:09:36.657064 21794 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 25.
W20260709 13:09:36.657122 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:36.659086 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.668021 21036 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:36.669224 29340 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:36.674566 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:36.677100 29249 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:36.629935 20968 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 18 to 21, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 21 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.691643 20968 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 14 to 15, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 15 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.697124 20968 catalog_manager.cc:5697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 14 to 18, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:36.716575 21794 raft_consensus.cc:1275] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:09:36.718202 29333 consensus_queue.cc:1048] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:36.730608 20968 catalog_manager.cc:5697] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.732832 20968 catalog_manager.cc:5697] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:36.734025 20968 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 17 to 18, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:09:36.740361 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:36.754726 21187 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:36.755224 21187 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:36.756073 29333 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:36.760007 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:36.768707 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:36.769373 20968 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 14 to 16, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 16 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:36.777665 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:36.777814 20968 catalog_manager.cc:5697] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 16 to 18, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:09:36.779934 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:36.779762 20968 catalog_manager.cc:5697] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.781510 20968 catalog_manager.cc:5697] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 13 to 17, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 17 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:09:36.778370 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:36.833518 29340 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.834124 29340 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:36.834568 29344 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.835115 29344 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:36.837323 29344 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:36.839416 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:36.840627 29340 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:36.842985 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:36.845438 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.848964 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:36.850600 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:36.851975 29344 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:09:36.855104 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:36.858129 29249 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:36.899887 29081 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:36.900565 29081 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:36.904269 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:36.904937 21221 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:36.907042 29081 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:36.909415 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:36.910123 21794 raft_consensus.cc:1803] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: voting while tombstoned based on last-logged opid 25.16
I20260709 13:09:36.910540 21794 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:36.911914 21110 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:36.912932 29354 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:36.913396 29354 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:36.913793 29354 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:36.945212 21113 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:36.958967 29354 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:36.962364 29354 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:36.963014 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:36.963688 21220 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:36.965137 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:36.965696 21794 raft_consensus.cc:1803] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: voting while tombstoned based on last-logged opid 25.16
I20260709 13:09:36.966012 21794 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:36.967417 21113 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:36.980327 21220 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
W20260709 13:09:36.982426 21113 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:09:36.998318 21794 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
I20260709 13:09:37.011026 21110 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:37.013491 29354 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:37.014039 29354 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:37.014919 29354 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:37.022369 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:37.026175 29340 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.026845 29340 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:37.040438 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:37.038226 29340 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:37.046694 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524f5c1a9e540c9946dd77d8f971ef1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:37.048724 21036 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:37.050144 29340 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:09:37.095567 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:37.108125 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:37.128713 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:37.137557 29249 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.138267 29249 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:37.142405 29249 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:37.144104 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:37.146162 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de27148f295b4ef5a3f79084b8ed5ae6" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:37.151335 21036 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:37.152521 29249 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:37.199656 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 64705.6us
I20260709 13:09:37.232733 29249 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.233275 29249 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:37.235869 29249 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:37.237349 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:37.239899 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:37.244704 21036 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:37.245826 29249 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:37.247676 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:37.346517 29249 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.348459 29249 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:37.388351 29249 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:37.422031 29344 raft_consensus.cc:493] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.422966 29344 raft_consensus.cc:515] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:37.441174 29372 raft_consensus.cc:493] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.442153 29372 raft_consensus.cc:515] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:37.445499 29372 leader_election.cc:290] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:37.479856 29372 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.480665 29372 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:37.482815 29344 leader_election.cc:290] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:37.483170 29372 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:37.483741 29369 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.486792 29369 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:37.488687 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:37.490893 29369 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:37.491061 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:37.491747 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 58442.1us
I20260709 13:09:37.512662 21220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:37.514973 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:37.522876 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:37.529471 21113 consensus_peers.cc:597] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:37.530274 21113 consensus_peers.cc:597] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:37.530956 21113 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:37.531550 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:37.540587 19967 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 21097)
I20260709 13:09:37.542178 29344 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.543087 29344 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:37.546183 29344 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:37.546645 20968 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 262) took 23214 ms (client timeout 15000 ms). Trace:
I20260709 13:09:37.548702 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:37.548177 20961 catalog_manager.cc:5697] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 16 to 21, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 21 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:09:37.553352 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:37.556042 20961 catalog_manager.cc:5697] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:37.576270 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080f4010b39e40a8b8efb7280faec38f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:37.577953 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa2ded676fc4d048c810a5f842100d1" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:37.578348 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a87917a1b44746a96a84470b77ea16" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:37.580679 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:37.547133 20968 rpcz_store.cc:269] 0709 13:09:14.331857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.334551 (+  2694us) service_pool.cc:224] Handling call
0709 13:09:37.546290 (+23211739us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:36.825609 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:36.826238 (+   629us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:36.826276 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:09:36.826296 (+    20us) tablet.cc:662] Decoding operations
0709 13:09:36.828660 (+  2364us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:36.828752 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:36.828786 (+    34us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:09:36.830586 (+  1800us) tablet.cc:701] Row locks acquired
0709 13:09:36.830601 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:09:36.830752 (+   151us) write_op.cc:270] Start()
0709 13:09:36.830869 (+   117us) write_op.cc:276] Timestamp: P: 1783602576830699 usec, L: 0
0709 13:09:36.830890 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:09:36.862644 (+ 31754us) log.cc:844] Serialized 7583 byte log entry
0709 13:09:36.972839 (+110195us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b540039d5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42485571 00007f7f42517e28 00007f7f45f52581 00007f7f45f51d58 00007f7f45f569f1 00007f7f45f569a9 00007f7f45f56939 00007f7f45f56901 00007f7f45f55b2d 000000000038f354 000000000038f199
0709 13:09:36.991901 (+ 19062us) op_driver.cc:464] REPLICATION: finished
0709 13:09:37.039962 (+ 48061us) write_op.cc:301] APPLY: starting
0709 13:09:37.040058 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:37.169176 (+129118us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:37.169205 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:37.223220 (+ 54015us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:37.234311 (+ 11091us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:09:37.234380 (+    69us) write_op.cc:312] APPLY: finished
0709 13:09:37.234874 (+   494us) log.cc:844] Serialized 172 byte log entry
0709 13:09:37.236390 (+  1516us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:37.240622 (+  4232us) write_op.cc:454] Released schema lock
0709 13:09:37.240691 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:37.241166 (+   475us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:37.241201 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1776485,"reported_tablets":49,"spinlock_wait_cycles":20736,"child_traces":[["op",{"apply.queue_time_us":42311,"apply.run_cpu_time_us":26979,"apply.run_wall_time_us":201182,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83585,"num_ops":19,"prepare.queue_time_us":13128,"prepare.run_cpu_time_us":8291,"prepare.run_wall_time_us":147430,"replication_time_us":160820,"spinlock_wait_cycles":71312640,"thread_start_us":67778,"threads_started":2}]]}
I20260709 13:09:37.602834 21036 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:37.612784 29344 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:37.614065 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 52704.1us
I20260709 13:09:37.623831 21215 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:09:37.625715 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:37.634223 21036 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:37.637755 21036 leader_election.cc:304] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:37.641377 21036 leader_election.cc:304] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:37.642127 29372 raft_consensus.cc:2749] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:37.642252 29344 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:37.642709 29369 raft_consensus.cc:2749] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:37.646890 20961 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 reported cstate change: term changed from 13 to 16, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 16 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:09:37.649538 21110 consensus_peers.cc:597] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:37.661442 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:37.664152 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:37.667519 29344 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:37.671777 21036 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:37.683495 29344 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:37.698534 29081 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:09:37.796247 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:37.797581 21113 consensus_peers.cc:597] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:37.799891 21113 consensus_peers.cc:597] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:37.802244 21113 consensus_peers.cc:597] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:37.811470 21113 consensus_peers.cc:597] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.001451 20970 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 22 to 23, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 23 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:38.041359 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:38.052129 29372 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.052829 29372 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:38.054134 29344 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.055042 29372 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:38.059069 29372 raft_consensus.cc:493] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.059814 29372 raft_consensus.cc:515] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:38.068675 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 246654us
I20260709 13:09:38.054759 29344 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:38.132751 29344 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:38.116411 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.136021 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:38.117367 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.121042 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.122402 20970 catalog_manager.cc:5697] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 21, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 21 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:38.141425 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.144464 21036 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.146839 29249 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:38.147600 29249 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.148204 29249 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:38.150635 29249 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:38.155694 29369 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.156384 29369 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:38.158833 29369 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:38.159668 29372 leader_election.cc:290] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:38.160379 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.168025 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beb666b5d504215a141add24a6d632f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.170506 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:38.169150 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:38.177697 21036 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.177801 21218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffee3ff6d914b9a8bf36e54cc2da28d" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.178020 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0de4eb0c8e649f09be36671e6dcc4cb" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:38.183043 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:38.184327 29344 raft_consensus.cc:493] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.184479 29249 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:38.185086 29344 raft_consensus.cc:515] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:38.189249 29344 leader_election.cc:290] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:38.196051 21036 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:38.197371 21036 leader_election.cc:304] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:38.208369 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.208493 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80e14318ee44feba007795498031464" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:38.211002 29372 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:38.212394 29344 raft_consensus.cc:2749] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:38.216689 21036 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.249495 29344 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:38.255967 21036 leader_election.cc:304] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:38.255453 20970 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 21 to 23, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 23 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:38.261909 29369 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.262161 29344 raft_consensus.cc:2749] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:38.262514 29369 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:38.263801 29381 ts_tablet_manager.cc:933] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:38.264915 29369 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:38.277832 20970 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 24 to 26, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 26 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:38.279944 20970 catalog_manager.cc:5697] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.281791 20970 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:38.283789 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.285221 20970 catalog_manager.cc:5697] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 24 to 28, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 28 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:38.290004 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.298960 29249 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.299744 29249 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:38.301940 29249 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:38.307646 20970 catalog_manager.cc:5697] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.309852 20970 catalog_manager.cc:5697] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 11 to 16, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 25 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:38.317032 20970 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 17 to 19, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 19 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:38.319902 20970 catalog_manager.cc:5697] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:38.318647 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:38.289515 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:38.322088 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a16f225ecc04514a485518bcd2b8112" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.322366 29249 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:38.322474 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.323170 29249 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:38.325502 29249 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:38.327219 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:38.332437 21036 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.333515 29249 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:38.334281 20961 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 228) took 39457 ms (client timeout 15000 ms). Trace:
I20260709 13:09:38.335227 29249 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.335961 29249 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:38.337447 29381 tablet_copy_client.cc:323] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.194:39351
I20260709 13:09:38.338644 29249 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:38.340632 21157 tablet_copy_service.cc:140] P 5e5e65640c8b4f5ca7eadc6a14312c55: Received BeginTabletCopySession request for tablet b7a0bb40f8e3428899935fdfa915dfd0 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:50996)
I20260709 13:09:38.341621 21157 tablet_copy_service.cc:161] P 5e5e65640c8b4f5ca7eadc6a14312c55: Beginning new tablet copy session on tablet b7a0bb40f8e3428899935fdfa915dfd0 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:50996: session id = b70fc04de37e4239a8336534f0c9c0fb-b7a0bb40f8e3428899935fdfa915dfd0
I20260709 13:09:38.340679 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.354946 29249 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.356065 29249 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:38.359375 29249 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:38.361668 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:38.334795 20961 rpcz_store.cc:269] 0709 13:08:58.877295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.877548 (+   253us) service_pool.cc:224] Handling call
0709 13:09:00.031526 (+1153978us) spinlock_profiling.cc:243] Waited 69.5 ms on lock 0x7b540039d5d4. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f42486090 00007f7f42524f3a 00007f7f482db1cc 00007f7f483d937e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc 00007f7f46899392 00007f7f4085748c 00007f7f408568c6
0709 13:09:38.334184 (+38302658us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:37.698349 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:37.699181 (+   832us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:37.699236 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:09:37.699261 (+    25us) tablet.cc:662] Decoding operations
0709 13:09:37.700841 (+  1580us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:37.700939 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:37.700973 (+    34us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:09:37.701592 (+   619us) tablet.cc:701] Row locks acquired
0709 13:09:37.701647 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:09:37.701848 (+   201us) write_op.cc:270] Start()
0709 13:09:37.702002 (+   154us) write_op.cc:276] Timestamp: P: 1783602577701773 usec, L: 0
0709 13:09:37.702024 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:09:37.702924 (+   900us) log.cc:844] Serialized 1326 byte log entry
0709 13:09:37.738582 (+ 35658us) op_driver.cc:464] REPLICATION: finished
0709 13:09:37.740459 (+  1877us) write_op.cc:301] APPLY: starting
0709 13:09:37.740550 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:37.743393 (+  2843us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:37.743524 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:37.745450 (+  1926us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:37.745697 (+   247us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:09:37.745745 (+    48us) write_op.cc:312] APPLY: finished
0709 13:09:37.746087 (+   342us) log.cc:844] Serialized 41 byte log entry
0709 13:09:37.746946 (+   859us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:37.747054 (+   108us) write_op.cc:454] Released schema lock
0709 13:09:37.747124 (+    70us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":2161212,"reported_tablets":50,"spinlock_wait_cycles":159758208,"child_traces":[["op",{"apply.queue_time_us":180,"apply.run_cpu_time_us":6457,"apply.run_wall_time_us":7109,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12546,"num_ops":3,"prepare.queue_time_us":30992,"prepare.run_cpu_time_us":4344,"prepare.run_wall_time_us":5032,"replication_time_us":36395,"thread_start_us":32132,"threads_started":1}]]}
I20260709 13:09:38.375298 29249 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.375963 29249 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:38.378961 29249 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:38.380095 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.386646 21157 tablet_copy_source_session.cc:215] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:38.393671 29381 ts_tablet_manager.cc:1916] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260709 13:09:38.419412 21036 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.420567 29249 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:38.430414 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.430802 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.432099 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:38.432531 21113 consensus_peers.cc:597] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.433355 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:38.435370 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:38.446321 29381 ts_tablet_manager.cc:1929] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 25.16
I20260709 13:09:38.455968 21036 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:38.457275 21036 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.457707 29381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a0bb40f8e3428899935fdfa915dfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:38.458554 21036 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:38.459777 29372 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:09:38.460503 29372 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:38.460569 21113 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:38.461330 29372 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:38.466148 21036 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.468328 29383 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:09:38.474550 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:38.480100 29381 tablet_copy_client.cc:806] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:38.480844 29381 tablet_copy_client.cc:670] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:38.486786 29381 tablet_copy_client.cc:538] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:38.511363 29383 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:38.511914 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.512001 29383 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:38.514560 29383 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:38.517496 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.518308 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.520967 21036 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.522665 29383 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:09:38.539033 29381 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
W20260709 13:09:38.542156 19967 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 21097)
W20260709 13:09:38.546108 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.564188 20970 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 172) took 2087 ms. Trace:
I20260709 13:09:38.576855 29383 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.577636 29383 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:38.580118 29383 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:38.584384 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.564692 20970 rpcz_store.cc:276] 0709 13:09:36.477165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.477337 (+   172us) service_pool.cc:224] Handling call
0709 13:09:38.564099 (+2086762us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:38.421700 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:38.425719 (+  4019us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:38.425766 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:09:38.425786 (+    20us) tablet.cc:662] Decoding operations
0709 13:09:38.426970 (+  1184us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:38.436436 (+  9466us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:38.436479 (+    43us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:09:38.438294 (+  1815us) tablet.cc:701] Row locks acquired
0709 13:09:38.438318 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:09:38.438484 (+   166us) write_op.cc:270] Start()
0709 13:09:38.438610 (+   126us) write_op.cc:276] Timestamp: P: 1783602578438435 usec, L: 0
0709 13:09:38.438633 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:09:38.439477 (+   844us) log.cc:844] Serialized 4464 byte log entry
0709 13:09:38.459666 (+ 20189us) op_driver.cc:464] REPLICATION: finished
0709 13:09:38.494242 (+ 34576us) write_op.cc:301] APPLY: starting
0709 13:09:38.494338 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:38.516834 (+ 22496us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:38.516939 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:38.547389 (+ 30450us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:38.547673 (+   284us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:09:38.547714 (+    41us) write_op.cc:312] APPLY: finished
0709 13:09:38.548076 (+   362us) log.cc:844] Serialized 105 byte log entry
0709 13:09:38.548901 (+   825us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:38.549050 (+   149us) write_op.cc:454] Released schema lock
0709 13:09:38.549111 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:38.553161 (+  4050us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:38.553197 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":441885,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":32634,"apply.run_cpu_time_us":16001,"apply.run_wall_time_us":56809,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50129,"num_ops":11,"prepare.queue_time_us":81174,"prepare.run_cpu_time_us":4980,"prepare.run_wall_time_us":24191,"replication_time_us":15680,"thread_start_us":113771,"threads_started":2}]]}
I20260709 13:09:38.592676 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:38.594830 21035 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:38.605736 29383 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:38.603960 20966 catalog_manager.cc:5697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 18 to 22, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 22 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:38.611394 20966 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 15 to 17, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 17 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:38.626505 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:38.635644 20966 catalog_manager.cc:5697] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.638020 20966 catalog_manager.cc:5697] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 21 to 27. New cstate: current_term: 27 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.639876 20966 catalog_manager.cc:5697] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 25 to 30, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 30 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.644017 20966 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 18 to 25, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 25 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:38.650460 20966 catalog_manager.cc:5697] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 20 to 23, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 23 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.660051 20966 catalog_manager.cc:5697] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 19 to 22, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 22 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:38.661645 20966 catalog_manager.cc:5697] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 16 to 19, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 19 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.664237 20966 catalog_manager.cc:5697] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 21 to 24, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 24 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.666163 20966 catalog_manager.cc:5697] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 17 to 25, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 25 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.672562 20966 catalog_manager.cc:5697] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.688545 20966 catalog_manager.cc:5697] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 18 to 20, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 20 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
W20260709 13:09:38.691555 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.692788 20966 catalog_manager.cc:5697] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 19 to 27. New cstate: current_term: 27 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.694701 20966 catalog_manager.cc:5697] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 18 to 20, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 20 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.696795 20966 catalog_manager.cc:5697] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 17 to 19, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 19 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:38.698299 20966 catalog_manager.cc:5697] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:38.699898 20966 catalog_manager.cc:5697] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:38.701198 20966 catalog_manager.cc:5697] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.706954 29249 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.707654 29249 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:09:38.708439 20966 catalog_manager.cc:5697] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 19 to 24, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 24 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:38.710076 29249 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:38.710407 20966 catalog_manager.cc:5697] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 19 to 24, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 24 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:38.712338 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.712853 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47de8d16a3804364a7fbc9bf598946c2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.712757 20966 catalog_manager.cc:5697] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 19 to 20, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 20 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.714504 21036 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.719995 20966 catalog_manager.cc:5697] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.721376 29383 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:38.722081 20966 catalog_manager.cc:5697] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:38.718626 21113 consensus_peers.cc:597] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.723676 20966 catalog_manager.cc:5697] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.735577 20966 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 16 to 21, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 21 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:38.737216 20966 catalog_manager.cc:5697] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 23 to 28, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 28 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:38.738983 20966 catalog_manager.cc:5697] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 16 to 19, leader changed from 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195) to 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193). New cstate: current_term: 19 leader_uuid: "9384358ce43f4e3fac2ae44bdf11c389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } }
I20260709 13:09:38.798686 29381 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
W20260709 13:09:38.810875 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.820259 29249 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.820945 29249 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:38.823257 29249 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:38.824257 29381 tablet_bootstrap.cc:492] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:09:38.825166 29381 ts_tablet_manager.cc:1403] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.286s	user 0.045s	sys 0.019s
I20260709 13:09:38.825632 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:38.828661 29381 raft_consensus.cc:359] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:38.829842 29381 raft_consensus.cc:740] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: FOLLOWER
I20260709 13:09:38.830463 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d1baf8579949fbb0d5d45a18fefe35" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:38.832011 21036 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:38.835685 29383 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:38.846694 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.830771 29381 consensus_queue.cc:260] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:38.862151 29381 ts_tablet_manager.cc:1434] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:09:38.864167 21157 tablet_copy_service.cc:342] P 5e5e65640c8b4f5ca7eadc6a14312c55: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b7a0bb40f8e3428899935fdfa915dfd0 received from {username='slave'} at 127.0.0.1:50996
I20260709 13:09:38.864696 21157 tablet_copy_service.cc:434] P 5e5e65640c8b4f5ca7eadc6a14312c55: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-b7a0bb40f8e3428899935fdfa915dfd0 on tablet b7a0bb40f8e3428899935fdfa915dfd0 with peer b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:38.873263 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800126ba0 after lost signal to thread 19970
W20260709 13:09:38.874605 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08003369c0 after lost signal to thread 21004
W20260709 13:09:38.875406 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800338700 after lost signal to thread 21096
W20260709 13:09:38.875720 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08002ef380 after lost signal to thread 21097
W20260709 13:09:38.876935 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800034f00 after lost signal to thread 21171
W20260709 13:09:38.877247 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800034f20 after lost signal to thread 21172
W20260709 13:09:38.878472 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08000395e0 after lost signal to thread 21245
W20260709 13:09:38.878778 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08000395a0 after lost signal to thread 21246
W20260709 13:09:38.880172 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800190880 after lost signal to thread 21825
W20260709 13:09:38.892742 21113 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:38.935969 20970 catalog_manager.cc:5697] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 21 to 23, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 23 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:38.989408 29249 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:38.990075 29249 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:39.002285 29249 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:39.002382 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:39.004593 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:39.007336 21035 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:39.008690 29249 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:39.010560 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:39.014798 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:39.025689 29249 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:39.026448 29249 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:39.028864 29249 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:39.029922 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:39.030486 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:39.032882 21036 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:39.033815 29249 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:39.037642 20966 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 247) took 3308 ms. Trace:
I20260709 13:09:39.038081 20966 rpcz_store.cc:276] 0709 13:09:35.728847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.729034 (+   187us) service_pool.cc:224] Handling call
0709 13:09:35.740666 (+ 11632us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b5400002490. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7e9c 00007f7f3f3f80f6 00007f7f48371a89 00007f7f48309da6 00007f7f482db088 00007f7f483d937e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc
0709 13:09:39.037538 (+3296872us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:38.763612 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:38.764191 (+   579us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:38.764232 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:09:38.764253 (+    21us) tablet.cc:662] Decoding operations
0709 13:09:38.766422 (+  2169us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:38.766504 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:38.766536 (+    32us) tablet.cc:685] Acquiring locks for 28 operations
0709 13:09:38.768874 (+  2338us) tablet.cc:701] Row locks acquired
0709 13:09:38.768894 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:09:38.769032 (+   138us) write_op.cc:270] Start()
0709 13:09:38.769152 (+   120us) write_op.cc:276] Timestamp: P: 1783602578768990 usec, L: 0
0709 13:09:38.769865 (+   713us) op_driver.cc:348] REPLICATION: starting
0709 13:09:38.770635 (+   770us) log.cc:844] Serialized 11098 byte log entry
0709 13:09:38.773824 (+  3189us) op_driver.cc:464] REPLICATION: finished
0709 13:09:38.774192 (+   368us) write_op.cc:301] APPLY: starting
0709 13:09:38.774271 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:38.838322 (+ 64051us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:38.838422 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:38.915101 (+ 76679us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:38.924449 (+  9348us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=56,mrs_lookups=0
0709 13:09:38.924511 (+    62us) write_op.cc:312] APPLY: finished
0709 13:09:38.925048 (+   537us) log.cc:844] Serialized 244 byte log entry
0709 13:09:38.931473 (+  6425us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:38.931708 (+   235us) write_op.cc:454] Released schema lock
0709 13:09:38.931778 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:38.935990 (+  4212us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:38.936028 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17181,"reported_tablets":51,"spinlock_wait_cycles":4607488,"child_traces":[["op",{"apply.queue_time_us":118,"apply.run_cpu_time_us":36102,"apply.run_wall_time_us":161742,"cfile_cache_hit":112,"cfile_cache_hit_bytes":117096,"num_ops":28,"prepare.queue_time_us":4525,"prepare.run_cpu_time_us":6604,"prepare.run_wall_time_us":10672,"replication_time_us":3809}]]}
W20260709 13:09:39.085671 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:39.132916 21792 raft_consensus.cc:1217] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Deduplicated request from leader. Original: 25.16->[26.17-26.17]   Dedup: 26.17->[]
W20260709 13:09:39.187206 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:39.269232 20969 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 260) took 73993 ms (client timeout 15000 ms). Trace:
W20260709 13:09:39.269754 20969 rpcz_store.cc:269] 0709 13:08:25.275537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.277474 (+  1937us) service_pool.cc:224] Handling call
0709 13:09:39.269010 (+73991536us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40281,"reported_tablets":49}
W20260709 13:09:39.271162 21113 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:39.318270 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 96624.8us
I20260709 13:09:39.333873 29249 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.354347 20966 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 25 to 26, leader changed from b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 26 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:39.370959 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:39.391690 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:39.357965 29249 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:39.420130 29249 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:39.420941 29249 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.421720 29249 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:39.428279 21212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:39.405754 29394 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.433840 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:39.440241 21036 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:39.441804 29383 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:39.448410 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:39.458962 29394 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:39.535698 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 75541.6us
W20260709 13:09:39.542689 19967 thread.cc:527] Waited for 10000ms trying to join with heartbeat (tid 21097)
I20260709 13:09:39.555509 29393 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.563052 29393 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:39.571143 29249 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:39.573010 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:39.573920 29393 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:39.578715 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:39.588560 21144 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:39.612414 20962 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 245) took 46780 ms (client timeout 15000 ms). Trace:
W20260709 13:09:39.627825 20962 rpcz_store.cc:269] 0709 13:08:52.832337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.835480 (+  3143us) service_pool.cc:224] Handling call
0709 13:09:39.612179 (+46776699us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24593,"reported_tablets":51}
I20260709 13:09:39.618686 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:39.620374 29393 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.637552 29393 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:39.615038 29394 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:39.654367 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 115492us
W20260709 13:09:39.677369 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:39.662366 29393 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:39.682031 21792 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:09:39.689893 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:39.716300 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:39.744923 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 113461us
W20260709 13:09:39.744755 21110 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 53121.2us
I20260709 13:09:39.748924 29405 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:09:39.749562 21036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 56237.2us
W20260709 13:09:39.755714 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 123839us
W20260709 13:09:39.766923 20963 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55748 (request call id 229) took 11945 ms (client timeout 15000 ms). Trace:
W20260709 13:09:39.767452 20963 rpcz_store.cc:269] 0709 13:09:27.821891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.224698 (+4402807us) service_pool.cc:224] Handling call
0709 13:09:39.766718 (+7542020us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4790,"reported_tablets":50}
I20260709 13:09:39.773545 29406 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Committing config change with OpId 26.18: config changed from index 14 to 18, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:39.781855 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:39.781989 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fc5a3f4db84eac831870d621ddfa54" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:39.788775 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:39.790158 21219 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:09:39.808375 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20ca56847f64504a85694d10f47558e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:39.810246 21800 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Committing config change with OpId 26.18: config changed from index 14 to 18, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:39.817170 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de53f43e2a0e4a91a1767df521fc2b8c" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:39.822391 21036 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:39.825402 29416 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:39.826804 21036 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:39.828873 29416 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:39.837585 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:39.839439 19967 tablet_replica.cc:333] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:39.841375 19967 raft_consensus.cc:2243] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:39.842213 19967 raft_consensus.cc:2272] T 3d158af381004432b3f3f21e53fd19ac P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:39.845057 29404 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:09:39.847893 19967 tablet_replica.cc:333] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:39.851558 21036 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:39.857874 19967 raft_consensus.cc:2243] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:39.863265 19967 raft_consensus.cc:2272] T 2875dc050a244109a0287cc6dafa8431 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:39.858764 29416 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:39.854378 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b7a0bb40f8e3428899935fdfa915dfd0 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:39.858147 21036 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:39.877503 29394 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:39.874699 19967 tablet_replica.cc:333] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:39.879354 19967 raft_consensus.cc:2243] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:39.883842 19967 raft_consensus.cc:2272] T d524f5c1a9e540c9946dd77d8f971ef1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:39.889217 29394 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.889842 29394 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:39.890450 20962 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 14 to 18, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 26 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:39.892026 29394 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:39.903553 21221 raft_consensus.cc:2955] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Committing config change with OpId 26.18: config changed from index 14 to 18, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:39.897187 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:39.913552 19967 tablet_replica.cc:333] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:39.921924 19967 raft_consensus.cc:2243] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:09:39.931083 19967 raft_consensus.cc:2272] T b4ee2463d8894a95be9b747f813f9106 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:39.939512 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:39.941640 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:39.942401 21113 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 394 similar messages]
I20260709 13:09:39.942685 19967 tablet_replica.cc:333] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:39.948010 19967 raft_consensus.cc:2243] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:39.950526 19967 pending_rounds.cc:70] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:39.952076 19967 pending_rounds.cc:77] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 24 index: 15 } timestamp: 7305636044626956288 op_type: NO_OP noop_request { }
I20260709 13:09:39.952602 19967 raft_consensus.cc:2889] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:39.953027 19967 raft_consensus.cc:2272] T 2e5425aa9a9a4fba958ae97f007977d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:39.956459 19967 tablet_replica.cc:333] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:39.961700 19967 raft_consensus.cc:2243] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:39.968493 19967 pending_rounds.cc:70] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 2 pending ops.
I20260709 13:09:39.968878 19967 pending_rounds.cc:77] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 19 index: 9 } timestamp: 7305636087040258048 op_type: NO_OP noop_request { }
I20260709 13:09:39.969513 19967 raft_consensus.cc:2889] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:39.969895 19967 pending_rounds.cc:77] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 17 index: 8 } timestamp: 7305636024143634432 op_type: NO_OP noop_request { }
I20260709 13:09:39.970331 19967 raft_consensus.cc:2889] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:39.971647 19967 raft_consensus.cc:2272] T 79a87917a1b44746a96a84470b77ea16 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:39.973268 21113 consensus_peers.cc:597] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:39.975198 20955 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8b6dd6862dd14baba4a82c5a837bc802 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 5e5e65640c8b4f5ca7eadc6a14312c55 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:39.975841 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:39.976799 21113 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:39.977212 19967 tablet_replica.cc:333] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:39.979081 21036 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:39.988382 29394 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:39.988588 19967 raft_consensus.cc:2243] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:39.990486 19967 raft_consensus.cc:2272] T d4a635811aa248b599c9361b7abf9f32 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:39.991304 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:39.993760 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:39.995677 29394 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:39.996245 20967 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 261) took 51295 ms (client timeout 15000 ms). Trace:
W20260709 13:09:39.996671 20967 rpcz_store.cc:269] 0709 13:08:48.700458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.700690 (+   232us) service_pool.cc:224] Handling call
0709 13:09:39.996032 (+51295342us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1003,"reported_tablets":49}
I20260709 13:09:39.996268 29394 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:40.001245 29394 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:40.004175 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:40.007356 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223e82a164f445c2b46667d58a5d2d1f" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:40.011494 21036 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:40.012737 29394 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:40.016855 19967 tablet_replica.cc:333] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.038587 19967 raft_consensus.cc:2243] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.039309 19967 pending_rounds.cc:70] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 2 pending ops.
I20260709 13:09:40.039569 19967 pending_rounds.cc:77] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 24 index: 15 } timestamp: 7305636107691233280 op_type: NO_OP noop_request { }
I20260709 13:09:40.039921 19967 raft_consensus.cc:2889] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.040215 19967 pending_rounds.cc:77] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 22 index: 14 } timestamp: 7305636047328514048 op_type: NO_OP noop_request { }
I20260709 13:09:40.040541 19967 raft_consensus.cc:2889] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.040848 19967 raft_consensus.cc:2272] T 6beb666b5d504215a141add24a6d632f P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.049953 19967 tablet_replica.cc:333] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.051043 19967 raft_consensus.cc:2243] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.051833 19967 pending_rounds.cc:70] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.052100 19967 pending_rounds.cc:77] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 22 index: 12 } timestamp: 7305636004369416192 op_type: NO_OP noop_request { }
I20260709 13:09:40.052487 19967 raft_consensus.cc:2889] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.052840 19967 raft_consensus.cc:2272] T 9f10495397de45019165a00ba7c4ec29 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.061657 29394 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.062347 29394 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:40.065007 29394 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:40.072926 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:40.082499 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:40.084712 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:40.084969 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae1ecaeba734906b89cfdc36c6abaf9" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:40.086619 19967 tablet_replica.cc:333] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.087821 19967 raft_consensus.cc:2243] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.088474 19967 pending_rounds.cc:70] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.088728 19967 pending_rounds.cc:77] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 18 index: 14 } timestamp: 7305636103206576128 op_type: NO_OP noop_request { }
I20260709 13:09:40.089097 19967 raft_consensus.cc:2889] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.092568 19967 raft_consensus.cc:2272] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.093941 21036 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:40.098383 29416 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:40.101142 19967 tablet_replica.cc:333] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.105505 19967 raft_consensus.cc:2243] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.108506 19967 pending_rounds.cc:70] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.108893 19967 pending_rounds.cc:77] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 20 index: 14 } timestamp: 7305636056534392832 op_type: NO_OP noop_request { }
I20260709 13:09:40.109433 19967 raft_consensus.cc:2889] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.114006 19967 raft_consensus.cc:2272] T e30c07cef4334b45ade426b26069b26c P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.129343 19967 tablet_replica.cc:333] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
W20260709 13:09:40.147194 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:40.151662 19967 raft_consensus.cc:2243] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.152479 19967 raft_consensus.cc:2272] T a0de4eb0c8e649f09be36671e6dcc4cb P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:40.168649 21113 consensus_peers.cc:597] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:40.176635 19967 tablet_replica.cc:333] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.177285 20965 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55780 (request call id 263) took 6781 ms. Trace:
I20260709 13:09:40.178023 20965 rpcz_store.cc:276] 0709 13:09:33.396167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.487568 (+2091401us) service_pool.cc:224] Handling call
0709 13:09:40.177093 (+4689525us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":49}
I20260709 13:09:40.187991 19967 raft_consensus.cc:2243] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.188891 19967 pending_rounds.cc:70] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.189198 19967 pending_rounds.cc:77] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 24 index: 12 } timestamp: 7305636104103915520 op_type: NO_OP noop_request { }
I20260709 13:09:40.189625 19967 raft_consensus.cc:2889] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.190025 19967 raft_consensus.cc:2272] T 72edcb8f7f3d437b8e7bb24c612600ab P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.198426 19967 tablet_replica.cc:333] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.208138 29394 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:40.209527 29394 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:40.211742 29394 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:40.212545 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:40.212826 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:40.214491 19967 raft_consensus.cc:2243] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.215137 21035 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:40.215463 19967 raft_consensus.cc:2272] T 752b0641d0a3410795a788ef8dc8e035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.216414 29394 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:40.219781 19967 tablet_replica.cc:333] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
W20260709 13:09:40.228780 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet b7a0bb40f8e3428899935fdfa915dfd0: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:09:40.235337 19967 raft_consensus.cc:2243] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.240038 19967 pending_rounds.cc:70] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.240387 19967 pending_rounds.cc:77] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 29 index: 16 } timestamp: 7305636139722530816 op_type: NO_OP noop_request { }
I20260709 13:09:40.241565 19967 raft_consensus.cc:2889] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.241889 19967 raft_consensus.cc:2272] T d4af1b94141049e0a78a816c8ba31d72 P 9384358ce43f4e3fac2ae44bdf11c389 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.250931 19967 tablet_replica.cc:333] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.260031 19967 raft_consensus.cc:2243] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.261080 19967 raft_consensus.cc:2272] T 223e82a164f445c2b46667d58a5d2d1f P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:40.266729 20964 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55764 (request call id 246) took 19538 ms (client timeout 15000 ms). Trace:
W20260709 13:09:40.267220 20964 rpcz_store.cc:269] 0709 13:09:20.728010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:28.210967 (+7482957us) service_pool.cc:224] Handling call
0709 13:09:28.557632 (+346665us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b5400002438. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f482f7de1 00007f7f482db173 00007f7f483d937e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc 00007f7f46899392 00007f7f4085748c 00007f7f408568c6 00007f7f4085626f
0709 13:09:40.265849 (+11708217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23223,"reported_tablets":51,"spinlock_wait_cycles":42406272}
W20260709 13:09:40.272718 21113 consensus_peers.cc:597] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:40.281644 19967 tablet_replica.cc:333] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.282699 19967 raft_consensus.cc:2243] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.283547 19967 raft_consensus.cc:2272] T 77143ea923a34396a4cbc31f42dd0a18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.295519 29416 raft_consensus.cc:493] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.296196 29416 raft_consensus.cc:515] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:40.303829 19967 tablet_replica.cc:333] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.309736 29416 leader_election.cc:290] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:40.311820 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:40.312846 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0631ac443bb4e6dbbbc599dd4b67d2e" candidate_uuid: "9384358ce43f4e3fac2ae44bdf11c389" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:40.320169 19967 raft_consensus.cc:2243] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:09:40.321362 21035 leader_election.cc:304] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9384358ce43f4e3fac2ae44bdf11c389; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:40.323961 29416 raft_consensus.cc:2749] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:40.330744 19967 raft_consensus.cc:2272] T 0d78b35ac92d402db8a5e3fe45be52b5 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.377246 19967 tablet_replica.cc:333] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.381014 19967 raft_consensus.cc:2243] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.389623 19967 raft_consensus.cc:2272] T b8d00e740dba47b2b43be9e4ef1ecb55 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:40.392287 21113 consensus_peers.cc:597] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:40.399945 19967 tablet_replica.cc:333] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.400979 19967 raft_consensus.cc:2243] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.411160 19967 raft_consensus.cc:2272] T c20ca56847f64504a85694d10f47558e P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:40.411427 21113 consensus_peers.cc:597] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:40.436973 21113 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:40.437282 19967 tablet_replica.cc:333] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.441051 19967 raft_consensus.cc:2243] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.442001 19967 raft_consensus.cc:2272] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.444967 19967 tablet_replica.cc:333] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.447068 19967 raft_consensus.cc:2243] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.451249 19967 pending_rounds.cc:70] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.452003 19967 pending_rounds.cc:77] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 17 index: 10 } timestamp: 7305636011532132352 op_type: NO_OP noop_request { }
I20260709 13:09:40.457976 19967 raft_consensus.cc:2889] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.459882 19967 raft_consensus.cc:2272] T 080f4010b39e40a8b8efb7280faec38f P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:40.470870 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:40.476269 19967 tablet_replica.cc:333] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.478228 19967 raft_consensus.cc:2243] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.491945 19967 pending_rounds.cc:70] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 2 pending ops.
I20260709 13:09:40.492247 19967 pending_rounds.cc:77] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 22 index: 14 } timestamp: 7305636113450303488 op_type: NO_OP noop_request { }
I20260709 13:09:40.492641 19967 raft_consensus.cc:2889] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260709 13:09:40.494619 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:40.495654 19967 pending_rounds.cc:77] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 21 index: 13 } timestamp: 7305636078423265280 op_type: NO_OP noop_request { }
I20260709 13:09:40.496101 19967 raft_consensus.cc:2889] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.496498 19967 raft_consensus.cc:2272] T de53f43e2a0e4a91a1767df521fc2b8c P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.504675 19967 tablet_replica.cc:333] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
W20260709 13:09:40.506197 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:40.514005 19967 raft_consensus.cc:2243] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:09:40.515525 19967 raft_consensus.cc:2272] T 731bbc776723436d954fe0b2a30a5b09 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.527475 19967 tablet_replica.cc:333] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.528443 19967 raft_consensus.cc:2243] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.529368 19967 raft_consensus.cc:2272] T 1ea0f23227964b96bff6287436e970cd P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.547389 19967 tablet_replica.cc:333] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.550369 19967 raft_consensus.cc:2243] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.551246 19967 raft_consensus.cc:2272] T 40d1baf8579949fbb0d5d45a18fefe35 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.566418 19967 tablet_replica.cc:333] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.567582 19967 raft_consensus.cc:2243] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.568341 19967 raft_consensus.cc:2272] T 5a16f225ecc04514a485518bcd2b8112 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.571446 19967 tablet_replica.cc:333] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.572404 19967 raft_consensus.cc:2243] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.573340 19967 raft_consensus.cc:2272] T e0631ac443bb4e6dbbbc599dd4b67d2e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.577129 19967 tablet_replica.cc:333] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.585079 19967 raft_consensus.cc:2243] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:09:40.586412 19967 raft_consensus.cc:2272] T 169e564bd720422bb798c652af4fb2eb P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.589835 19967 tablet_replica.cc:333] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.600037 19967 raft_consensus.cc:2243] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.600963 19967 raft_consensus.cc:2272] T 6e4456f3a9364c7289dfce3163ab4760 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.604082 19967 tablet_replica.cc:333] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.605010 19967 raft_consensus.cc:2243] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.606104 19967 pending_rounds.cc:70] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.606402 19967 pending_rounds.cc:77] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 18 index: 12 } timestamp: 7305636076983947264 op_type: NO_OP noop_request { }
I20260709 13:09:40.606807 19967 raft_consensus.cc:2889] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.607234 19967 raft_consensus.cc:2272] T 8b6dd6862dd14baba4a82c5a837bc802 P 9384358ce43f4e3fac2ae44bdf11c389 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.616883 19967 tablet_replica.cc:333] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.617832 19967 raft_consensus.cc:2243] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:09:40.618896 19967 raft_consensus.cc:2272] T e714d9095b9c430bba199e0209c0b946 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.622196 19967 tablet_replica.cc:333] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.625425 19967 raft_consensus.cc:2243] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.635989 19967 raft_consensus.cc:2272] T de27148f295b4ef5a3f79084b8ed5ae6 P 9384358ce43f4e3fac2ae44bdf11c389 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.639345 19967 tablet_replica.cc:333] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.640215 19967 raft_consensus.cc:2243] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.640863 19967 pending_rounds.cc:70] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.641194 19967 pending_rounds.cc:77] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 19 index: 10 } timestamp: 7305636063518265344 op_type: NO_OP noop_request { }
I20260709 13:09:40.641638 19967 raft_consensus.cc:2889] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.642055 19967 raft_consensus.cc:2272] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 9384358ce43f4e3fac2ae44bdf11c389 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.644970 19967 tablet_replica.cc:333] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.645901 19967 raft_consensus.cc:2243] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:09:40.648331 19967 raft_consensus.cc:2272] T 3ca4319420e54bc19f621da03c811035 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:40.649119 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:40.662403 19967 tablet_replica.cc:333] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.663443 19967 raft_consensus.cc:2243] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:09:40.664712 19967 raft_consensus.cc:2272] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.667764 19967 tablet_replica.cc:333] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.668939 19967 raft_consensus.cc:2243] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.671792 19967 raft_consensus.cc:2272] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.674986 19967 tablet_replica.cc:333] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.676261 19967 raft_consensus.cc:2243] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.677013 19967 raft_consensus.cc:2272] T 5196cb9fb3c1488180a6421083b18ff9 P 9384358ce43f4e3fac2ae44bdf11c389 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.680074 19967 tablet_replica.cc:333] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.680954 19967 raft_consensus.cc:2243] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.681715 19967 raft_consensus.cc:2272] T cc53d52fdeaa441b9e9d329e69d94440 P 9384358ce43f4e3fac2ae44bdf11c389 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.684551 19967 tablet_replica.cc:333] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.685384 19967 raft_consensus.cc:2243] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.686038 19967 raft_consensus.cc:2272] T a473266451804da089da06e9c927e7ee P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.689584 19967 tablet_replica.cc:333] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.699973 19967 raft_consensus.cc:2243] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.700908 19967 pending_rounds.cc:70] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.701412 19967 pending_rounds.cc:77] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 18 index: 10 } timestamp: 7305636082624212992 op_type: NO_OP noop_request { }
I20260709 13:09:40.701846 19967 raft_consensus.cc:2889] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.702261 19967 raft_consensus.cc:2272] T 47de8d16a3804364a7fbc9bf598946c2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.705348 19967 tablet_replica.cc:333] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.706226 19967 raft_consensus.cc:2243] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.707499 19967 pending_rounds.cc:70] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.710180 19967 pending_rounds.cc:77] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 23 index: 15 } timestamp: 7305636067052355584 op_type: NO_OP noop_request { }
I20260709 13:09:40.710660 19967 raft_consensus.cc:2889] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.711082 19967 raft_consensus.cc:2272] T b7a0bb40f8e3428899935fdfa915dfd0 P 9384358ce43f4e3fac2ae44bdf11c389 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.713898 19967 tablet_replica.cc:333] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.720714 19967 raft_consensus.cc:2243] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.721462 19967 raft_consensus.cc:2272] T 1fa2ded676fc4d048c810a5f842100d1 P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.724272 19967 tablet_replica.cc:333] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.726085 19967 raft_consensus.cc:2243] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.736646 19967 raft_consensus.cc:2272] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.740024 19967 tablet_replica.cc:333] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.740810 19967 raft_consensus.cc:2243] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.741441 19967 raft_consensus.cc:2272] T 41268ef73d2c487dab473150fa26d73e P 9384358ce43f4e3fac2ae44bdf11c389 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.744557 19967 tablet_replica.cc:333] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.745350 19967 raft_consensus.cc:2243] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.751443 19967 raft_consensus.cc:2272] T 54fc5a3f4db84eac831870d621ddfa54 P 9384358ce43f4e3fac2ae44bdf11c389 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:40.748261 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:40.754907 19967 tablet_replica.cc:333] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.755748 19967 raft_consensus.cc:2243] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.767828 19967 raft_consensus.cc:2272] T 9417f68ba15d49c6915ab46793aab93c P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.771118 19967 tablet_replica.cc:333] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.772010 19967 raft_consensus.cc:2243] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.772846 19967 raft_consensus.cc:2272] T c80e14318ee44feba007795498031464 P 9384358ce43f4e3fac2ae44bdf11c389 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:40.775847 19967 tablet_replica.cc:333] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.776608 19967 raft_consensus.cc:2243] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:40.777225 19967 pending_rounds.cc:70] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Trying to abort 1 pending ops.
I20260709 13:09:40.777506 19967 pending_rounds.cc:77] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389: Aborting op as it isn't in flight: id { term: 23 index: 15 } timestamp: 7305636084259397632 op_type: NO_OP noop_request { }
I20260709 13:09:40.778015 19967 raft_consensus.cc:2889] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:40.778432 19967 raft_consensus.cc:2272] T 6b5d4dd9fad84e018bc7168755b0ba6b P 9384358ce43f4e3fac2ae44bdf11c389 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:40.780504 21113 consensus_peers.cc:597] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:40.781273 19967 tablet_replica.cc:333] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389: stopping tablet replica
I20260709 13:09:40.787786 19967 raft_consensus.cc:2243] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:09:40.789078 19967 raft_consensus.cc:2272] T 9833b15c99c34c269ca3afa7dac3f175 P 9384358ce43f4e3fac2ae44bdf11c389 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:40.858361 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:40.903460 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:40.996104 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:41.032380 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:41.108596 21113 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:41.199041 21113 consensus_peers.cc:597] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:41.214922 20955 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8b6dd6862dd14baba4a82c5a837bc802 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 5e5e65640c8b4f5ca7eadc6a14312c55 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:41.221503 21113 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:41.266300 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:41.315055 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:41.319928 29427 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.320570 29427 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:41.323526 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:41.328342 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:41.335270 21113 leader_election.cc:336] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.342419 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:41.343755 29339 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:41.326583 29427 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:41.371349 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:41.384835 29415 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.386404 29413 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.387159 29413 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:41.390590 29415 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:41.393396 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:41.392057 29413 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:41.398010 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:41.398367 29415 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:41.404350 21767 leader_election.cc:336] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.398658 21145 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
W20260709 13:09:41.399324 21187 leader_election.cc:336] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.407835 21187 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:41.408540 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:41.409219 29413 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:41.412775 29415 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:41.413261 29415 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.413699 29415 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:41.432776 21215 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:41.435889 29415 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:41.438555 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet b7a0bb40f8e3428899935fdfa915dfd0: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.453567 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:41.454192 21145 raft_consensus.cc:3060] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:41.454276 21767 leader_election.cc:336] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.454988 29415 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:41.466614 21184 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41268ef73d2c487dab473150fa26d73e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:41.466609 21139 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:09:41.468657 29325 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:09:41.481230 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:41.475538 21145 raft_consensus.cc:2468] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 21.
I20260709 13:09:41.489363 21765 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:41.490298 29415 raft_consensus.cc:2804] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:41.490911 29415 raft_consensus.cc:697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:41.491914 29415 consensus_queue.cc:237] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:41.510696 29325 raft_consensus.cc:2955] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Committing config change with OpId 24.14: config changed from index -1 to 14, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:41.516893 21139 raft_consensus.cc:2955] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Committing config change with OpId 24.14: config changed from index -1 to 14, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:41.530555 20964 catalog_manager.cc:5697] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 20 to 21, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 21 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:41.543311 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:41.556689 21215 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:41.551100 20963 catalog_manager.cc:5697] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 14, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 24 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:41.571230 29415 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.572009 29415 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:41.578963 29415 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:41.583388 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:41.584928 21767 leader_election.cc:336] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:41.587626 21184 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a16f225ecc04514a485518bcd2b8112. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:41.590912 21139 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:09:41.592854 29413 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:41.597903 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:41.603588 29415 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:41.655110 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:41.661237 29395 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.661939 29395 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:41.663782 29395 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:41.669741 21767 leader_election.cc:336] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:41.674255 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 57079.9us
I20260709 13:09:41.684772 29436 raft_consensus.cc:2955] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:41.687552 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:41.688284 21219 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 19.
I20260709 13:09:41.691222 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:41.692022 29440 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:41.692413 29440 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.693840 29440 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:41.694327 21139 raft_consensus.cc:2955] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:41.711244 21145 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:41.723624 29440 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:41.732478 29413 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.733068 29413 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:41.740496 21219 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:41.741644 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:41.742169 29339 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:09:41.745239 21187 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.746516 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:41.747691 29413 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:41.748468 29436 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:41.749971 21110 consensus_peers.cc:597] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295): Couldn't send request to peer b70fc04de37e4239a8336534f0c9c0fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5196cb9fb3c1488180a6421083b18ff9. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:41.777622 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:41.796509 29440 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:41.804735 21767 leader_election.cc:336] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.814260 21215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:41.816074 21215 raft_consensus.cc:3060] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:41.814390 29436 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.822263 29436 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:41.825186 29436 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:41.826184 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:41.826887 21792 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:09:41.829455 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:41.838418 29436 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:41.839004 29436 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.839504 29436 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:41.849599 29339 raft_consensus.cc:2955] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:41.855301 20965 catalog_manager.cc:5697] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index -1 to 13, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 22 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:41.863606 21219 raft_consensus.cc:2955] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:41.860948 21215 raft_consensus.cc:2468] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 20.
I20260709 13:09:41.864347 29436 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:41.867118 29436 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:41.869343 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:41.869972 21792 raft_consensus.cc:3060] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:41.871757 21018 auto_rebalancer.cc:629] Failed to schedule move for tablet 731bbc776723436d954fe0b2a30a5b09: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.879767 29427 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.880398 29427 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:41.882639 29427 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:41.884632 29325 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.885247 29325 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:41.888922 29436 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.889523 29436 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:41.892851 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:41.894838 29325 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:41.896520 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:41.903744 29436 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:41.905138 21792 raft_consensus.cc:2468] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:41.906738 21184 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:41.908111 29436 raft_consensus.cc:2804] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:41.908995 29436 raft_consensus.cc:697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:41.909951 29436 consensus_queue.cc:237] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:41.921936 21187 leader_election.cc:336] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:41.937328 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 44332us
I20260709 13:09:41.951195 21145 raft_consensus.cc:1275] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:09:41.960589 21765 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:41.961473 29440 raft_consensus.cc:2804] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:41.962041 29440 raft_consensus.cc:697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:41.963019 29440 consensus_queue.cc:237] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:41.975313 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 96807us
I20260709 13:09:41.978432 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:41.973915 20965 catalog_manager.cc:5697] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 23 to 24, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 24 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:41.980634 29427 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:41.981194 29427 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:41.983242 29427 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:41.986958 29415 consensus_queue.cc:1048] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:09:41.992746 21187 leader_election.cc:336] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:41.993598 21187 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.994024 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:41.994922 21187 leader_election.cc:336] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:41.995344 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:41.996068 29413 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:41.996739 29413 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:42.019665 20965 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index -1 to 12, NON_VOTER b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) added. New cstate: current_term: 17 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } } }
I20260709 13:09:42.030229 29452 ts_tablet_manager.cc:933] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:42.033764 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 50811.5us
I20260709 13:09:42.037683 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:42.038322 21221 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 25.
I20260709 13:09:42.042121 29450 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.042768 29450 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:42.055449 29450 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:42.055192 20963 catalog_manager.cc:5697] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 19 to 20, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 20 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:42.064627 29395 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.048704 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:42.065269 29395 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:42.066146 29450 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:42.066545 29450 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.066890 29450 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:42.068234 19967 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 21093)
I20260709 13:09:42.068835 29395 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:42.072652 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:42.073424 21214 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 22.
I20260709 13:09:42.074584 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:42.075353 29395 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:42.075767 29395 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.076126 29395 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:42.085896 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:42.086575 21221 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 19.
I20260709 13:09:42.096508 29450 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:42.096905 21767 leader_election.cc:336] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.100003 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:42.100590 21214 raft_consensus.cc:3060] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:42.102311 21767 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:42.102932 29450 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:42.107910 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:42.109571 29427 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:42.109961 29427 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.110338 29427 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:42.110522 29447 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.111548 21214 raft_consensus.cc:2468] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 26.
I20260709 13:09:42.111104 29447 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:42.113859 29447 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:42.113976 29395 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:42.116706 29413 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.117219 29413 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:42.119679 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:42.119231 29413 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:42.120388 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:42.121768 21145 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 21.
I20260709 13:09:42.122903 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:42.123684 29413 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:42.124048 29413 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.124408 29413 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:42.138365 21187 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.146986 21111 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:42.153656 29339 raft_consensus.cc:2804] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:42.162918 29395 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:42.164274 21767 leader_election.cc:336] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.158849 29452 tablet_copy_client.cc:323] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:09:42.135092 29427 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:42.191115 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:42.191821 21214 raft_consensus.cc:3060] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:42.206983 29457 ts_tablet_manager.cc:933] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:42.209594 29339 raft_consensus.cc:697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
I20260709 13:09:42.216378 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 41268ef73d2c487dab473150fa26d73e from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:09:42.218297 21214 raft_consensus.cc:2468] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:42.225579 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 41268ef73d2c487dab473150fa26d73e from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-41268ef73d2c487dab473150fa26d73e
I20260709 13:09:42.217841 29413 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:42.223102 29339 consensus_queue.cc:237] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:42.229174 21765 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:42.228044 29441 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.248782 29462 ts_tablet_manager.cc:933] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: Initiating tablet copy from peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:42.239966 29395 raft_consensus.cc:2804] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:42.261519 29457 tablet_copy_client.cc:323] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.195:46249
I20260709 13:09:42.276290 29427 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:42.279266 21231 tablet_copy_source_session.cc:215] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:42.279434 29395 raft_consensus.cc:697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:42.317371 29395 consensus_queue.cc:237] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:42.268807 29441 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:42.290939 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:42.321103 29441 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:42.324337 29452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41268ef73d2c487dab473150fa26d73e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:42.295990 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:09:42.301573 29413 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:42.308717 20955 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8b6dd6862dd14baba4a82c5a837bc802 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 5e5e65640c8b4f5ca7eadc6a14312c55 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:42.338882 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:42.346674 21794 raft_consensus.cc:1275] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:09:42.348528 29413 consensus_queue.cc:1048] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:09:42.346613 21187 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.355852 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:42.356463 21145 raft_consensus.cc:3060] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:42.342981 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 92256.5us
I20260709 13:09:42.361946 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca4319420e54bc19f621da03c811035" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:42.362689 21214 raft_consensus.cc:3060] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:42.372208 21231 tablet_copy_service.cc:140] P 46c1676121b64b7d9bcf2227cc639dd3: Received BeginTabletCopySession request for tablet 5a16f225ecc04514a485518bcd2b8112 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:54620)
I20260709 13:09:42.372784 21231 tablet_copy_service.cc:161] P 46c1676121b64b7d9bcf2227cc639dd3: Beginning new tablet copy session on tablet 5a16f225ecc04514a485518bcd2b8112 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:54620: session id = b70fc04de37e4239a8336534f0c9c0fb-5a16f225ecc04514a485518bcd2b8112
I20260709 13:09:42.382436 21231 tablet_copy_source_session.cc:215] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:42.383000 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:42.383728 21139 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 26.
I20260709 13:09:42.389777 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:42.395344 21145 raft_consensus.cc:2468] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 22.
I20260709 13:09:42.398253 21185 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:42.408685 21214 raft_consensus.cc:2468] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5e65640c8b4f5ca7eadc6a14312c55 in term 20.
I20260709 13:09:42.416929 29415 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:42.417593 29415 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9384358ce43f4e3fac2ae44bdf11c389)
I20260709 13:09:42.413050 21111 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 
I20260709 13:09:42.417816 20963 catalog_manager.cc:5697] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 25 to 26, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 26 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:42.413259 29462 tablet_copy_client.cc:323] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.127.194:39351
I20260709 13:09:42.428587 29471 raft_consensus.cc:2804] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:42.429235 29471 raft_consensus.cc:697] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:42.430516 29466 raft_consensus.cc:2804] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:09:42.437606 21767 leader_election.cc:336] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.430115 29471 consensus_queue.cc:237] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:42.451422 29466 raft_consensus.cc:697] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Becoming Leader. State: Replica: 5e5e65640c8b4f5ca7eadc6a14312c55, State: Running, Role: LEADER
W20260709 13:09:42.455863 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:42.426311 29415 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:42.455641 29466 consensus_queue.cc:237] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:42.470899 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 68868.4us
W20260709 13:09:42.495040 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 105341us
I20260709 13:09:42.512362 21157 tablet_copy_service.cc:140] P 5e5e65640c8b4f5ca7eadc6a14312c55: Received BeginTabletCopySession request for tablet 5196cb9fb3c1488180a6421083b18ff9 from peer b70fc04de37e4239a8336534f0c9c0fb ({username='slave'} at 127.0.0.1:50996)
I20260709 13:09:42.529968 21157 tablet_copy_service.cc:161] P 5e5e65640c8b4f5ca7eadc6a14312c55: Beginning new tablet copy session on tablet 5196cb9fb3c1488180a6421083b18ff9 from peer b70fc04de37e4239a8336534f0c9c0fb at {username='slave'} at 127.0.0.1:50996: session id = b70fc04de37e4239a8336534f0c9c0fb-5196cb9fb3c1488180a6421083b18ff9
I20260709 13:09:42.533484 29415 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:42.535763 29457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a16f225ecc04514a485518bcd2b8112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:42.513737 29452 tablet_copy_client.cc:806] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:42.550139 29452 tablet_copy_client.cc:670] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:42.550655 20963 catalog_manager.cc:5697] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 21 to 22, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 22 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:42.588065 29452 tablet_copy_client.cc:538] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:42.592296 21215 raft_consensus.cc:1275] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:09:42.603510 21187 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:42.617367 21113 consensus_peers.cc:597] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:42.619833 21113 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.620332 21113 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:42.630577 29461 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:42.632836 21113 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.610082 29441 consensus_queue.cc:1048] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
W20260709 13:09:42.634092 21113 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:42.637298 21113 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:42.637974 21113 leader_election.cc:336] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.638326 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:42.639003 29450 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:09:42.640218 21113 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.621662 20964 catalog_manager.cc:5697] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 22 to 23, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 23 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:42.651765 21767 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:42.659842 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 67189us
W20260709 13:09:42.660463 29475 log.cc:927] Time spent T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:09:42.657121 21157 tablet_copy_source_session.cc:215] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:42.662719 29415 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 election: Requested vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:42.663733 20963 catalog_manager.cc:5697] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: term changed from 19 to 20, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194). New cstate: current_term: 20 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:42.665795 21767 leader_election.cc:336] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.670444 29452 tablet_bootstrap.cc:492] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
W20260709 13:09:42.683893 20957 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64008.4us
I20260709 13:09:42.690327 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55"
I20260709 13:09:42.691089 21145 raft_consensus.cc:3060] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:42.705744 21145 raft_consensus.cc:2468] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 27.
I20260709 13:09:42.706034 29462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5196cb9fb3c1488180a6421083b18ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:42.707247 21765 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb; no voters: 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:42.708452 29415 raft_consensus.cc:2804] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:09:42.708994 29415 raft_consensus.cc:697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 27 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:42.709882 29415 consensus_queue.cc:237] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:42.710824 29457 tablet_copy_client.cc:806] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:42.711755 29457 tablet_copy_client.cc:670] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
W20260709 13:09:42.721064 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet b7a0bb40f8e3428899935fdfa915dfd0: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:42.725212 29457 tablet_copy_client.cc:538] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:09:42.767889 21113 consensus_peers.cc:597] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:42.793866 29462 tablet_copy_client.cc:806] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:42.794636 29462 tablet_copy_client.cc:670] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Starting download of 1 WAL segments...
W20260709 13:09:42.808948 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:42.810536 29462 tablet_copy_client.cc:538] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:42.812458 21221 raft_consensus.cc:1275] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:09:42.818071 21215 raft_consensus.cc:1275] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:09:42.819330 29476 consensus_queue.cc:1048] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:09:42.821375 29466 consensus_queue.cc:1048] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:09:42.825990 21767 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:42.828824 21113 consensus_peers.cc:597] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:42.804729 29457 tablet_bootstrap.cc:492] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
W20260709 13:09:42.847991 21113 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:42.854815 21219 raft_consensus.cc:1275] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:09:42.856529 29461 consensus_queue.cc:1048] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:42.857012 20963 catalog_manager.cc:5697] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 26 to 27, leader changed from 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 27 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:42.876468 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:09:42.894598 29462 tablet_bootstrap.cc:492] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap starting.
W20260709 13:09:42.895516 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.061198 21113 consensus_peers.cc:597] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:43.068701 19967 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 21093)
I20260709 13:09:43.070801 21145 raft_consensus.cc:1275] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:09:43.072650 29471 consensus_queue.cc:1048] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:09:43.083382 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.083940 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 73427.2us
W20260709 13:09:43.090378 21187 consensus_peers.cc:597] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:43.109222 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.119794 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:43.173017 21767 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:43.188851 21139 raft_consensus.cc:1275] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:09:43.190661 29415 consensus_queue.cc:1048] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:09:43.213162 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:43.218206 29452 tablet_bootstrap.cc:492] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:43.219415 29452 tablet_bootstrap.cc:492] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:09:43.220216 29452 ts_tablet_manager.cc:1403] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.550s	user 0.060s	sys 0.004s
I20260709 13:09:43.223706 29452 raft_consensus.cc:359] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:43.224694 29452 raft_consensus.cc:740] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:09:43.240597 29452 consensus_queue.cc:260] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:43.249109 29452 ts_tablet_manager.cc:1434] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:09:43.268530 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-41268ef73d2c487dab473150fa26d73e received from {username='slave'} at 127.0.0.1:54620
I20260709 13:09:43.269062 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-41268ef73d2c487dab473150fa26d73e on tablet 41268ef73d2c487dab473150fa26d73e with peer b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:43.286808 29457 tablet_bootstrap.cc:492] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:43.287971 29457 tablet_bootstrap.cc:492] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:09:43.288810 29457 ts_tablet_manager.cc:1403] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.484s	user 0.048s	sys 0.007s
I20260709 13:09:43.292016 29457 raft_consensus.cc:359] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:43.292925 29457 raft_consensus.cc:740] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:09:43.293701 29457 consensus_queue.cc:260] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:43.296648 29457 ts_tablet_manager.cc:1434] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:43.298897 21231 tablet_copy_service.cc:342] P 46c1676121b64b7d9bcf2227cc639dd3: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-5a16f225ecc04514a485518bcd2b8112 received from {username='slave'} at 127.0.0.1:54620
I20260709 13:09:43.299453 21231 tablet_copy_service.cc:434] P 46c1676121b64b7d9bcf2227cc639dd3: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-5a16f225ecc04514a485518bcd2b8112 on tablet 5a16f225ecc04514a485518bcd2b8112 with peer b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:43.335382 21113 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:43.343885 29462 tablet_bootstrap.cc:492] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:43.344966 29462 tablet_bootstrap.cc:492] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: Bootstrap complete.
I20260709 13:09:43.350615 29462 ts_tablet_manager.cc:1403] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: Time spent bootstrapping tablet: real 0.456s	user 0.052s	sys 0.007s
I20260709 13:09:43.354046 29462 raft_consensus.cc:359] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:43.355105 29462 raft_consensus.cc:740] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Initialized, Role: LEARNER
I20260709 13:09:43.355894 29462 consensus_queue.cc:260] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: NON_VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: true } }
I20260709 13:09:43.359898 29462 ts_tablet_manager.cc:1434] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:09:43.361651 21157 tablet_copy_service.cc:342] P 5e5e65640c8b4f5ca7eadc6a14312c55: Request end of tablet copy session b70fc04de37e4239a8336534f0c9c0fb-5196cb9fb3c1488180a6421083b18ff9 received from {username='slave'} at 127.0.0.1:50996
I20260709 13:09:43.362109 21157 tablet_copy_service.cc:434] P 5e5e65640c8b4f5ca7eadc6a14312c55: ending tablet copy session b70fc04de37e4239a8336534f0c9c0fb-5196cb9fb3c1488180a6421083b18ff9 on tablet 5196cb9fb3c1488180a6421083b18ff9 with peer b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:43.434275 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:43.449080 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:43.458204 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.461000 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.476318 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.477406 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:43.626225 21794 raft_consensus.cc:1217] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEARNER]: Deduplicated request from leader. Original: 24.13->[24.14-24.14]   Dedup: 24.14->[]
W20260709 13:09:43.645476 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.646059 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:43.650285 21113 consensus_peers.cc:597] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.682677 21113 consensus_peers.cc:597] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.684222 21113 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:43.694296 21794 raft_consensus.cc:1217] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEARNER]: Deduplicated request from leader. Original: 22.12->[22.13-22.13]   Dedup: 22.13->[]
W20260709 13:09:43.691190 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:43.701143 21794 raft_consensus.cc:1217] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEARNER]: Deduplicated request from leader. Original: 17.11->[17.12-17.12]   Dedup: 17.12->[]
W20260709 13:09:43.799075 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:43.863739 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.947568 21113 consensus_peers.cc:597] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:43.954156 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:43.968010 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:44.069429 19967 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:44.080673 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:44.098050 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:44.098963 21113 consensus_peers.cc:597] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:44.144873 29427 raft_consensus.cc:1064] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:09:44.147847 29427 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:44.176193 29458 raft_consensus.cc:1064] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:09:44.187103 21799 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 LEARNER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:09:44.190687 21113 consensus_peers.cc:597] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.212196 29427 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:44.183581 29458 consensus_queue.cc:237] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 12, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:44.237639 21221 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:44.264694 29466 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:09:44.276809 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:44.286732 29482 raft_consensus.cc:1064] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: attempting to promote NON_VOTER b70fc04de37e4239a8336534f0c9c0fb to VOTER
I20260709 13:09:44.321955 29482 consensus_queue.cc:237] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 11, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:44.348758 21113 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:44.362766 21799 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:09:44.368566 29458 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:44.376060 21139 raft_consensus.cc:1275] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:09:44.389946 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.391809 29466 raft_consensus.cc:2955] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 12 to 13, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.397976 21215 raft_consensus.cc:2955] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.402009 29503 consensus_queue.cc:1048] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:09:44.430902 20955 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8b6dd6862dd14baba4a82c5a837bc802 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 5e5e65640c8b4f5ca7eadc6a14312c55 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.433107 21799 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 LEARNER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:09:44.433945 20964 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: config changed from index 12 to 13, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.435458 29501 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:44.440204 21147 raft_consensus.cc:1275] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:09:44.443605 29502 consensus_queue.cc:1048] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:09:44.446095 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.474540 21794 raft_consensus.cc:2955] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } attrs { replace: true } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.487732 29502 raft_consensus.cc:2955] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Committing config change with OpId 24.15: config changed from index 14 to 15, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.505573 21139 raft_consensus.cc:2955] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index 14 to 15, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:44.539084 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:09:44.539155 29458 raft_consensus.cc:2955] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index 13 to 14, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.549960 21799 raft_consensus.cc:2955] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.565374 21792 raft_consensus.cc:2955] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index 14 to 15, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.566160 21147 raft_consensus.cc:2955] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:44.590867 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:44.643309 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:44.656345 21113 consensus_peers.cc:597] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:44.662712 21113 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:09:44.690168 21147 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:44.693624 20962 catalog_manager.cc:5697] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 13 to 14, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.697109 21792 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:44.711796 21219 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Refusing update from remote peer 5e5e65640c8b4f5ca7eadc6a14312c55: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:44.740010 29461 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:44.746229 29461 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:44.752310 20962 catalog_manager.cc:5697] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: config changed from index 14 to 15, b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196) changed from NON_VOTER to VOTER. New cstate: current_term: 24 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.752720 29466 raft_consensus.cc:2955] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.757743 21792 raft_consensus.cc:2955] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:44.759440 21219 raft_consensus.cc:2955] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
W20260709 13:09:44.855571 21767 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:44.859568 21767 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:44.894824 21111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 55947.1us
I20260709 13:09:44.895597 20955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5196cb9fb3c1488180a6421083b18ff9 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:09:44.915088 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:44.939109 20957 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 122404us
W20260709 13:09:44.945665 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet b7a0bb40f8e3428899935fdfa915dfd0: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:44.956000 21113 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111) [suppressed 445 similar messages]
I20260709 13:09:44.948699 20963 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 reported cstate change: config changed from index 13 to 14, VOTER 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193) evicted. New cstate: current_term: 17 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:45.069941 19967 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:45.132325 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:45.135257 21187 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:45.174417 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.198122 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.214112 21113 consensus_peers.cc:597] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:45.256283 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.329254 21113 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:45.337332 21767 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:45.352888 21113 consensus_peers.cc:597] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:45.499493 21113 consensus_peers.cc:597] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:45.555835 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:45.610756 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.641754 21187 consensus_peers.cc:597] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:45.698895 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:45.776157 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 72068.2us
W20260709 13:09:45.778529 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 75501.8us
W20260709 13:09:45.799969 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.841073 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.865473 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.869926 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.879546 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.894673 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.899253 21767 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:45.962961 21113 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:45.967748 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:46.070469 19967 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:46.116030 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.170023 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:46.249131 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:46.275378 21113 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.397194 21113 consensus_peers.cc:597] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.397809 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:46.420034 21113 consensus_peers.cc:597] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.438649 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.447830 20957 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44397.7us
W20260709 13:09:46.462184 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.473814 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5196cb9fb3c1488180a6421083b18ff9: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:46.479242 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:46.536883 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.542855 21113 consensus_peers.cc:597] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:46.613754 21113 consensus_peers.cc:597] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.751736 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.792917 21113 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:46.981403 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:47.040910 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:47.052899 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:47.070971 19967 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:47.147019 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:47.182449 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:47.226940 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:47.230252 21113 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:47.237030 21113 consensus_peers.cc:597] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:47.302371 21767 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:47.528303 21767 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:47.623939 20957 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 53345.8us
W20260709 13:09:47.641938 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5196cb9fb3c1488180a6421083b18ff9: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:47.659581 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:47.671517 21113 consensus_peers.cc:597] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:47.683951 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:47.738008 21187 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:47.741334 21113 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:47.781813 21113 consensus_peers.cc:597] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:47.792096 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:47.961612 21767 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:48.020186 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 51889us
W20260709 13:09:48.022974 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 52408us
W20260709 13:09:48.027678 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 60336.1us
W20260709 13:09:48.029727 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:48.047197 21184 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:33295, user_credentials={real_user=slave}} blocked reactor thread for 93025.7us
W20260709 13:09:48.056279 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:48.067497 21113 consensus_peers.cc:597] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:48.079320 19967 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:49.079880 19967 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:49.113189 21111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.001s	sys 0.001s
W20260709 13:09:49.113880 21111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.001s	sys 0.001s
W20260709 13:09:49.134855 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet b7a0bb40f8e3428899935fdfa915dfd0: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.175032 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08002ce320 after lost signal to thread 19970
I20260709 13:09:49.203097 29547 raft_consensus.cc:493] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.203994 29547 raft_consensus.cc:515] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.207500 29547 leader_election.cc:290] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.209661 29548 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.210433 29548 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.189602 29537 raft_consensus.cc:493] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.248023 29537 raft_consensus.cc:515] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.249801 29540 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.252172 29541 raft_consensus.cc:493] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.253170 29537 leader_election.cc:290] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.252842 29541 raft_consensus.cc:515] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.261709 29542 raft_consensus.cc:493] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.261819 29537 raft_consensus.cc:493] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:49.264094 29537 raft_consensus.cc:515] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.262852 29542 raft_consensus.cc:515] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.266729 29537 leader_election.cc:290] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.263370 29541 leader_election.cc:290] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.250519 29540 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.267179 20963 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38938 (request call id 198) took 1164 ms. Trace:
W20260709 13:09:49.270629 21767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.115s	user 0.003s	sys 0.019s
W20260709 13:09:49.273859 21767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.003s	sys 0.020s
I20260709 13:09:49.266799 29542 leader_election.cc:290] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.269461 20963 rpcz_store.cc:276] 0709 13:09:48.103026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:48.106988 (+  3962us) service_pool.cc:224] Handling call
0709 13:09:49.264917 (+1157929us) spinlock_profiling.cc:243] Waited 4.08 ms on lock 0x7b5400002490. stack: 00007f7f3f3fd343 00007f7f3f3fd240 00007f7f3ed8c1d4 00007f7f48aca304 00007f7f48aca299 00007f7f3f2b7db1 00007f7f3f3f7e9c 00007f7f3f3f80f6 00007f7f48371a89 00007f7f48309da6 00007f7f482db088 00007f7f483d937e 00007f7f4689a2a4 00007f7f4689a232 00007f7f4689a161 00007f7f4689a0dc
0709 13:09:49.266980 (+  2063us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":0,"spinlock_wait_cycles":9377664}
I20260709 13:09:49.297600 29537 raft_consensus.cc:493] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.297851 29532 raft_consensus.cc:493] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:49.298359 29532 raft_consensus.cc:515] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.298300 29537 raft_consensus.cc:515] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.300783 29537 leader_election.cc:290] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.303390 29519 raft_consensus.cc:493] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:49.303622 29541 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.304023 29519 raft_consensus.cc:515] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.304316 29541 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.297622 29535 raft_consensus.cc:493] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.306438 29519 leader_election.cc:290] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.309612 29548 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:49.306877 29535 raft_consensus.cc:515] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:49.310217 20955 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8b6dd6862dd14baba4a82c5a837bc802 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 5e5e65640c8b4f5ca7eadc6a14312c55 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 18 leader_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } } }
I20260709 13:09:49.328670 29532 leader_election.cc:290] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.326941 29542 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:49.329775 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800367a80 after lost signal to thread 21004
W20260709 13:09:49.330832 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 47191.1us
I20260709 13:09:49.330161 29542 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.341975 29474 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:49.342675 29474 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.329175 29537 raft_consensus.cc:493] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.343833 29537 raft_consensus.cc:515] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.353562 29553 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:49.354492 29537 leader_election.cc:290] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.354317 29553 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.359120 21793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d158af381004432b3f3f21e53fd19ac" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.359347 29548 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.363226 29543 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.363057 29548 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.363927 29543 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.365294 29548 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:49.366158 29543 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.368856 29514 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:49.369552 29514 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.371904 21800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb8a7df9f774ce0afbd5b26fa2a88d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.372785 29556 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.373903 29514 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.359856 29535 leader_election.cc:290] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.375643 29548 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.376276 29548 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.376617 29553 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.373472 29556 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.378613 29548 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:49.360301 29542 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:49.386782 29542 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:49.388041 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08001b6940 after lost signal to thread 21171
I20260709 13:09:49.379385 29556 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.387483 29542 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.407441 29535 raft_consensus.cc:493] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:49.408196 29535 raft_consensus.cc:515] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.410706 29542 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:49.389169 29527 raft_consensus.cc:493] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.414034 29458 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.413962 29527 raft_consensus.cc:515] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.414727 29458 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.416638 29527 leader_election.cc:290] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.417333 29535 leader_election.cc:290] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.416810 29458 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:49.413235 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9417f68ba15d49c6915ab46793aab93c" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.431934 29514 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.432639 29514 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.435316 29514 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.435366 29535 raft_consensus.cc:493] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.438450 29535 raft_consensus.cc:515] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.439144 21144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.441053 29474 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.441213 29535 leader_election.cc:290] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.436292 29559 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.451524 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bbc776723436d954fe0b2a30a5b09" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.452428 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.452659 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5425aa9a9a4fba958ae97f007977d2" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.454053 21139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4af1b94141049e0a78a816c8ba31d72" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.455019 21140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e4456f3a9364c7289dfce3163ab4760" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.455379 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752b0641d0a3410795a788ef8dc8e035" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
W20260709 13:09:49.457293 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 124735us
I20260709 13:09:49.461849 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea0f23227964b96bff6287436e970cd" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.465265 29541 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.470191 29559 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:49.470844 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800286720 after lost signal to thread 21245
W20260709 13:09:49.471231 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08003b5620 after lost signal to thread 21246
I20260709 13:09:49.472661 29559 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.474659 29540 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:49.479972 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08000f9180 after lost signal to thread 21825
W20260709 13:09:49.480340 21001 debug-util.cc:398] Leaking SignalData structure 0x7b080033f5c0 after lost signal to thread 21826
I20260709 13:09:49.486940 29561 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.487668 29561 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:49.490060 29561 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:49.493132 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4a635811aa248b599c9361b7abf9f32" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.498224 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc53d52fdeaa441b9e9d329e69d94440" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.535065 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9833b15c99c34c269ca3afa7dac3f175" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:49.536094 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 66942.9us
I20260709 13:09:49.537153 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.538862 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8d00e740dba47b2b43be9e4ef1ecb55" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.541134 29550 raft_consensus.cc:493] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.541757 29550 raft_consensus.cc:515] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.544314 29550 leader_election.cc:290] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.550966 29543 raft_consensus.cc:493] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b70fc04de37e4239a8336534f0c9c0fb)
I20260709 13:09:49.544260 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ee2463d8894a95be9b747f813f9106" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.551652 29543 raft_consensus.cc:515] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.553414 29514 raft_consensus.cc:493] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:49.554098 29543 leader_election.cc:290] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:49.554068 29514 raft_consensus.cc:515] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:49.556432 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 45182.2us
I20260709 13:09:49.563807 29514 leader_election.cc:290] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:49.565106 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e30c07cef4334b45ade426b26069b26c" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.565308 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2875dc050a244109a0287cc6dafa8431" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.568104 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.568248 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.569024 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.570933 21141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77143ea923a34396a4cbc31f42dd0a18" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.571363 21147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.572252 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.572559 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8c9b46a490540a2bb0a90f7bfbd1e53" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:49.573381 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:49.573601 21219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:49.574148 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:49.574636 21113 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:49.578773 29535 raft_consensus.cc:493] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
W20260709 13:09:49.580920 21187 leader_election.cc:336] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.582996 21221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a473266451804da089da06e9c927e7ee" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.584115 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a4c442e73243eda44d43ba0eaf0d2a" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.595455 29535 raft_consensus.cc:515] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:49.596508 21187 leader_election.cc:336] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.584640 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e714d9095b9c430bba199e0209c0b946" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.585844 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d5d573fe0fd4b91af3ea4dd4f6dc7f3" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.586836 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9028fa09ccd24f5494e4e5de5a8e6e18" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.590794 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.597597 29535 leader_election.cc:290] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:49.597128 21187 consensus_peers.cc:597] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:49.598955 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:49.599498 21187 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.599948 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.600811 29561 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:09:49.602427 21187 leader_election.cc:336] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.602890 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.603677 29561 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:49.603890 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d78b35ac92d402db8a5e3fe45be52b5" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:49.604671 21187 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.605077 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.606917 29561 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:09:49.594658 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:49.607995 21767 leader_election.cc:336] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.610105 21767 leader_election.cc:336] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.608120 21187 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.611533 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:49.611752 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:49.615535 21767 leader_election.cc:336] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.615597 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e564bd720422bb798c652af4fb2eb" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.608594 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:49.611980 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72edcb8f7f3d437b8e7bb24c612600ab" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.612309 29561 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 13:09:49.613087 21187 leader_election.cc:336] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.598392 21113 leader_election.cc:336] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.617584 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.617694 21113 leader_election.cc:304] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.618453 29561 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:49.618515 29535 raft_consensus.cc:2749] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:49.619694 21187 leader_election.cc:336] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.620365 21187 leader_election.cc:336] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.617154 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:09:49.622088 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.622816 29561 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:09:49.623795 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:49.625025 21767 leader_election.cc:336] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.619726 21113 leader_election.cc:336] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.625924 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.625989 21113 leader_election.cc:304] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.628911 29520 raft_consensus.cc:2749] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:09:49.629223 21113 leader_election.cc:336] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.630378 21113 leader_election.cc:304] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.631422 29523 raft_consensus.cc:2749] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:49.629808 29550 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:09:49.632469 29550 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.633016 21765 leader_election.cc:304] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.633849 29514 raft_consensus.cc:2749] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:49.634982 29550 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:49.636672 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:49.636538 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:49.637522 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:49.638414 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:49.644107 21184 leader_election.cc:304] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.644965 29543 raft_consensus.cc:2749] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:49.646564 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.647038 21184 leader_election.cc:304] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.647451 29514 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:49.649443 29514 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:49.650003 29514 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:49.651104 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:49.638998 21113 leader_election.cc:336] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.651815 21113 leader_election.cc:304] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.652367 29514 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:49.652736 21113 leader_election.cc:336] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.653272 21113 leader_election.cc:336] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.655464 29550 raft_consensus.cc:2749] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:09:49.658993 29514 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:49.660305 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:49.660900 29561 raft_consensus.cc:2749] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:49.662906 29514 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:49.663383 21765 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:49.664142 29514 raft_consensus.cc:2749] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:49.660979 21113 leader_election.cc:336] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.665267 21113 leader_election.cc:304] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.665295 21145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" is_pre_election: true
I20260709 13:09:49.666112 29520 raft_consensus.cc:2749] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:49.667558 21113 leader_election.cc:336] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.668108 21113 leader_election.cc:336] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.668504 21113 leader_election.cc:304] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.669194 29520 raft_consensus.cc:2749] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:49.670670 21113 leader_election.cc:336] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.671141 21113 leader_election.cc:304] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.672613 29523 raft_consensus.cc:2749] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:49.675325 21184 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.676254 29529 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:49.676501 21184 leader_election.cc:304] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.677273 29529 raft_consensus.cc:2749] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:49.681635 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:49.682713 21187 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.683239 21187 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.683370 29514 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:49.686650 21113 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:49.689951 29529 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:49.691059 21187 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.693140 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.692582 21184 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.685110 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:49.690021 21113 leader_election.cc:336] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:49.694577 29514 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:49.699049 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:09:49.714331 29561 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:49.715952 21111 leader_election.cc:304] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.716249 29543 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:49.716784 29550 raft_consensus.cc:2749] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:49.721753 21111 leader_election.cc:304] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.722622 29550 raft_consensus.cc:2749] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:49.724380 21111 leader_election.cc:304] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.725348 29520 raft_consensus.cc:2749] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:49.732697 21111 leader_election.cc:304] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.733458 29550 raft_consensus.cc:2749] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:49.743588 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41268ef73d2c487dab473150fa26d73e" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.745620 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f10495397de45019165a00ba7c4ec29" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.747601 21214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:49.749428 21217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293026cede06406bad9e7b5fc490abe2" candidate_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
W20260709 13:09:49.758773 21767 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:49.772001 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:49.773442 29559 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:09:49.788373 21113 consensus_peers.cc:597] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:49.805325 21187 leader_election.cc:336] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:49.814390 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:49.815028 21113 consensus_peers.cc:597] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:49.824153 21187 leader_election.cc:304] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 9384358ce43f4e3fac2ae44bdf11c389, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.826383 29540 raft_consensus.cc:2749] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:49.849498 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 44220.7us
W20260709 13:09:49.856972 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:09:49.885047 21111 leader_election.cc:304] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:49.886461 21111 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5e65640c8b4f5ca7eadc6a14312c55; no voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:49.887655 29535 raft_consensus.cc:2749] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:49.888465 29535 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:49.967643 21187 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111) [suppressed 410 similar messages]
W20260709 13:09:49.984443 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:50.051460 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:50.080402 19967 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:50.223857 21113 consensus_peers.cc:597] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:50.231863 21113 consensus_peers.cc:597] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:50.287798 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:50.301400 21113 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:50.332549 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:50.348387 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:50.398435 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:50.496351 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:50.556478 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:50.709702 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:50.822818 21767 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:50.823976 21113 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:50.839610 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:50.899019 21113 consensus_peers.cc:597] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:50.924453 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:51.061108 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:51.080943 19967 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:51.085713 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.145344 21113 consensus_peers.cc:597] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:51.183462 21113 consensus_peers.cc:597] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:51.228945 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.238336 21113 consensus_peers.cc:597] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:51.247071 21187 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:51.324522 21113 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:51.500110 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.504487 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:51.533345 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.600804 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.627416 21767 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:51.705220 21767 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:51.722164 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.788933 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:51.815454 21187 consensus_peers.cc:597] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:51.818162 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:51.910044 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.910668 21113 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.944785 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:51.953603 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.982398 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:51.995157 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.081462 19967 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:52.129545 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:52.255991 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:52.266201 21767 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:52.272076 21113 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.312299 21113 consensus_peers.cc:597] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.343628 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:52.492107 21113 consensus_peers.cc:597] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.624847 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:52.642280 21113 consensus_peers.cc:597] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:52.722774 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.768285 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.845541 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.854387 21113 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.878183 21113 consensus_peers.cc:597] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.928419 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:52.951133 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:52.990568 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:09:52.996663 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:53.033034 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:53.082047 19967 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:53.165431 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:09:53.178300 20959 maintenance_manager.cc:419] P b068e036d3f142f4b255651ac6a0e101: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033368
W20260709 13:09:53.188247 21767 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:53.321110 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:53.375308 20958 maintenance_manager.cc:643] P b068e036d3f142f4b255651ac6a0e101: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.196s	user 0.036s	sys 0.008s Metrics: {"bytes_written":7211,"delete_count":0,"lbm_write_time_us":1194,"lbm_writes_lt_1ms":7,"reinsert_count":0,"thread_start_us":39715,"threads_started":1,"update_count":292,"wal-append.queue_time_us":28744}
I20260709 13:09:53.377648 20959 maintenance_manager.cc:419] P b068e036d3f142f4b255651ac6a0e101: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.359430
W20260709 13:09:53.454101 21113 consensus_peers.cc:597] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:53.495554 21187 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:53.510650 21113 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:53.552990 21113 consensus_peers.cc:597] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:53.571760 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:09:53.761592 20958 maintenance_manager.cc:643] P b068e036d3f142f4b255651ac6a0e101: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.378s	user 0.083s	sys 0.035s Metrics: {"cfile_cache_miss":10,"cfile_cache_miss_bytes":29100,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":11624,"dirs.run_cpu_time_us":584,"dirs.run_wall_time_us":7447,"lbm_read_time_us":2349,"lbm_reads_lt_1ms":26,"lbm_write_time_us":20000,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":11,"peak_mem_usage":315023,"reinsert_count":0,"thread_start_us":27586,"threads_started":2,"update_count":317}
I20260709 13:09:53.765090 20959 maintenance_manager.cc:419] P b068e036d3f142f4b255651ac6a0e101: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:09:53.765946 20958 maintenance_manager.cc:643] P b068e036d3f142f4b255651ac6a0e101: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 13:09:53.828482 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:53.839879 21113 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:53.847014 21113 consensus_peers.cc:597] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:53.903753 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:53.948223 21113 consensus_peers.cc:597] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:54.012044 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.013659 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 43538us
W20260709 13:09:54.028849 20957 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 89009.8us
W20260709 13:09:54.032562 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5196cb9fb3c1488180a6421083b18ff9: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:54.050434 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.082562 19967 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:54.100337 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.104369 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.140919 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:54.169449 21767 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:54.232733 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:54.293143 21187 consensus_peers.cc:597] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:54.295055 21113 consensus_peers.cc:597] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.305260 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.436641 21767 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:54.479377 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:54.489569 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.496436 21765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:46249, user_credentials={real_user=slave}} blocked reactor thread for 45255.3us
W20260709 13:09:54.520277 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:54.557806 21767 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:54.582472 21113 consensus_peers.cc:597] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.602991 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:54.654786 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:54.719509 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.838683 21113 consensus_peers.cc:597] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.893361 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:54.948248 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:54.967815 21113 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111) [suppressed 482 similar messages]
W20260709 13:09:55.045755 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:55.083055 19967 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 21093)
W20260709 13:09:55.099968 21113 consensus_peers.cc:597] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:09:55.102573 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.131897 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.145157 21113 consensus_peers.cc:597] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.206486 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:55.335234 21113 consensus_peers.cc:597] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.345813 21113 consensus_peers.cc:597] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:55.364019 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08001682c0 after lost signal to thread 19970
W20260709 13:09:55.371346 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800141380 after lost signal to thread 21004
W20260709 13:09:55.372779 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800050e80 after lost signal to thread 21171
W20260709 13:09:55.373993 21242 debug-util.cc:398] Leaking SignalData structure 0x7b08000cb580 after lost signal to thread 21245
W20260709 13:09:55.375270 21242 debug-util.cc:398] Leaking SignalData structure 0x7b0800202fa0 after lost signal to thread 21825
W20260709 13:09:55.398764 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.401463 21113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 55704.4us
W20260709 13:09:55.507737 21113 consensus_peers.cc:597] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.558161 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.588461 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.702493 21767 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:55.732698 21187 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:55.811107 21113 consensus_peers.cc:597] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.845147 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:09:55.879999 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:55.881006 21113 consensus_peers.cc:597] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:09:55.965967 19967 tablet_server.cc:196] TabletServer@127.19.127.193:0 shutdown complete.
W20260709 13:09:55.977708 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:56.067940 21113 consensus_peers.cc:597] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:56.238898 21113 consensus_peers.cc:597] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:56.318574 21113 consensus_peers.cc:597] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:56.340423 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:09:56.361619 19967 tablet_server.cc:179] TabletServer@127.19.127.194:0 shutting down...
W20260709 13:09:56.362103 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:56.383201 21765 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.429306 21185 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.443106 21113 consensus_peers.cc:597] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:56.443661 21113 consensus_peers.cc:597] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:56.448611 21185 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.460116 21185 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.478132 21185 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.480917 21765 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.513749 21765 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.549080 21765 consensus_peers.cc:597] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.553838 21765 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.556423 21767 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:56.556699 21765 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.573741 21765 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.582501 21765 consensus_peers.cc:597] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.585430 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:09:56.587623 21185 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.590739 19967 tablet_replica.cc:333] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.594018 19967 raft_consensus.cc:2243] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:56.603752 19967 raft_consensus.cc:2272] T b4ee2463d8894a95be9b747f813f9106 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:56.605912 21765 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.608805 21185 consensus_peers.cc:597] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.614230 21185 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.616086 21113 consensus_peers.cc:597] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:56.617084 21185 consensus_peers.cc:597] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.628163 19967 tablet_replica.cc:333] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.629405 19967 raft_consensus.cc:2243] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:09:56.631016 19967 raft_consensus.cc:2272] T de53f43e2a0e4a91a1767df521fc2b8c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.636824 19967 tablet_replica.cc:333] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.638001 19967 raft_consensus.cc:2243] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:09:56.645539 19967 raft_consensus.cc:2272] T d524f5c1a9e540c9946dd77d8f971ef1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.650161 19967 tablet_replica.cc:333] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.652069 19967 raft_consensus.cc:2243] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:56.653987 19967 raft_consensus.cc:2272] T 79a87917a1b44746a96a84470b77ea16 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.658696 19967 tablet_replica.cc:333] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.664882 19967 raft_consensus.cc:2243] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:56.665740 19967 raft_consensus.cc:2272] T d4a635811aa248b599c9361b7abf9f32 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:56.669661 21765 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.675832 21185 consensus_peers.cc:597] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.678333 19967 tablet_replica.cc:333] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.686124 19967 raft_consensus.cc:2243] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:56.686964 19967 raft_consensus.cc:2272] T b59f0de1289c4c98ae7ce302ca584a30 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.690534 19967 tablet_replica.cc:333] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
W20260709 13:09:56.697551 21185 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.712373 21187 consensus_peers.cc:597] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:56.715530 21765 consensus_peers.cc:597] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.714818 21113 consensus_peers.cc:597] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:56.716866 19967 raft_consensus.cc:2243] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:56.726406 19967 raft_consensus.cc:2272] T 9f10495397de45019165a00ba7c4ec29 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.731678 19967 tablet_replica.cc:333] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
W20260709 13:09:56.732692 21765 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:56.740176 19967 raft_consensus.cc:2243] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:09:56.745348 19967 raft_consensus.cc:2272] T 2e5425aa9a9a4fba958ae97f007977d2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.748625 19967 tablet_replica.cc:333] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
W20260709 13:09:56.750522 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:56.757258 21113 consensus_peers.cc:597] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:56.765710 19967 raft_consensus.cc:2243] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:56.766803 19967 raft_consensus.cc:2272] T 12e3a58d24d64816bce2752c6adb7e33 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:56.770015 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:56.769709 19967 tablet_replica.cc:333] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.783229 19967 raft_consensus.cc:2243] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:09:56.784755 19967 raft_consensus.cc:2272] T 6b5d4dd9fad84e018bc7168755b0ba6b P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:56.789203 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:56.823920 21185 consensus_peers.cc:597] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.840323 21185 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:56.856493 21113 consensus_peers.cc:597] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:56.789012 19967 tablet_replica.cc:333] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.884189 19967 raft_consensus.cc:2243] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:56.890511 19967 raft_consensus.cc:2272] T 6beb666b5d504215a141add24a6d632f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.893944 19967 tablet_replica.cc:333] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.894886 19967 raft_consensus.cc:2243] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:09:56.896188 19967 raft_consensus.cc:2272] T 77143ea923a34396a4cbc31f42dd0a18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.899170 19967 tablet_replica.cc:333] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.900089 19967 raft_consensus.cc:2243] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:09:56.901302 19967 raft_consensus.cc:2272] T 72edcb8f7f3d437b8e7bb24c612600ab P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.904349 19967 tablet_replica.cc:333] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.907337 19967 raft_consensus.cc:2243] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:56.910435 19967 raft_consensus.cc:2272] T 3d51215a492444539c4905a13bd0c12d P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.915818 19967 tablet_replica.cc:333] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.916873 19967 raft_consensus.cc:2243] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:09:56.918228 19967 raft_consensus.cc:2272] T 080f4010b39e40a8b8efb7280faec38f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:56.940879 19967 tablet_replica.cc:333] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:56.943758 19967 raft_consensus.cc:2243] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:09:56.949119 21113 consensus_peers.cc:597] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:09:56.947527 19967 raft_consensus.cc:2272] T c20ca56847f64504a85694d10f47558e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:56.991806 21767 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:09:57.036650 19967 tablet_replica.cc:333] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.037585 19967 raft_consensus.cc:2243] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.039108 19967 raft_consensus.cc:2272] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.042122 19967 tablet_replica.cc:333] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.042975 19967 raft_consensus.cc:2243] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.044449 19967 raft_consensus.cc:2272] T 752b0641d0a3410795a788ef8dc8e035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.049290 19967 tablet_replica.cc:333] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.050189 19967 raft_consensus.cc:2243] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 30 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.051712 19967 raft_consensus.cc:2272] T d4af1b94141049e0a78a816c8ba31d72 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.054626 19967 tablet_replica.cc:333] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.056578 19967 raft_consensus.cc:2243] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.057958 19967 raft_consensus.cc:2272] T 223e82a164f445c2b46667d58a5d2d1f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.061266 19967 tablet_replica.cc:333] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.062064 19967 raft_consensus.cc:2243] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.062676 19967 raft_consensus.cc:2272] T b8d00e740dba47b2b43be9e4ef1ecb55 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.065217 19967 tablet_replica.cc:333] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.065891 19967 raft_consensus.cc:2243] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.067065 19967 raft_consensus.cc:2272] T e0631ac443bb4e6dbbbc599dd4b67d2e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.070166 19967 tablet_replica.cc:333] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.070837 19967 raft_consensus.cc:2243] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.072300 19967 raft_consensus.cc:2272] T 6e4456f3a9364c7289dfce3163ab4760 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.074858 19967 tablet_replica.cc:333] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.075799 19967 raft_consensus.cc:2243] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.077126 19967 raft_consensus.cc:2272] T 8b6dd6862dd14baba4a82c5a837bc802 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.079139 19967 tablet_replica.cc:333] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.079777 19967 raft_consensus.cc:2243] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.080426 19967 raft_consensus.cc:2272] T e714d9095b9c430bba199e0209c0b946 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.082630 19967 tablet_replica.cc:333] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.083251 19967 raft_consensus.cc:2243] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.084525 19967 raft_consensus.cc:2272] T de27148f295b4ef5a3f79084b8ed5ae6 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.088505 19967 tablet_replica.cc:333] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.089393 19967 raft_consensus.cc:2243] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.090758 19967 raft_consensus.cc:2272] T 731bbc776723436d954fe0b2a30a5b09 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.093842 19967 tablet_replica.cc:333] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.094693 19967 raft_consensus.cc:2243] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.095717 19967 raft_consensus.cc:2272] T 1ea0f23227964b96bff6287436e970cd P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.099042 19967 tablet_replica.cc:333] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.100083 19967 raft_consensus.cc:2243] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.100785 19967 raft_consensus.cc:2272] T d8a4c442e73243eda44d43ba0eaf0d2a P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.111406 19967 tablet_replica.cc:333] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.114832 19967 raft_consensus.cc:2243] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.116967 19967 raft_consensus.cc:2272] T 40d1baf8579949fbb0d5d45a18fefe35 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.129410 19967 tablet_replica.cc:333] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.130368 19967 raft_consensus.cc:2243] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.133863 19967 raft_consensus.cc:2272] T d693d94d4c0943d9ab08114c33fc2e7f P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.140282 19967 tablet_replica.cc:333] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.153855 19967 raft_consensus.cc:2243] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.155968 19967 raft_consensus.cc:2272] T 5a16f225ecc04514a485518bcd2b8112 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:57.165000 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:57.174898 19967 tablet_replica.cc:333] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.175892 19967 raft_consensus.cc:2243] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.187260 19967 raft_consensus.cc:2272] T 47de8d16a3804364a7fbc9bf598946c2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.194105 19967 tablet_replica.cc:333] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.195271 19967 raft_consensus.cc:2243] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Raft consensus shutting down.
W20260709 13:09:57.204543 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:57.205200 21767 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:09:57.205698 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:09:57.208891 19967 raft_consensus.cc:2272] T b7a0bb40f8e3428899935fdfa915dfd0 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.227583 19967 tablet_replica.cc:333] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.231408 19967 raft_consensus.cc:2243] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.232688 19967 raft_consensus.cc:2272] T 3ca4319420e54bc19f621da03c811035 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.235904 19967 tablet_replica.cc:333] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.236716 19967 raft_consensus.cc:2243] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.237347 19967 raft_consensus.cc:2272] T 9417f68ba15d49c6915ab46793aab93c P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.240103 19967 tablet_replica.cc:333] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.240830 19967 raft_consensus.cc:2243] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.242398 19967 raft_consensus.cc:2272] T a473266451804da089da06e9c927e7ee P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.249511 19967 tablet_replica.cc:333] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.250440 19967 raft_consensus.cc:2243] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.252847 19967 raft_consensus.cc:2272] T c80e14318ee44feba007795498031464 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.257898 19967 tablet_replica.cc:333] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.259682 19967 raft_consensus.cc:2243] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.260530 19967 raft_consensus.cc:2272] T ce55364f10c348819e7c63c08633b184 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.263450 19967 tablet_replica.cc:333] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
W20260709 13:09:57.264230 21113 consensus_peers.cc:597] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:57.264379 19967 raft_consensus.cc:2243] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.265089 19967 raft_consensus.cc:2272] T 9833b15c99c34c269ca3afa7dac3f175 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.268340 19967 tablet_replica.cc:333] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.269181 19967 raft_consensus.cc:2243] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.270416 19967 raft_consensus.cc:2272] T 2fbcef88f59d4f9ea0f01023c7265069 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.273765 19967 tablet_replica.cc:333] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.274637 19967 raft_consensus.cc:2243] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.275488 19967 raft_consensus.cc:2272] T 9028fa09ccd24f5494e4e5de5a8e6e18 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.278260 19967 tablet_replica.cc:333] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.279007 19967 raft_consensus.cc:2243] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:09:57.279582 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:09:57.279614 19967 raft_consensus.cc:2272] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.283888 19967 tablet_replica.cc:333] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.284651 19967 raft_consensus.cc:2243] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.285421 19967 raft_consensus.cc:2272] T cc53d52fdeaa441b9e9d329e69d94440 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.288429 19967 tablet_replica.cc:333] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.289158 19967 raft_consensus.cc:2243] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.290171 19967 raft_consensus.cc:2272] T 5196cb9fb3c1488180a6421083b18ff9 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.292737 19967 tablet_replica.cc:333] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.293715 19967 raft_consensus.cc:2243] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.294381 19967 raft_consensus.cc:2272] T 1fa2ded676fc4d048c810a5f842100d1 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.297415 19967 tablet_replica.cc:333] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.298244 19967 raft_consensus.cc:2243] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.299533 19967 raft_consensus.cc:2272] T 53c918605cdf4c39b832a695968308a3 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.302380 19967 tablet_replica.cc:333] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.303119 19967 raft_consensus.cc:2243] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.303821 19967 raft_consensus.cc:2272] T 293026cede06406bad9e7b5fc490abe2 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.306661 19967 tablet_replica.cc:333] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.307416 19967 raft_consensus.cc:2243] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:57.308369 19967 raft_consensus.cc:2272] T 41268ef73d2c487dab473150fa26d73e P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:57.311544 19967 tablet_replica.cc:333] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55: stopping tablet replica
I20260709 13:09:57.312289 19967 raft_consensus.cc:2243] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:09:57.313548 19967 raft_consensus.cc:2272] T 54fc5a3f4db84eac831870d621ddfa54 P 5e5e65640c8b4f5ca7eadc6a14312c55 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:57.318785 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:57.344684 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:57.404026 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet b7a0bb40f8e3428899935fdfa915dfd0: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:57.559233 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:57.762110 29562 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:57.763621 29562 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:57.795038 21185 leader_election.cc:336] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:57.801244 21187 leader_election.cc:336] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:57.806552 21187 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:57.803021 29562 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:57.816746 29562 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:09:57.820060 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:57.847759 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:57.851617 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:57.904398 29562 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:57.905112 29562 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:57.913338 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:57.914012 21187 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:57.920624 21185 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:57.921111 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:57.922148 29592 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:57.923337 29562 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:57.941090 29562 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:57.941793 29562 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:57.949080 29592 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:57.949874 29592 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:57.952307 29592 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:57.954242 21187 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:57.963150 29562 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:57.965464 21185 leader_election.cc:336] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:57.973778 21185 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:57.974243 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:57.976500 21187 leader_election.cc:336] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:57.977223 21187 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:57.978554 29562 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:57.976672 29592 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:57.998581 29575 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:57.999600 29575 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
W20260709 13:09:58.003756 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08003a9ca0 after lost signal to thread 19970
W20260709 13:09:58.005649 21001 debug-util.cc:398] Leaking SignalData structure 0x7b08003a4380 after lost signal to thread 21004
I20260709 13:09:58.005755 29575 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.006515 21001 debug-util.cc:398] Leaking SignalData structure 0x7b080039bb60 after lost signal to thread 21171
W20260709 13:09:58.008518 21001 debug-util.cc:398] Leaking SignalData structure 0x7b080040ce20 after lost signal to thread 21245
W20260709 13:09:58.009212 21185 leader_election.cc:336] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.010754 21001 debug-util.cc:398] Leaking SignalData structure 0x7b0800349c00 after lost signal to thread 21825
W20260709 13:09:58.040681 21187 leader_election.cc:336] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.041260 21187 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.042068 29562 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:58.046511 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:09:58.051036 29562 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.051873 29562 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:58.065594 21185 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.066680 29592 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.067497 29592 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.072000 29575 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.072788 29575 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.077752 29562 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:58.078697 29575 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:58.082460 29592 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.092260 21185 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.110819 21185 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.111581 21187 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.112258 29586 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.113893 21187 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.117564 29586 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.121934 29586 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:58.127851 21765 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.118620 29592 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:09:58.135433 29592 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.145654 29592 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.151686 29575 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.152469 29575 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.154793 29575 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.146557 21767 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.159713 21767 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.162647 29586 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:58.161777 29592 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.117794 21187 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.164667 21187 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.171365 29592 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.172302 29592 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.174655 29592 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.180053 21185 leader_election.cc:336] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.180166 29575 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:58.182638 21187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:42843, user_credentials={real_user=slave}} blocked reactor thread for 64853.3us
W20260709 13:09:58.186463 21187 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:58.184020 21185 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.189621 21185 leader_election.cc:336] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.187005 21187 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.193167 29592 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:09:58.202472 29592 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.203474 29592 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.206120 29592 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.208434 21187 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:58.208678 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:58.212217 29586 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:09:58.211220 21185 leader_election.cc:336] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.214396 29586 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.216717 29586 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:58.226972 21765 leader_election.cc:336] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.226404 21187 leader_election.cc:336] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.232476 21187 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.233527 29592 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:58.234654 21187 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.235347 21187 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.236088 29592 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:09:58.237867 21187 leader_election.cc:336] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.238301 21187 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.238999 29592 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:58.244298 29592 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.245229 29592 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:58.252790 21185 leader_election.cc:336] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.254442 29592 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:58.255221 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:58.256028 21799 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 23.
I20260709 13:09:58.256960 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:58.257658 21213 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:58.257704 29592 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.258399 29592 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.259016 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:58.260432 29575 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:58.260923 29575 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.261302 29575 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:58.263693 29601 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.264657 29601 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:58.265841 21185 leader_election.cc:336] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.268386 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:58.270424 29601 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:58.270794 21184 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
I20260709 13:09:58.272795 29562 raft_consensus.cc:2749] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:58.273655 29592 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:58.273728 21765 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.275317 21187 leader_election.cc:336] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.275847 21187 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.277177 29562 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:09:58.278529 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:09:58.284257 29575 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:58.291712 21185 leader_election.cc:336] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.299782 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d693d94d4c0943d9ab08114c33fc2e7f" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:58.300421 21799 raft_consensus.cc:3060] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:58.302695 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:58.313865 21799 raft_consensus.cc:2468] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 24.
I20260709 13:09:58.321532 29586 raft_consensus.cc:2764] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:09:58.314213 29601 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.322598 29601 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.325192 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:58.325269 21184 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:58.326576 29562 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.327195 29562 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.329195 29562 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:58.329645 29562 raft_consensus.cc:2804] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:58.332078 29562 raft_consensus.cc:697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
W20260709 13:09:58.332304 21767 leader_election.cc:336] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.333595 29601 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:58.334724 29562 consensus_queue.cc:237] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.327023 29575 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:58.348268 21185 leader_election.cc:336] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.351928 29575 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.352710 29575 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.354840 29575 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.365670 21187 leader_election.cc:336] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.366173 21187 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.367368 29592 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:58.372236 21765 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:58.383724 29586 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:58.390276 20962 catalog_manager.cc:5697] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 23 to 24, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 24 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:58.392578 21765 leader_election.cc:336] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.393133 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:58.394443 21767 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:58.395679 29607 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:58.398042 21185 leader_election.cc:336] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.401079 29592 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.401790 29592 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.402557 29562 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.403143 29562 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.404162 29592 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:58.405030 29562 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.409969 21187 leader_election.cc:336] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.410427 21187 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.411116 29592 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.412271 29562 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:58.411731 29592 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.417303 29592 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:58.422173 29592 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.422945 29592 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.426299 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:58.426944 21799 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 26.
I20260709 13:09:58.428208 21184 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:58.429010 29592 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:58.429592 29592 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:58.429998 29592 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.430410 29592 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:58.433917 29562 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.434701 29562 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.437048 29562 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.438629 21185 leader_election.cc:336] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.439231 21185 leader_election.cc:336] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.441622 29592 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.445464 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a0bb40f8e3428899935fdfa915dfd0" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:58.445726 29575 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.446122 21799 raft_consensus.cc:3060] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:58.446399 29575 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.449226 29575 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:58.450701 21187 leader_election.cc:336] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.452384 21187 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.453222 29575 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:58.453207 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:58.454191 21792 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
W20260709 13:09:58.454602 21187 leader_election.cc:336] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.455042 21187 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.455790 29575 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:58.456728 21187 leader_election.cc:336] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.457481 21799 raft_consensus.cc:2468] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 27.
I20260709 13:09:58.458642 21184 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:58.459455 29575 raft_consensus.cc:2804] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:09:58.459968 29575 raft_consensus.cc:697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:58.451917 29592 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
W20260709 13:09:58.460235 21187 leader_election.cc:336] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.461233 29575 consensus_queue.cc:237] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:58.462721 19967 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 21168)
W20260709 13:09:58.475327 21185 leader_election.cc:336] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.476231 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
W20260709 13:09:58.477602 21185 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.478363 29592 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:09:58.482076 21187 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:58.484185 21185 leader_election.cc:336] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.484683 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.485477 29575 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:58.486588 21185 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.487309 21185 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.487703 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.488399 29575 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:58.536600 20962 catalog_manager.cc:5697] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 26 to 27, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 27 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:58.611896 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:09:58.662369 21765 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:58.668077 29575 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.668673 29575 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:58.672834 29575 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:58.675735 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:09:58.676456 21185 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.683235 21187 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:58.683712 21187 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:09:58.684762 29592 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.685451 29592 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:58.687832 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:09:58.688262 29562 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:58.694123 29592 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:58.696604 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
W20260709 13:09:58.699123 21185 leader_election.cc:336] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.699617 21185 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, b70fc04de37e4239a8336534f0c9c0fb
W20260709 13:09:58.699939 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:09:58.701711 29575 raft_consensus.cc:2749] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:58.721320 29592 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.722011 29592 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:09:58.730680 21185 leader_election.cc:336] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.742383 29601 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.743224 29601 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.745628 29601 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:58.746214 29601 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.747731 21799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:09:58.748558 21799 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 17.
I20260709 13:09:58.748838 29592 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:58.750197 21184 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
W20260709 13:09:58.751085 21185 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:58.751646 29575 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:58.752076 29575 raft_consensus.cc:493] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.752439 29575 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:58.761395 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:58.762089 21213 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 26.
W20260709 13:09:58.769346 21185 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:58.773655 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
W20260709 13:09:58.774827 21765 leader_election.cc:336] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.777470 29586 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:58.777935 29586 raft_consensus.cc:493] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.778306 29586 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:58.746855 29601 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.789691 29601 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:58.790215 29586 raft_consensus.cc:515] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.792646 29586 leader_election.cc:290] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:09:58.803794 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3" is_pre_election: true
I20260709 13:09:58.804525 21213 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 23.
I20260709 13:09:58.812996 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 
I20260709 13:09:58.814612 29586 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:58.814994 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c918605cdf4c39b832a695968308a3" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:58.800005 29575 raft_consensus.cc:515] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.815608 21213 raft_consensus.cc:3060] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:58.815042 29586 raft_consensus.cc:493] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:58.817093 29586 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:58.824465 21213 raft_consensus.cc:2468] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 27.
W20260709 13:09:58.831205 21185 consensus_peers.cc:597] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:58.833444 29575 leader_election.cc:290] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:09:58.835041 21792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5196cb9fb3c1488180a6421083b18ff9" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb"
I20260709 13:09:58.835652 21792 raft_consensus.cc:3060] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:58.835778 21765 consensus_peers.cc:597] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:58.845916 21185 leader_election.cc:336] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.849802 21185 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:58.855921 21792 raft_consensus.cc:2468] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
I20260709 13:09:58.858107 21184 leader_election.cc:304] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:58.859335 29575 raft_consensus.cc:2804] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:58.859766 29586 raft_consensus.cc:515] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.860832 29575 raft_consensus.cc:697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Becoming Leader. State: Replica: 46c1676121b64b7d9bcf2227cc639dd3, State: Running, Role: LEADER
I20260709 13:09:58.861919 29575 consensus_queue.cc:237] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.862462 29586 leader_election.cc:290] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Requested vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:09:58.870536 21765 leader_election.cc:336] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:09:58.871490 21765 leader_election.cc:336] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.882735 21765 leader_election.cc:304] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:58.883729 29586 raft_consensus.cc:2804] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:09:58.884352 29586 raft_consensus.cc:697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 27 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:58.885463 29586 consensus_queue.cc:237] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.889835 21213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fbcef88f59d4f9ea0f01023c7265069" candidate_uuid: "b70fc04de37e4239a8336534f0c9c0fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "46c1676121b64b7d9bcf2227cc639dd3"
I20260709 13:09:58.890501 21213 raft_consensus.cc:3060] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:58.889999 21765 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:58.902480 21213 raft_consensus.cc:2468] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70fc04de37e4239a8336534f0c9c0fb in term 24.
W20260709 13:09:58.910684 21767 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:58.924214 21765 leader_election.cc:336] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:09:58.933617 21792 raft_consensus.cc:1275] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:09:58.935451 29575 consensus_queue.cc:1048] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:09:58.938272 21765 leader_election.cc:304] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:09:58.939538 29601 raft_consensus.cc:2804] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:58.940022 29601 raft_consensus.cc:697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Becoming Leader. State: Replica: b70fc04de37e4239a8336534f0c9c0fb, State: Running, Role: LEADER
I20260709 13:09:58.941004 29601 consensus_queue.cc:237] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:09:58.937817 20962 catalog_manager.cc:5697] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 reported cstate change: term changed from 17 to 18, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195). New cstate: current_term: 18 leader_uuid: "46c1676121b64b7d9bcf2227cc639dd3" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:58.946156 20963 catalog_manager.cc:5697] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 26 to 27, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 27 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:58.962129 21185 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:58.978377 21792 raft_consensus.cc:1275] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:09:58.980098 29575 consensus_queue.cc:1048] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:09:59.005358 21185 consensus_peers.cc:597] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.028483 21185 consensus_peers.cc:597] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:59.043602 20963 catalog_manager.cc:5697] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb reported cstate change: term changed from 23 to 24, leader changed from 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194) to b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196). New cstate: current_term: 24 leader_uuid: "b70fc04de37e4239a8336534f0c9c0fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:59.049335 21185 consensus_peers.cc:597] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.071959 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:59.072932 21765 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.085860 21765 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.111320 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:09:59.135167 21765 consensus_peers.cc:597] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.159991 21765 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.185302 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:59.188192 21185 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.205807 21185 consensus_peers.cc:597] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.211472 21185 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.222582 21765 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.270147 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:09:59.296201 21765 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.303042 21185 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.303802 21185 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.307610 21187 consensus_peers.cc:597] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:09:59.354815 21765 consensus_peers.cc:597] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:59.394781 21185 consensus_peers.cc:597] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:59.400739 21221 raft_consensus.cc:1275] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 13:09:59.410502 21765 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.424213 21185 consensus_peers.cc:597] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:59.429987 21767 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:59.440896 29601 consensus_queue.cc:1048] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:09:59.465318 19967 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 21168)
I20260709 13:09:59.489552 21213 raft_consensus.cc:1275] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Refusing update from remote peer b70fc04de37e4239a8336534f0c9c0fb: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:09:59.491392 21792 raft_consensus.cc:1275] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Refusing update from remote peer 46c1676121b64b7d9bcf2227cc639dd3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:09:59.493286 29562 consensus_queue.cc:1048] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:59.510349 29601 consensus_queue.cc:1048] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:09:59.538967 21765 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.539548 21765 consensus_peers.cc:597] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:59.571512 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:09:59.638396 29562 raft_consensus.cc:493] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:59.639012 29562 raft_consensus.cc:515] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:59.648005 21765 consensus_peers.cc:597] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:59.652387 29562 leader_election.cc:290] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:59.704589 21187 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:59.714995 21187 consensus_peers.cc:597] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:09:59.766021 29592 raft_consensus.cc:493] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:59.766652 29592 raft_consensus.cc:515] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:59.769731 29592 leader_election.cc:290] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:59.777346 21187 leader_election.cc:336] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:59.802582 21187 consensus_peers.cc:597] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:09:59.805671 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:09:59.808357 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:09:59.812673 21767 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:09:59.834357 29592 raft_consensus.cc:493] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:59.835067 29592 raft_consensus.cc:515] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:09:59.845715 21187 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:59.852427 29562 raft_consensus.cc:493] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:59.853014 29562 raft_consensus.cc:515] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:59.855170 29562 leader_election.cc:290] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:09:59.860664 29592 leader_election.cc:290] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:59.868889 21187 leader_election.cc:336] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:09:59.876988 29575 raft_consensus.cc:493] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:59.877585 29575 raft_consensus.cc:515] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:09:59.881608 29575 leader_election.cc:290] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:09:59.886488 21187 leader_election.cc:336] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:09:59.917922 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:09:59.978303 21767 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111) [suppressed 588 similar messages]
I20260709 13:09:59.999040 29627 raft_consensus.cc:493] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:09:59.999749 29627 raft_consensus.cc:515] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
W20260709 13:10:00.013712 21187 leader_election.cc:336] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.014583 29627 leader_election.cc:290] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.019218 21187 consensus_peers.cc:597] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:10:00.061710 29627 raft_consensus.cc:493] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.062315 29627 raft_consensus.cc:515] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.065536 29592 raft_consensus.cc:493] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.066131 29592 raft_consensus.cc:515] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } }
I20260709 13:10:00.071595 29592 leader_election.cc:290] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.072691 21187 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.076313 29627 leader_election.cc:290] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.079198 21187 leader_election.cc:336] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.131733 29627 raft_consensus.cc:493] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.132354 29627 raft_consensus.cc:515] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.139814 29627 leader_election.cc:290] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.155021 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:10:00.155709 29592 raft_consensus.cc:493] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.157862 29592 raft_consensus.cc:515] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.156368 29562 raft_consensus.cc:493] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
W20260709 13:10:00.157163 21187 leader_election.cc:336] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.161002 29562 raft_consensus.cc:515] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.163949 29562 leader_election.cc:290] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:10:00.165764 29592 leader_election.cc:290] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.181069 21187 consensus_peers.cc:597] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:00.188539 21187 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:00.189180 21187 leader_election.cc:336] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.218380 29575 raft_consensus.cc:493] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.218904 29575 raft_consensus.cc:515] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.220999 29575 leader_election.cc:290] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.228361 21187 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.270351 29592 raft_consensus.cc:493] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.271064 29592 raft_consensus.cc:515] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.277709 29592 leader_election.cc:290] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.281342 21187 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.335080 29592 raft_consensus.cc:493] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.349396 29592 raft_consensus.cc:515] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.353937 29592 leader_election.cc:290] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.353462 21187 consensus_peers.cc:597] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:10:00.368198 29592 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.369050 29592 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:00.373207 29592 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), b70fc04de37e4239a8336534f0c9c0fb (127.19.127.196:33295)
I20260709 13:10:00.373564 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6dd6862dd14baba4a82c5a837bc802" candidate_uuid: "46c1676121b64b7d9bcf2227cc639dd3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b70fc04de37e4239a8336534f0c9c0fb" is_pre_election: true
I20260709 13:10:00.374318 21795 raft_consensus.cc:2468] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c1676121b64b7d9bcf2227cc639dd3 in term 18.
W20260709 13:10:00.375711 21187 consensus_peers.cc:597] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:00.376256 21187 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:00.388954 21187 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:00.389597 21187 consensus_peers.cc:597] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:10:00.442130 29624 raft_consensus.cc:493] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.442816 29624 raft_consensus.cc:515] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:00.444893 29592 raft_consensus.cc:493] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.449697 29592 raft_consensus.cc:515] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.455392 29575 raft_consensus.cc:493] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.455955 29575 raft_consensus.cc:515] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.457926 29575 leader_election.cc:290] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.459225 21767 leader_election.cc:336] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.463400 29592 leader_election.cc:290] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
I20260709 13:10:00.464056 29624 leader_election.cc:290] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.477945 21765 leader_election.cc:336] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.486006 21765 leader_election.cc:304] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.487455 29624 raft_consensus.cc:2749] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:10:00.495769 29592 raft_consensus.cc:493] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.497093 29592 raft_consensus.cc:515] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.499630 29592 leader_election.cc:290] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.542702 21185 leader_election.cc:336] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.543351 29592 raft_consensus.cc:493] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.564419 29592 raft_consensus.cc:515] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.566679 29592 leader_election.cc:290] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.544691 21187 leader_election.cc:336] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:00.567580 21187 leader_election.cc:336] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.570410 21185 leader_election.cc:304] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.575804 29592 raft_consensus.cc:2749] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:10:00.577134 21185 leader_election.cc:336] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.584460 21185 leader_election.cc:304] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.586655 29575 raft_consensus.cc:2749] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:00.588979 21185 leader_election.cc:336] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.581610 29592 raft_consensus.cc:493] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:00.590615 29592 raft_consensus.cc:515] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } }
I20260709 13:10:00.594431 29592 leader_election.cc:290] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351)
W20260709 13:10:00.583088 21187 consensus_peers.cc:597] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:00.603818 21187 leader_election.cc:336] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:00.604455 21187 leader_election.cc:336] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:00.607424 21185 leader_election.cc:304] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.624805 29592 raft_consensus.cc:2749] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:10:00.625876 21185 leader_election.cc:336] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.627463 21185 leader_election.cc:304] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.628233 29592 raft_consensus.cc:2749] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:10:00.636327 21185 leader_election.cc:336] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.652750 21185 leader_election.cc:304] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.655422 29592 raft_consensus.cc:2749] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:00.658064 21187 consensus_peers.cc:597] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:00.659273 21187 leader_election.cc:336] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:00.658645 21185 leader_election.cc:336] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.660527 21185 leader_election.cc:304] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.661280 29592 raft_consensus.cc:2749] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:00.662175 21185 leader_election.cc:336] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.664783 21185 leader_election.cc:304] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.666810 29592 raft_consensus.cc:2749] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:00.667970 21185 leader_election.cc:336] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.670131 21185 leader_election.cc:304] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.671001 29592 raft_consensus.cc:2749] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:10:00.672058 21185 leader_election.cc:336] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.672520 21185 leader_election.cc:304] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.673228 29592 raft_consensus.cc:2749] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:00.681502 21185 leader_election.cc:336] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.682088 21185 leader_election.cc:304] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.682878 29592 raft_consensus.cc:2749] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 13:10:00.683920 21185 leader_election.cc:336] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.684381 21185 leader_election.cc:304] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.685091 29592 raft_consensus.cc:2749] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:10:00.686326 21185 leader_election.cc:336] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:10:00.693079 21767 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:10:00.687027 21185 leader_election.cc:304] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.694291 29592 raft_consensus.cc:2749] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:10:00.695523 21185 leader_election.cc:336] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.696008 21185 leader_election.cc:304] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.696734 29592 raft_consensus.cc:2749] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:00.699426 21185 leader_election.cc:336] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.771152 21185 leader_election.cc:304] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.772351 29592 raft_consensus.cc:2749] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:10:00.773370 21185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39351, user_credentials={real_user=slave}} blocked reactor thread for 73972.5us
W20260709 13:10:00.773919 21185 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.774339 21185 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3, b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.775139 29592 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:10:00.776332 21185 leader_election.cc:336] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.776885 21185 leader_election.cc:304] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.777608 29592 raft_consensus.cc:2749] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:00.778939 21185 leader_election.cc:336] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.779431 21185 leader_election.cc:304] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.780189 29592 raft_consensus.cc:2749] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:00.781250 21185 leader_election.cc:336] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.781718 21185 leader_election.cc:304] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.782421 29592 raft_consensus.cc:2749] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:10:00.835379 19967 tablet_server.cc:196] TabletServer@127.19.127.194:0 shutdown complete.
W20260709 13:10:00.838883 21185 leader_election.cc:336] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.839569 21185 leader_election.cc:304] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.840436 29592 raft_consensus.cc:2749] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:00.843263 21185 leader_election.cc:336] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:00.843926 21185 leader_election.cc:304] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46c1676121b64b7d9bcf2227cc639dd3; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:00.844766 29592 raft_consensus.cc:2749] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:10:00.860818 21765 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:01.021205 19967 tablet_server.cc:179] TabletServer@127.19.127.195:0 shutting down...
W20260709 13:10:01.055593 21765 consensus_peers.cc:597] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.059944 21765 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.073271 21187 consensus_peers.cc:597] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:01.074282 21765 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.076064 21765 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.079905 21187 consensus_peers.cc:597] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:01.091178 21765 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.092609 21765 consensus_peers.cc:597] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.115746 21765 consensus_peers.cc:597] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:01.128538 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:01.129652 19967 tablet_replica.cc:333] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.130756 19967 raft_consensus.cc:2243] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.132090 19967 raft_consensus.cc:2272] T b59f0de1289c4c98ae7ce302ca584a30 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.135349 19967 tablet_replica.cc:333] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.136340 19967 raft_consensus.cc:2243] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.137584 19967 raft_consensus.cc:2272] T 3d158af381004432b3f3f21e53fd19ac P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.140425 19967 tablet_replica.cc:333] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.141357 19967 raft_consensus.cc:2243] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.142014 19967 raft_consensus.cc:2272] T d524f5c1a9e540c9946dd77d8f971ef1 P 46c1676121b64b7d9bcf2227cc639dd3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.144979 19967 tablet_replica.cc:333] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.145833 19967 raft_consensus.cc:2243] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.146615 19967 raft_consensus.cc:2272] T 2e5425aa9a9a4fba958ae97f007977d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.149433 19967 tablet_replica.cc:333] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.150226 19967 raft_consensus.cc:2243] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.157385 19967 raft_consensus.cc:2272] T 2875dc050a244109a0287cc6dafa8431 P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.160738 19967 tablet_replica.cc:333] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.161872 19967 raft_consensus.cc:2243] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.163097 19967 raft_consensus.cc:2272] T 79a87917a1b44746a96a84470b77ea16 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.166625 19967 tablet_replica.cc:333] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.168020 19967 raft_consensus.cc:2243] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.169351 19967 raft_consensus.cc:2272] T 12e3a58d24d64816bce2752c6adb7e33 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:01.171470 21765 consensus_peers.cc:597] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:01.175712 19967 tablet_replica.cc:333] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.176925 19967 raft_consensus.cc:2243] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.177695 19967 raft_consensus.cc:2272] T 6b5d4dd9fad84e018bc7168755b0ba6b P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.189157 19967 tablet_replica.cc:333] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.190111 19967 raft_consensus.cc:2243] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.192318 19967 raft_consensus.cc:2272] T 6beb666b5d504215a141add24a6d632f P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.198794 19967 tablet_replica.cc:333] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.199795 19967 raft_consensus.cc:2243] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.201247 19967 raft_consensus.cc:2272] T 9f10495397de45019165a00ba7c4ec29 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.204638 19967 tablet_replica.cc:333] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.206416 19967 raft_consensus.cc:2243] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.208314 19967 raft_consensus.cc:2272] T 5ffee3ff6d914b9a8bf36e54cc2da28d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.212531 19967 tablet_replica.cc:333] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.213469 19967 raft_consensus.cc:2243] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.214784 19967 raft_consensus.cc:2272] T e30c07cef4334b45ade426b26069b26c P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.217724 19967 tablet_replica.cc:333] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.218475 19967 raft_consensus.cc:2243] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.219193 19967 raft_consensus.cc:2272] T a0de4eb0c8e649f09be36671e6dcc4cb P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.222151 19967 tablet_replica.cc:333] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.222990 19967 raft_consensus.cc:2243] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.223655 19967 raft_consensus.cc:2272] T 77143ea923a34396a4cbc31f42dd0a18 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.226507 19967 tablet_replica.cc:333] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.227444 19967 raft_consensus.cc:2243] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.228412 19967 raft_consensus.cc:2272] T 72edcb8f7f3d437b8e7bb24c612600ab P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.231776 19967 tablet_replica.cc:333] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.232764 19967 raft_consensus.cc:2243] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.233476 19967 raft_consensus.cc:2272] T 3d51215a492444539c4905a13bd0c12d P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.236819 19967 tablet_replica.cc:333] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.237794 19967 raft_consensus.cc:2243] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.238641 19967 raft_consensus.cc:2272] T 752b0641d0a3410795a788ef8dc8e035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.241796 19967 tablet_replica.cc:333] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.242820 19967 raft_consensus.cc:2243] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.243657 19967 raft_consensus.cc:2272] T d4af1b94141049e0a78a816c8ba31d72 P 46c1676121b64b7d9bcf2227cc639dd3 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.246901 19967 tablet_replica.cc:333] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.247879 19967 raft_consensus.cc:2243] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.248668 19967 raft_consensus.cc:2272] T 080f4010b39e40a8b8efb7280faec38f P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.251986 19967 tablet_replica.cc:333] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.252810 19967 raft_consensus.cc:2243] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.253588 19967 raft_consensus.cc:2272] T c20ca56847f64504a85694d10f47558e P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.256508 19967 tablet_replica.cc:333] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.257323 19967 raft_consensus.cc:2243] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.258347 19967 raft_consensus.cc:2272] T 9ae1ecaeba734906b89cfdc36c6abaf9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:01.258571 21767 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:10:01.261229 19967 tablet_replica.cc:333] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.262257 19967 raft_consensus.cc:2243] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.263055 19967 raft_consensus.cc:2272] T 0d78b35ac92d402db8a5e3fe45be52b5 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.266583 19967 tablet_replica.cc:333] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.267611 19967 raft_consensus.cc:2243] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.268548 19967 raft_consensus.cc:2272] T 53c918605cdf4c39b832a695968308a3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.272754 19967 tablet_replica.cc:333] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.273723 19967 raft_consensus.cc:2243] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.274498 19967 raft_consensus.cc:2272] T de53f43e2a0e4a91a1767df521fc2b8c P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.277453 19967 tablet_replica.cc:333] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.278350 19967 raft_consensus.cc:2243] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.279206 19967 raft_consensus.cc:2272] T 731bbc776723436d954fe0b2a30a5b09 P 46c1676121b64b7d9bcf2227cc639dd3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.282418 19967 tablet_replica.cc:333] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.283272 19967 raft_consensus.cc:2243] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.284036 19967 raft_consensus.cc:2272] T 40d1baf8579949fbb0d5d45a18fefe35 P 46c1676121b64b7d9bcf2227cc639dd3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.286736 19967 tablet_replica.cc:333] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.287622 19967 raft_consensus.cc:2243] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.288975 19967 raft_consensus.cc:2272] T d693d94d4c0943d9ab08114c33fc2e7f P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.292364 19967 tablet_replica.cc:333] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.293275 19967 raft_consensus.cc:2243] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.294073 19967 raft_consensus.cc:2272] T d8a4c442e73243eda44d43ba0eaf0d2a P 46c1676121b64b7d9bcf2227cc639dd3 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.297435 19967 tablet_replica.cc:333] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.298406 19967 raft_consensus.cc:2243] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.300392 19967 raft_consensus.cc:2272] T 5a16f225ecc04514a485518bcd2b8112 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.303510 19967 tablet_replica.cc:333] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.304405 19967 raft_consensus.cc:2243] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.305138 19967 raft_consensus.cc:2272] T 8b6dd6862dd14baba4a82c5a837bc802 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.308395 19967 tablet_replica.cc:333] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.309970 19967 raft_consensus.cc:2243] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.312171 19967 raft_consensus.cc:2272] T 6e4456f3a9364c7289dfce3163ab4760 P 46c1676121b64b7d9bcf2227cc639dd3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.316442 19967 tablet_replica.cc:333] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.317260 19967 raft_consensus.cc:2243] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.320145 19967 raft_consensus.cc:2272] T de27148f295b4ef5a3f79084b8ed5ae6 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.323369 19967 tablet_replica.cc:333] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.324303 19967 raft_consensus.cc:2243] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.325781 19967 raft_consensus.cc:2272] T e714d9095b9c430bba199e0209c0b946 P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.328977 19967 tablet_replica.cc:333] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.329789 19967 raft_consensus.cc:2243] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.330415 19967 raft_consensus.cc:2272] T 169e564bd720422bb798c652af4fb2eb P 46c1676121b64b7d9bcf2227cc639dd3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.333349 19967 tablet_replica.cc:333] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.334137 19967 raft_consensus.cc:2243] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.334854 19967 raft_consensus.cc:2272] T 1ea0f23227964b96bff6287436e970cd P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.337740 19967 tablet_replica.cc:333] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.338510 19967 raft_consensus.cc:2243] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.339790 19967 raft_consensus.cc:2272] T ce55364f10c348819e7c63c08633b184 P 46c1676121b64b7d9bcf2227cc639dd3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.342353 19967 tablet_replica.cc:333] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.343084 19967 raft_consensus.cc:2243] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.344432 19967 raft_consensus.cc:2272] T e8c9b46a490540a2bb0a90f7bfbd1e53 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.347370 19967 tablet_replica.cc:333] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.348258 19967 raft_consensus.cc:2243] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.348866 19967 raft_consensus.cc:2272] T 3ca4319420e54bc19f621da03c811035 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.361318 19967 tablet_replica.cc:333] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.362223 19967 raft_consensus.cc:2243] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.363698 19967 raft_consensus.cc:2272] T 9417f68ba15d49c6915ab46793aab93c P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.366709 19967 tablet_replica.cc:333] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.367537 19967 raft_consensus.cc:2243] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.368777 19967 raft_consensus.cc:2272] T 5d5d573fe0fd4b91af3ea4dd4f6dc7f3 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.371531 19967 tablet_replica.cc:333] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.372278 19967 raft_consensus.cc:2243] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.373984 19967 raft_consensus.cc:2272] T 5196cb9fb3c1488180a6421083b18ff9 P 46c1676121b64b7d9bcf2227cc639dd3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.376889 19967 tablet_replica.cc:333] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.377748 19967 raft_consensus.cc:2243] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.378379 19967 raft_consensus.cc:2272] T 2fbcef88f59d4f9ea0f01023c7265069 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.381279 19967 tablet_replica.cc:333] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.382014 19967 raft_consensus.cc:2243] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.382645 19967 raft_consensus.cc:2272] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.385581 19967 tablet_replica.cc:333] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.386327 19967 raft_consensus.cc:2243] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.387544 19967 raft_consensus.cc:2272] T 293026cede06406bad9e7b5fc490abe2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.390338 19967 tablet_replica.cc:333] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.391062 19967 raft_consensus.cc:2243] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.394207 19967 raft_consensus.cc:2272] T 41268ef73d2c487dab473150fa26d73e P 46c1676121b64b7d9bcf2227cc639dd3 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:01.395416 21187 consensus_peers.cc:597] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:01.397121 21765 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:01.397137 19967 tablet_replica.cc:333] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
W20260709 13:10:01.398030 21765 consensus_peers.cc:597] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:01.398203 19967 raft_consensus.cc:2243] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.398917 19967 raft_consensus.cc:2272] T 54fc5a3f4db84eac831870d621ddfa54 P 46c1676121b64b7d9bcf2227cc639dd3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.402653 19967 tablet_replica.cc:333] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.403537 19967 raft_consensus.cc:2243] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.404199 19967 raft_consensus.cc:2272] T 47de8d16a3804364a7fbc9bf598946c2 P 46c1676121b64b7d9bcf2227cc639dd3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.407330 19967 tablet_replica.cc:333] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.408079 19967 raft_consensus.cc:2243] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.409614 19967 raft_consensus.cc:2272] T b7a0bb40f8e3428899935fdfa915dfd0 P 46c1676121b64b7d9bcf2227cc639dd3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.412963 19967 tablet_replica.cc:333] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3: stopping tablet replica
I20260709 13:10:01.413614 19967 raft_consensus.cc:2243] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.415035 19967 raft_consensus.cc:2272] T a473266451804da089da06e9c927e7ee P 46c1676121b64b7d9bcf2227cc639dd3 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:01.418365 21765 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:01.457481 21765 consensus_peers.cc:597] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Couldn't send request to peer 46c1676121b64b7d9bcf2227cc639dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.528414 21767 consensus_peers.cc:597] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:01.530236 21765 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:01.601323 21767 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:10:01.622015 21765 consensus_peers.cc:597] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:01.638653 21765 consensus_peers.cc:597] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:01.646248 21765 consensus_peers.cc:597] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:01.655707 21765 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:01.692778 21765 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:01.707944 21765 consensus_peers.cc:597] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:01.736176 21767 consensus_peers.cc:597] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:10:01.800032 21767 consensus_peers.cc:597] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:10:01.841537 21767 consensus_peers.cc:597] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:10:01.842346 21765 consensus_peers.cc:597] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:01.951645 29624 raft_consensus.cc:493] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5e5e65640c8b4f5ca7eadc6a14312c55)
I20260709 13:10:01.952296 29624 raft_consensus.cc:515] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:01.958349 29624 leader_election.cc:290] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:10:01.996219 21767 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:01.997920 21765 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.001888 21765 leader_election.cc:304] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:02.007682 29624 raft_consensus.cc:2749] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:10:02.009873 21765 leader_election.cc:336] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:10:02.011962 21767 consensus_peers.cc:597] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:02.020946 21765 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:02.067437 21767 consensus_peers.cc:597] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:10:02.151360 21765 consensus_peers.cc:597] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:02.256959 21767 consensus_peers.cc:597] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:10:02.267220 29624 raft_consensus.cc:493] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.267899 29624 raft_consensus.cc:515] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.272493 29624 leader_election.cc:290] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:10:02.280966 21767 leader_election.cc:336] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:02.299667 29628 raft_consensus.cc:493] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.300307 29628 raft_consensus.cc:515] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.302567 29628 leader_election.cc:290] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:10:02.332113 21767 leader_election.cc:336] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:02.338871 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5196cb9fb3c1488180a6421083b18ff9: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:02.336180 21765 consensus_peers.cc:597] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Couldn't send request to peer 5e5e65640c8b4f5ca7eadc6a14312c55. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:02.340533 21765 leader_election.cc:336] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.341405 21765 leader_election.cc:304] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:02.342305 29624 raft_consensus.cc:2749] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:02.346319 29624 raft_consensus.cc:493] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.346941 29624 raft_consensus.cc:515] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.350198 29624 leader_election.cc:290] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:10:02.357569 21767 leader_election.cc:336] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:02.365103 21765 leader_election.cc:336] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.365721 21765 leader_election.cc:304] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:02.366547 29624 raft_consensus.cc:2749] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:10:02.390647 21765 leader_election.cc:336] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:02.391258 21765 leader_election.cc:304] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:02.392113 29624 raft_consensus.cc:2749] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:02.394232 21765 leader_election.cc:336] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.406675 29624 raft_consensus.cc:493] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.407384 29624 raft_consensus.cc:515] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.409777 29624 leader_election.cc:290] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:10:02.427955 29624 raft_consensus.cc:493] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.428594 29624 raft_consensus.cc:515] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } attrs { replace: true } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
W20260709 13:10:02.431639 21767 consensus_peers.cc:597] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:10:02.436882 29624 leader_election.cc:290] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:10:02.438772 21767 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:02.446837 21765 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:10:02.447751 21765 leader_election.cc:336] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.449275 21765 leader_election.cc:304] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:10:02.450115 29624 raft_consensus.cc:2749] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:10:02.462291 21767 consensus_peers.cc:597] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:02.472792 21765 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:02.473469 21765 leader_election.cc:304] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 5e5e65640c8b4f5ca7eadc6a14312c55, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:02.474277 29624 raft_consensus.cc:2749] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:10:02.475556 21765 leader_election.cc:336] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
W20260709 13:10:02.518850 21767 consensus_peers.cc:597] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:10:02.553103 19967 tablet_server.cc:196] TabletServer@127.19.127.195:0 shutdown complete.
W20260709 13:10:02.573307 21767 consensus_peers.cc:597] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb -> Peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Couldn't send request to peer 9384358ce43f4e3fac2ae44bdf11c389. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:10:02.586491 29628 raft_consensus.cc:493] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.587184 29628 raft_consensus.cc:515] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.590281 29628 leader_election.cc:290] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:10:02.624845 29628 raft_consensus.cc:493] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.625541 29628 raft_consensus.cc:515] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.628778 29628 leader_election.cc:290] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:10:02.631942 21765 leader_election.cc:336] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:10:02.635345 21765 leader_election.cc:336] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.635922 21765 leader_election.cc:304] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:10:02.636847 29628 raft_consensus.cc:2749] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:02.679344 19967 tablet_server.cc:179] TabletServer@127.19.127.196:0 shutting down...
W20260709 13:10:02.679468 21765 leader_election.cc:336] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
I20260709 13:10:02.684027 29628 raft_consensus.cc:493] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.684653 29628 raft_consensus.cc:515] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.686844 29628 leader_election.cc:290] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:10:02.687572 21765 leader_election.cc:336] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.688097 21765 leader_election.cc:304] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:10:02.688892 29628 raft_consensus.cc:2749] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:10:02.695999 29628 raft_consensus.cc:493] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.696561 29628 raft_consensus.cc:515] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.699980 29624 raft_consensus.cc:493] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.700562 29624 raft_consensus.cc:515] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5e5e65640c8b4f5ca7eadc6a14312c55" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39351 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.703863 29628 leader_election.cc:290] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:10:02.707165 29624 leader_election.cc:290] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
W20260709 13:10:02.707612 21767 leader_election.cc:336] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:02.712321 21765 leader_election.cc:336] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:10:02.713222 21765 leader_election.cc:336] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.713670 21765 leader_election.cc:304] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:10:02.714481 29624 raft_consensus.cc:2749] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:10:02.727754 21765 leader_election.cc:336] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5e5e65640c8b4f5ca7eadc6a14312c55 (127.19.127.194:39351): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39351: connect: Connection refused (error 111)
W20260709 13:10:02.729138 21765 leader_election.cc:336] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.729691 21765 leader_election.cc:304] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:02.733065 29624 raft_consensus.cc:2749] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:02.734175 21765 leader_election.cc:336] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.734685 21765 leader_election.cc:304] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 5e5e65640c8b4f5ca7eadc6a14312c55
I20260709 13:10:02.735491 29624 raft_consensus.cc:2749] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:10:02.753684 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:02.756891 19967 tablet_replica.cc:333] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.758028 19967 raft_consensus.cc:2243] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.758916 19967 raft_consensus.cc:2272] T 41268ef73d2c487dab473150fa26d73e P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.762753 19967 tablet_replica.cc:333] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.763963 19967 raft_consensus.cc:2243] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.764953 19967 raft_consensus.cc:2272] T 6beb666b5d504215a141add24a6d632f P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.768874 19967 tablet_replica.cc:333] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.769855 19967 raft_consensus.cc:2243] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.770581 19967 raft_consensus.cc:2272] T 5196cb9fb3c1488180a6421083b18ff9 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.773936 29624 raft_consensus.cc:493] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 46c1676121b64b7d9bcf2227cc639dd3)
I20260709 13:10:02.774518 29624 raft_consensus.cc:515] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9384358ce43f4e3fac2ae44bdf11c389" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 42843 } } peers { permanent_uuid: "46c1676121b64b7d9bcf2227cc639dd3" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 46249 } } peers { permanent_uuid: "b70fc04de37e4239a8336534f0c9c0fb" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 33295 } attrs { promote: false } }
I20260709 13:10:02.775456 19967 tablet_replica.cc:333] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.776700 19967 raft_consensus.cc:2243] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.777803 29624 leader_election.cc:290] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843), 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249)
I20260709 13:10:02.778131 19967 raft_consensus.cc:2272] T cc53d52fdeaa441b9e9d329e69d94440 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.782399 19967 tablet_replica.cc:333] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.783442 19967 raft_consensus.cc:2243] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.784724 19967 raft_consensus.cc:2272] T b8d00e740dba47b2b43be9e4ef1ecb55 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:02.786012 21767 leader_election.cc:336] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:02.788903 19967 tablet_replica.cc:333] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.795718 19967 raft_consensus.cc:2243] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.797572 19967 raft_consensus.cc:2272] T 0d78b35ac92d402db8a5e3fe45be52b5 P b70fc04de37e4239a8336534f0c9c0fb [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:02.799770 21765 leader_election.cc:336] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 46c1676121b64b7d9bcf2227cc639dd3 (127.19.127.195:46249): Network error: Client connection negotiation failed: client connection to 127.19.127.195:46249: connect: Connection refused (error 111)
I20260709 13:10:02.800450 21765 leader_election.cc:304] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b70fc04de37e4239a8336534f0c9c0fb; no voters: 46c1676121b64b7d9bcf2227cc639dd3, 9384358ce43f4e3fac2ae44bdf11c389
I20260709 13:10:02.801304 29624 raft_consensus.cc:2749] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:10:02.802156 19967 tablet_replica.cc:333] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.803548 19967 raft_consensus.cc:2243] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.805416 19967 raft_consensus.cc:2272] T 9833b15c99c34c269ca3afa7dac3f175 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.814131 19967 tablet_replica.cc:333] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.815214 19967 raft_consensus.cc:2243] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.816092 19967 raft_consensus.cc:2272] T 2875dc050a244109a0287cc6dafa8431 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.819415 19967 tablet_replica.cc:333] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.820320 19967 raft_consensus.cc:2243] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.821650 19967 raft_consensus.cc:2272] T c80e14318ee44feba007795498031464 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.825632 19967 tablet_replica.cc:333] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.826591 19967 raft_consensus.cc:2243] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.827946 19967 raft_consensus.cc:2272] T 12e3a58d24d64816bce2752c6adb7e33 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.831398 19967 tablet_replica.cc:333] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.832451 19967 raft_consensus.cc:2243] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.833748 19967 raft_consensus.cc:2272] T a0de4eb0c8e649f09be36671e6dcc4cb P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.837163 19967 tablet_replica.cc:333] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.838191 19967 raft_consensus.cc:2243] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.841594 19967 raft_consensus.cc:2272] T 8b6dd6862dd14baba4a82c5a837bc802 P b70fc04de37e4239a8336534f0c9c0fb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.845913 19967 tablet_replica.cc:333] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.847043 19967 raft_consensus.cc:2243] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.848325 19967 raft_consensus.cc:2272] T 169e564bd720422bb798c652af4fb2eb P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.851532 19967 tablet_replica.cc:333] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.852445 19967 raft_consensus.cc:2243] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.853623 19967 raft_consensus.cc:2272] T e0631ac443bb4e6dbbbc599dd4b67d2e P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.856781 19967 tablet_replica.cc:333] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.857674 19967 raft_consensus.cc:2243] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.858848 19967 raft_consensus.cc:2272] T 2fbcef88f59d4f9ea0f01023c7265069 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.861807 19967 tablet_replica.cc:333] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.862643 19967 raft_consensus.cc:2243] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.863888 19967 raft_consensus.cc:2272] T 9028fa09ccd24f5494e4e5de5a8e6e18 P b70fc04de37e4239a8336534f0c9c0fb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.866842 19967 tablet_replica.cc:333] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.867748 19967 raft_consensus.cc:2243] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.868515 19967 raft_consensus.cc:2272] T e8c9b46a490540a2bb0a90f7bfbd1e53 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.872437 19967 tablet_replica.cc:333] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.873378 19967 raft_consensus.cc:2243] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.874050 19967 raft_consensus.cc:2272] T 223e82a164f445c2b46667d58a5d2d1f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.877022 19967 tablet_replica.cc:333] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.877863 19967 raft_consensus.cc:2243] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.878548 19967 raft_consensus.cc:2272] T e30c07cef4334b45ade426b26069b26c P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.881498 19967 tablet_replica.cc:333] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.882270 19967 raft_consensus.cc:2243] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.883721 19967 raft_consensus.cc:2272] T 1fa2ded676fc4d048c810a5f842100d1 P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.887272 19967 tablet_replica.cc:333] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.888170 19967 raft_consensus.cc:2243] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.889430 19967 raft_consensus.cc:2272] T 53c918605cdf4c39b832a695968308a3 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.892532 19967 tablet_replica.cc:333] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.893410 19967 raft_consensus.cc:2243] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.894397 19967 raft_consensus.cc:2272] T 293026cede06406bad9e7b5fc490abe2 P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.897608 19967 tablet_replica.cc:333] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.898442 19967 raft_consensus.cc:2243] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.899639 19967 raft_consensus.cc:2272] T 3d158af381004432b3f3f21e53fd19ac P b70fc04de37e4239a8336534f0c9c0fb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.902817 19967 tablet_replica.cc:333] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.903795 19967 raft_consensus.cc:2243] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.904544 19967 raft_consensus.cc:2272] T 5a16f225ecc04514a485518bcd2b8112 P b70fc04de37e4239a8336534f0c9c0fb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.907970 19967 tablet_replica.cc:333] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.908797 19967 raft_consensus.cc:2243] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.910059 19967 raft_consensus.cc:2272] T d8a4c442e73243eda44d43ba0eaf0d2a P b70fc04de37e4239a8336534f0c9c0fb [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.913282 19967 tablet_replica.cc:333] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.914137 19967 raft_consensus.cc:2243] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.914789 19967 raft_consensus.cc:2272] T d693d94d4c0943d9ab08114c33fc2e7f P b70fc04de37e4239a8336534f0c9c0fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.917583 19967 tablet_replica.cc:333] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.918429 19967 raft_consensus.cc:2243] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.919596 19967 raft_consensus.cc:2272] T 3d51215a492444539c4905a13bd0c12d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.922616 19967 tablet_replica.cc:333] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.923466 19967 raft_consensus.cc:2243] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.924096 19967 raft_consensus.cc:2272] T b59f0de1289c4c98ae7ce302ca584a30 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.927071 19967 tablet_replica.cc:333] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.927825 19967 raft_consensus.cc:2243] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.928627 19967 raft_consensus.cc:2272] T 5ffee3ff6d914b9a8bf36e54cc2da28d P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.931493 19967 tablet_replica.cc:333] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.932253 19967 raft_consensus.cc:2243] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.933005 19967 raft_consensus.cc:2272] T b7a0bb40f8e3428899935fdfa915dfd0 P b70fc04de37e4239a8336534f0c9c0fb [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.937040 19967 tablet_replica.cc:333] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.937865 19967 raft_consensus.cc:2243] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.939110 19967 raft_consensus.cc:2272] T fdb8a7df9f774ce0afbd5b26fa2a88d2 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.942560 19967 tablet_replica.cc:333] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.943403 19967 raft_consensus.cc:2243] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:02.944026 19967 raft_consensus.cc:2272] T ce55364f10c348819e7c63c08633b184 P b70fc04de37e4239a8336534f0c9c0fb [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.947146 19967 tablet_replica.cc:333] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.947932 19967 raft_consensus.cc:2243] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.949097 19967 raft_consensus.cc:2272] T d4a635811aa248b599c9361b7abf9f32 P b70fc04de37e4239a8336534f0c9c0fb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:02.952241 19967 tablet_replica.cc:333] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb: stopping tablet replica
I20260709 13:10:02.952926 19967 raft_consensus.cc:2243] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:02.954178 19967 raft_consensus.cc:2272] T b4ee2463d8894a95be9b747f813f9106 P b70fc04de37e4239a8336534f0c9c0fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.358538 19967 tablet_server.cc:196] TabletServer@127.19.127.196:0 shutdown complete.
I20260709 13:10:04.420727 19967 master.cc:562] Master@127.19.127.254:37743 shutting down...
W20260709 13:10:04.464608 21018 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:10:04.464915 21018 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:10:04.465130 21018 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:10:13.824673 20957 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111) [suppressed 326 similar messages]
W20260709 13:10:13.828384 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet b7a0bb40f8e3428899935fdfa915dfd0: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
W20260709 13:10:18.751803 20957 catalog_manager.cc:4755] TS 9384358ce43f4e3fac2ae44bdf11c389 (127.19.127.193:42843): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5196cb9fb3c1488180a6421083b18ff9: Network error: Client connection negotiation failed: client connection to 127.19.127.193:42843: connect: Connection refused (error 111)
I20260709 13:10:21.475562 19967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:21.476630 19967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:21.476985 19967 tablet_replica.cc:333] T 00000000000000000000000000000000 P b068e036d3f142f4b255651ac6a0e101: stopping tablet replica
I20260709 13:10:21.636503 19967 master.cc:584] Master@127.19.127.254:37743 shutdown complete.
I20260709 13:10:21.690683 19967 test_util.cc:182] -----------------------------------------------
I20260709 13:10:21.690902 19967 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602314247669-19967-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (283971 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:10:21.694401 19967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.127.254:37437
I20260709 13:10:21.695626 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:21.702042 29641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:21.702462 29643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:21.702071 29640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:21.703408 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:21.706863 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:21.707070 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:21.707191 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602621707180 us; error 0 us; skew 500 ppm
I20260709 13:10:21.707742 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:21.710237 19967 webserver.cc:533] Webserver started at http://127.19.127.254:38691/ using document root <none> and password file <none>
I20260709 13:10:21.710703 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:21.710855 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:21.711066 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:21.712428 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/master-0-root/instance:
uuid: "917e8e70948348e9b76b5d231e54f10e"
format_stamp: "Formatted at 2026-07-09 13:10:21 on dist-test-slave-s3n6"
I20260709 13:10:21.716979 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:10:21.720504 29648 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:21.721306 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:21.721628 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/master-0-root
uuid: "917e8e70948348e9b76b5d231e54f10e"
format_stamp: "Formatted at 2026-07-09 13:10:21 on dist-test-slave-s3n6"
I20260709 13:10:21.721921 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:21.750681 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:21.752005 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:21.791864 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.254:37437
I20260709 13:10:21.791968 29699 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.254:37437 every 8 connection(s)
I20260709 13:10:21.796197 29700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:21.809708 29700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e: Bootstrap starting.
I20260709 13:10:21.815831 29700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.821367 29700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e: No bootstrap required, opened a new log
I20260709 13:10:21.824395 29700 raft_consensus.cc:359] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917e8e70948348e9b76b5d231e54f10e" member_type: VOTER }
I20260709 13:10:21.824971 29700 raft_consensus.cc:385] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.825234 29700 raft_consensus.cc:740] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917e8e70948348e9b76b5d231e54f10e, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.825914 29700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917e8e70948348e9b76b5d231e54f10e" member_type: VOTER }
I20260709 13:10:21.826594 29700 raft_consensus.cc:399] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:21.826975 29700 raft_consensus.cc:493] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:21.827307 29700 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.832085 29700 raft_consensus.cc:515] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917e8e70948348e9b76b5d231e54f10e" member_type: VOTER }
I20260709 13:10:21.832629 29700 leader_election.cc:304] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 917e8e70948348e9b76b5d231e54f10e; no voters: 
I20260709 13:10:21.833742 29700 leader_election.cc:290] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:21.834097 29703 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:21.835505 29703 raft_consensus.cc:697] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 1 LEADER]: Becoming Leader. State: Replica: 917e8e70948348e9b76b5d231e54f10e, State: Running, Role: LEADER
I20260709 13:10:21.836210 29703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917e8e70948348e9b76b5d231e54f10e" member_type: VOTER }
I20260709 13:10:21.836730 29700 sys_catalog.cc:565] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:21.842998 29704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "917e8e70948348e9b76b5d231e54f10e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917e8e70948348e9b76b5d231e54f10e" member_type: VOTER } }
I20260709 13:10:21.843741 29704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:21.843940 29705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 917e8e70948348e9b76b5d231e54f10e. Latest consensus state: current_term: 1 leader_uuid: "917e8e70948348e9b76b5d231e54f10e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "917e8e70948348e9b76b5d231e54f10e" member_type: VOTER } }
I20260709 13:10:21.844501 29705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:21.846998 29712 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:21.852139 29712 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:21.855578 19967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:21.861902 29712 catalog_manager.cc:1383] Generated new cluster ID: f01c5df1da0f43868d74639e1def6d6c
I20260709 13:10:21.862232 29712 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:21.877068 29712 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:21.878367 29712 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:21.891433 29712 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e: Generated new TSK 0
I20260709 13:10:21.892087 29712 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:21.923087 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:21.929878 29721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:21.930395 29722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:21.932757 29724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:21.933396 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:21.937537 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:21.937913 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:21.938153 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602621938125 us; error 0 us; skew 500 ppm
I20260709 13:10:21.938977 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:21.942214 19967 webserver.cc:533] Webserver started at http://127.19.127.193:37711/ using document root <none> and password file <none>
I20260709 13:10:21.942675 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:21.942829 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:21.943034 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:21.944270 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/instance:
uuid: "2ba28923c4534c78903f759f0a4e1265"
format_stamp: "Formatted at 2026-07-09 13:10:21 on dist-test-slave-s3n6"
I20260709 13:10:21.948807 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:10:21.952304 29729 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:21.953184 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:21.953575 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "2ba28923c4534c78903f759f0a4e1265"
format_stamp: "Formatted at 2026-07-09 13:10:21 on dist-test-slave-s3n6"
I20260709 13:10:21.953933 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:21.965548 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:21.966764 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:21.968261 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:21.970642 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:21.970870 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:21.971098 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:21.971342 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.014083 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:35599
I20260709 13:10:22.014194 29791 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:35599 every 8 connection(s)
I20260709 13:10:22.019778 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:22.028179 29796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:22.029866 29797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:22.032380 29799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:22.033905 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:22.034590 29792 heartbeater.cc:344] Connected to a master server at 127.19.127.254:37437
I20260709 13:10:22.035030 29792 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:22.036216 29792 heartbeater.cc:507] Master 127.19.127.254:37437 requested a full tablet report, sending...
I20260709 13:10:22.037264 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:22.037514 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:22.037716 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602622037696 us; error 0 us; skew 500 ppm
I20260709 13:10:22.038532 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:22.039028 29665 ts_manager.cc:194] Registered new tserver with Master: 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:22.041653 19967 webserver.cc:533] Webserver started at http://127.19.127.194:35289/ using document root <none> and password file <none>
I20260709 13:10:22.041791 29665 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59030
I20260709 13:10:22.042400 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:22.042673 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:22.042999 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:22.044327 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/instance:
uuid: "d08b1a18bb214fa1b08560d4d8149deb"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-s3n6"
I20260709 13:10:22.048916 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:10:22.052642 29804 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.053450 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:10:22.053769 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-1-root
uuid: "d08b1a18bb214fa1b08560d4d8149deb"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-s3n6"
I20260709 13:10:22.054031 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:22.070034 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:22.071409 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:22.072898 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:22.075152 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:22.075398 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.075637 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:22.075806 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.118810 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.194:39603
I20260709 13:10:22.118911 29866 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.194:39603 every 8 connection(s)
I20260709 13:10:22.124145 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:22.132284 29870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:22.132447 29871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:22.136183 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:22.136272 29867 heartbeater.cc:344] Connected to a master server at 127.19.127.254:37437
W20260709 13:10:22.136322 29874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:22.137008 29867 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:22.137857 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:22.138159 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:22.138093 29867 heartbeater.cc:507] Master 127.19.127.254:37437 requested a full tablet report, sending...
I20260709 13:10:22.138497 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602622138475 us; error 0 us; skew 500 ppm
I20260709 13:10:22.139319 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:22.140623 29665 ts_manager.cc:194] Registered new tserver with Master: d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:22.141947 19967 webserver.cc:533] Webserver started at http://127.19.127.195:33295/ using document root <none> and password file <none>
I20260709 13:10:22.142297 29665 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59046
I20260709 13:10:22.142480 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:22.142793 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:22.143061 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:22.144680 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/instance:
uuid: "74191176370841578d396cc01f230f06"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-s3n6"
I20260709 13:10:22.149525 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:10:22.152925 29878 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.153767 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:10:22.154083 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-2-root
uuid: "74191176370841578d396cc01f230f06"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-s3n6"
I20260709 13:10:22.154393 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:22.176496 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:22.177837 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:22.179508 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:22.182801 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:22.183071 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.183380 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:22.183561 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.227092 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.195:40257
I20260709 13:10:22.227221 29940 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.195:40257 every 8 connection(s)
I20260709 13:10:22.232359 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:22.240986 29944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:22.241395 29945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:22.244445 29947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:22.244987 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:22.246012 29941 heartbeater.cc:344] Connected to a master server at 127.19.127.254:37437
I20260709 13:10:22.246088 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:22.246505 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:22.246522 29941 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:22.246713 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602622246694 us; error 0 us; skew 500 ppm
I20260709 13:10:22.247545 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:22.247684 29941 heartbeater.cc:507] Master 127.19.127.254:37437 requested a full tablet report, sending...
I20260709 13:10:22.250089 29665 ts_manager.cc:194] Registered new tserver with Master: 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:22.250391 19967 webserver.cc:533] Webserver started at http://127.19.127.196:41247/ using document root <none> and password file <none>
I20260709 13:10:22.250950 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:22.251154 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:22.251462 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:22.251771 29665 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59048
I20260709 13:10:22.252826 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-3-root/instance:
uuid: "75162022240a4fe1a113333e77964ca3"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-s3n6"
I20260709 13:10:22.258170 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:10:22.261829 29952 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.262804 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:10:22.263121 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-3-root
uuid: "75162022240a4fe1a113333e77964ca3"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-s3n6"
I20260709 13:10:22.263474 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:22.288558 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:22.289848 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:22.291405 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:22.293846 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:22.294034 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.294334 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:22.294474 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.339274 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.196:46707
I20260709 13:10:22.339437 30014 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.196:46707 every 8 connection(s)
I20260709 13:10:22.353753 30015 heartbeater.cc:344] Connected to a master server at 127.19.127.254:37437
I20260709 13:10:22.354301 30015 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:22.355206 30015 heartbeater.cc:507] Master 127.19.127.254:37437 requested a full tablet report, sending...
I20260709 13:10:22.357532 29665 ts_manager.cc:194] Registered new tserver with Master: 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:22.357699 19967 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015129083s
I20260709 13:10:22.359012 29665 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59064
I20260709 13:10:22.384910 29665 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59070:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:10:22.645566 29906 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4a836f7bfb414eb5bba33a3be51d4f (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:22.647145 29906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4a836f7bfb414eb5bba33a3be51d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.647032 29979 tablet_service.cc:1511] Processing CreateTablet for tablet b4e73d2d9e7449d4b87ea5ee4324c2c9 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:22.648679 29979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4e73d2d9e7449d4b87ea5ee4324c2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.645116 29978 tablet_service.cc:1511] Processing CreateTablet for tablet 0e543a21ae8243058d010255b40860e9 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:22.650878 29978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e543a21ae8243058d010255b40860e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.650651 29976 tablet_service.cc:1511] Processing CreateTablet for tablet 8405db80d77d4b8c9737d33de3ca1f74 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:22.654050 29977 tablet_service.cc:1511] Processing CreateTablet for tablet 99a9fd11d5a348389207646896abb152 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:22.656600 29905 tablet_service.cc:1511] Processing CreateTablet for tablet 252766f9503842f48be95fdd3e3035c2 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:22.658114 29905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 252766f9503842f48be95fdd3e3035c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.658447 29975 tablet_service.cc:1511] Processing CreateTablet for tablet 45a7caaa2edf4bafacd9da8900ebeeb2 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:22.659857 29904 tablet_service.cc:1511] Processing CreateTablet for tablet 0e543a21ae8243058d010255b40860e9 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:22.661391 29904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e543a21ae8243058d010255b40860e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.652608 29976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8405db80d77d4b8c9737d33de3ca1f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.663247 29903 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7968a480124ac9ad9734c6ffc0dbbb (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:22.676707 29900 tablet_service.cc:1511] Processing CreateTablet for tablet b1ff57a4871b4c2197e7c7f41230d429 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:22.677829 29903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7968a480124ac9ad9734c6ffc0dbbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.664635 29902 tablet_service.cc:1511] Processing CreateTablet for tablet 8405db80d77d4b8c9737d33de3ca1f74 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:22.642271 29980 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4a836f7bfb414eb5bba33a3be51d4f (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:22.679546 29902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8405db80d77d4b8c9737d33de3ca1f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.680037 29977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99a9fd11d5a348389207646896abb152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.667454 29757 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4a836f7bfb414eb5bba33a3be51d4f (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:22.680757 29975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a7caaa2edf4bafacd9da8900ebeeb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.681663 29980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4a836f7bfb414eb5bba33a3be51d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.681977 29757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4a836f7bfb414eb5bba33a3be51d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.669992 29755 tablet_service.cc:1511] Processing CreateTablet for tablet b4e73d2d9e7449d4b87ea5ee4324c2c9 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:22.685041 29755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4e73d2d9e7449d4b87ea5ee4324c2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.671197 29754 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7968a480124ac9ad9734c6ffc0dbbb (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:22.672079 29899 tablet_service.cc:1511] Processing CreateTablet for tablet 6ac2292232bc434b88b2786ed56f7adb (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:22.689103 29899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ac2292232bc434b88b2786ed56f7adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.688948 29832 tablet_service.cc:1511] Processing CreateTablet for tablet 252766f9503842f48be95fdd3e3035c2 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:22.672562 29751 tablet_service.cc:1511] Processing CreateTablet for tablet b1ff57a4871b4c2197e7c7f41230d429 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:22.690951 29832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 252766f9503842f48be95fdd3e3035c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.675323 29753 tablet_service.cc:1511] Processing CreateTablet for tablet 99a9fd11d5a348389207646896abb152 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:22.673440 29750 tablet_service.cc:1511] Processing CreateTablet for tablet 95db6009685e4a3d88d098ea07105ad7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:22.665565 29974 tablet_service.cc:1511] Processing CreateTablet for tablet 95db6009685e4a3d88d098ea07105ad7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:22.697072 29974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95db6009685e4a3d88d098ea07105ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.666050 29901 tablet_service.cc:1511] Processing CreateTablet for tablet 45a7caaa2edf4bafacd9da8900ebeeb2 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:22.700099 29901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a7caaa2edf4bafacd9da8900ebeeb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.676017 29752 tablet_service.cc:1511] Processing CreateTablet for tablet 8405db80d77d4b8c9737d33de3ca1f74 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:22.668761 29756 tablet_service.cc:1511] Processing CreateTablet for tablet 252766f9503842f48be95fdd3e3035c2 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:22.674013 29973 tablet_service.cc:1511] Processing CreateTablet for tablet 6ac2292232bc434b88b2786ed56f7adb (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:22.704682 29973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ac2292232bc434b88b2786ed56f7adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.692999 29831 tablet_service.cc:1511] Processing CreateTablet for tablet b4e73d2d9e7449d4b87ea5ee4324c2c9 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:22.708304 29831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4e73d2d9e7449d4b87ea5ee4324c2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.711746 29900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1ff57a4871b4c2197e7c7f41230d429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.715612 29754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7968a480124ac9ad9734c6ffc0dbbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.718144 29750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95db6009685e4a3d88d098ea07105ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.719189 29751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1ff57a4871b4c2197e7c7f41230d429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.729758 29753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99a9fd11d5a348389207646896abb152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.721565 29828 tablet_service.cc:1511] Processing CreateTablet for tablet 99a9fd11d5a348389207646896abb152 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:22.722702 29829 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7968a480124ac9ad9734c6ffc0dbbb (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:22.724606 29827 tablet_service.cc:1511] Processing CreateTablet for tablet 45a7caaa2edf4bafacd9da8900ebeeb2 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:22.726359 29826 tablet_service.cc:1511] Processing CreateTablet for tablet b1ff57a4871b4c2197e7c7f41230d429 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:22.728235 29825 tablet_service.cc:1511] Processing CreateTablet for tablet 95db6009685e4a3d88d098ea07105ad7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:22.732393 29828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99a9fd11d5a348389207646896abb152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.720724 29830 tablet_service.cc:1511] Processing CreateTablet for tablet 0e543a21ae8243058d010255b40860e9 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:22.733752 29827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a7caaa2edf4bafacd9da8900ebeeb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.733870 29756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 252766f9503842f48be95fdd3e3035c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.735994 29826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1ff57a4871b4c2197e7c7f41230d429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.736330 29752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8405db80d77d4b8c9737d33de3ca1f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.744163 29825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95db6009685e4a3d88d098ea07105ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.749158 29829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7968a480124ac9ad9734c6ffc0dbbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.750252 29830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e543a21ae8243058d010255b40860e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.774310 29749 tablet_service.cc:1511] Processing CreateTablet for tablet 6ac2292232bc434b88b2786ed56f7adb (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:22.775197 29824 tablet_service.cc:1511] Processing CreateTablet for tablet 737f2cb09b0a48b1987133f213820765 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:22.775863 29749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ac2292232bc434b88b2786ed56f7adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.776674 29824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737f2cb09b0a48b1987133f213820765. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.776517 29898 tablet_service.cc:1511] Processing CreateTablet for tablet 737f2cb09b0a48b1987133f213820765 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:22.777979 29898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737f2cb09b0a48b1987133f213820765. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.789718 29972 tablet_service.cc:1511] Processing CreateTablet for tablet 737f2cb09b0a48b1987133f213820765 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:22.790405 29748 tablet_service.cc:1511] Processing CreateTablet for tablet e388ba02893f419997372341b6f561d8 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:22.791512 29972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737f2cb09b0a48b1987133f213820765. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.791909 29748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e388ba02893f419997372341b6f561d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.792275 29897 tablet_service.cc:1511] Processing CreateTablet for tablet e388ba02893f419997372341b6f561d8 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:22.793710 29897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e388ba02893f419997372341b6f561d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.795212 29823 tablet_service.cc:1511] Processing CreateTablet for tablet e388ba02893f419997372341b6f561d8 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:22.796628 29823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e388ba02893f419997372341b6f561d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.801646 29971 tablet_service.cc:1511] Processing CreateTablet for tablet c6b780417ded422690ec3d46dda49a07 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:22.803257 29971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b780417ded422690ec3d46dda49a07. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:22.850924 29713 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:22.875481 29976 tablet_service.cc:1511] Processing CreateTablet for tablet 7255b6d0c90146d7bb85a327da87cec0 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:22.886320 29971 tablet_service.cc:1511] Processing CreateTablet for tablet 0ef88786080e40e1a8a8b623e9c30106 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:22.887888 29976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7255b6d0c90146d7bb85a327da87cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.888602 29971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ef88786080e40e1a8a8b623e9c30106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.897794 29832 tablet_service.cc:1511] Processing CreateTablet for tablet c6b780417ded422690ec3d46dda49a07 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:22.899679 29832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b780417ded422690ec3d46dda49a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.904521 30034 tablet_bootstrap.cc:492] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:22.905718 29757 tablet_service.cc:1511] Processing CreateTablet for tablet 277299b9ea1b456eb943ddf7be26fe05 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:22.867190 29980 tablet_service.cc:1511] Processing CreateTablet for tablet 277299b9ea1b456eb943ddf7be26fe05 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:22.907138 29904 tablet_service.cc:1511] Processing CreateTablet for tablet 277299b9ea1b456eb943ddf7be26fe05 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:22.910534 29755 tablet_service.cc:1511] Processing CreateTablet for tablet 21581ff708354cb0ad0ed6ae935d4eef (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:22.914220 29905 tablet_service.cc:1511] Processing CreateTablet for tablet 7255b6d0c90146d7bb85a327da87cec0 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:22.882550 29978 tablet_service.cc:1511] Processing CreateTablet for tablet 6d91cac27fc9443b86fc3821776294de (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:22.923977 29757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 277299b9ea1b456eb943ddf7be26fe05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.930306 29756 tablet_service.cc:1511] Processing CreateTablet for tablet 6d91cac27fc9443b86fc3821776294de (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:22.932145 29830 tablet_service.cc:1511] Processing CreateTablet for tablet 6d91cac27fc9443b86fc3821776294de (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:22.928869 29826 tablet_service.cc:1511] Processing CreateTablet for tablet 7255b6d0c90146d7bb85a327da87cec0 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:22.945541 29906 tablet_service.cc:1511] Processing CreateTablet for tablet 0ef88786080e40e1a8a8b623e9c30106 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:22.946897 29980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 277299b9ea1b456eb943ddf7be26fe05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.950649 29900 tablet_service.cc:1511] Processing CreateTablet for tablet 21581ff708354cb0ad0ed6ae935d4eef (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:22.955696 29828 tablet_service.cc:1511] Processing CreateTablet for tablet 21581ff708354cb0ad0ed6ae935d4eef (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:22.959878 30034 tablet_bootstrap.cc:654] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.959990 30035 tablet_bootstrap.cc:492] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:22.961123 29830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d91cac27fc9443b86fc3821776294de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.961511 29831 tablet_service.cc:1511] Processing CreateTablet for tablet e388e95cfde842949825ab8dbdc03e91 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:22.961792 29824 tablet_service.cc:1511] Processing CreateTablet for tablet 125fc2e410394efba24979999a6785b7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:22.962800 30036 tablet_bootstrap.cc:492] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:22.963163 29831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e388e95cfde842949825ab8dbdc03e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.964210 30037 tablet_bootstrap.cc:492] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:22.969965 30036 tablet_bootstrap.cc:654] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.969350 29903 tablet_service.cc:1511] Processing CreateTablet for tablet e388e95cfde842949825ab8dbdc03e91 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:22.971079 29903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e388e95cfde842949825ab8dbdc03e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.972294 29973 tablet_service.cc:1511] Processing CreateTablet for tablet 125fc2e410394efba24979999a6785b7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:22.975905 29905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7255b6d0c90146d7bb85a327da87cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.976194 29755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21581ff708354cb0ad0ed6ae935d4eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.976778 29900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21581ff708354cb0ad0ed6ae935d4eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.977285 29752 tablet_service.cc:1511] Processing CreateTablet for tablet 0ef88786080e40e1a8a8b623e9c30106 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:22.978652 29752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ef88786080e40e1a8a8b623e9c30106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.979590 29906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ef88786080e40e1a8a8b623e9c30106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.981094 29902 tablet_service.cc:1511] Processing CreateTablet for tablet 125fc2e410394efba24979999a6785b7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:22.982117 29756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d91cac27fc9443b86fc3821776294de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.982436 29902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125fc2e410394efba24979999a6785b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.982052 29751 tablet_service.cc:1511] Processing CreateTablet for tablet c6b780417ded422690ec3d46dda49a07 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:22.983572 29978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d91cac27fc9443b86fc3821776294de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.984330 29753 tablet_service.cc:1511] Processing CreateTablet for tablet e388e95cfde842949825ab8dbdc03e91 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:22.985751 29973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125fc2e410394efba24979999a6785b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.983780 29751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b780417ded422690ec3d46dda49a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.999678 29753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e388e95cfde842949825ab8dbdc03e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.999848 29828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21581ff708354cb0ad0ed6ae935d4eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.003067 29826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7255b6d0c90146d7bb85a327da87cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.003952 29904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 277299b9ea1b456eb943ddf7be26fe05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.006476 29824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125fc2e410394efba24979999a6785b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.014591 30037 tablet_bootstrap.cc:654] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.017452 30034 tablet_bootstrap.cc:492] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:23.023912 30034 ts_tablet_manager.cc:1403] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.120s	user 0.006s	sys 0.010s
I20260709 13:10:23.025960 29972 tablet_service.cc:1511] Processing CreateTablet for tablet caad1999ec5545d9a8eb2ade21997cab (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:23.026840 29823 tablet_service.cc:1511] Processing CreateTablet for tablet caad1999ec5545d9a8eb2ade21997cab (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:23.027067 29979 tablet_service.cc:1511] Processing CreateTablet for tablet 0bbc73ad9e6848228c0d2cae81843b7a (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:23.027392 29972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caad1999ec5545d9a8eb2ade21997cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.027256 30034 raft_consensus.cc:359] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.030654 30034 raft_consensus.cc:385] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.031006 30034 raft_consensus.cc:740] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.031641 29979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbc73ad9e6848228c0d2cae81843b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.029891 29754 tablet_service.cc:1511] Processing CreateTablet for tablet 0bbc73ad9e6848228c0d2cae81843b7a (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:23.032671 29754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbc73ad9e6848228c0d2cae81843b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.028319 29823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caad1999ec5545d9a8eb2ade21997cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.029307 29749 tablet_service.cc:1511] Processing CreateTablet for tablet caad1999ec5545d9a8eb2ade21997cab (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:23.040060 29749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caad1999ec5545d9a8eb2ade21997cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.028394 29897 tablet_service.cc:1511] Processing CreateTablet for tablet 0bbc73ad9e6848228c0d2cae81843b7a (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:23.043598 29897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbc73ad9e6848228c0d2cae81843b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.037891 30034 consensus_queue.cc:260] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.045078 29792 heartbeater.cc:499] Master 127.19.127.254:37437 was elected leader, sending a full tablet report...
I20260709 13:10:23.069577 30036 tablet_bootstrap.cc:492] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:23.070156 30036 ts_tablet_manager.cc:1403] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.108s	user 0.008s	sys 0.003s
I20260709 13:10:23.092964 30035 tablet_bootstrap.cc:654] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.105904 30036 raft_consensus.cc:359] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:23.106745 30036 raft_consensus.cc:385] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.107085 30036 raft_consensus.cc:740] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.112879 30036 consensus_queue.cc:260] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:23.114836 30015 heartbeater.cc:499] Master 127.19.127.254:37437 was elected leader, sending a full tablet report...
I20260709 13:10:23.118012 30037 tablet_bootstrap.cc:492] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:23.118634 30037 ts_tablet_manager.cc:1403] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.155s	user 0.009s	sys 0.021s
I20260709 13:10:23.122004 30037 raft_consensus.cc:359] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:23.122748 30037 raft_consensus.cc:385] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.124562 30034 ts_tablet_manager.cc:1434] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.100s	user 0.012s	sys 0.000s
I20260709 13:10:23.125878 30034 tablet_bootstrap.cc:492] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:23.145382 29867 heartbeater.cc:499] Master 127.19.127.254:37437 was elected leader, sending a full tablet report...
I20260709 13:10:23.123085 30037 raft_consensus.cc:740] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.172655 30036 ts_tablet_manager.cc:1434] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.102s	user 0.003s	sys 0.017s
I20260709 13:10:23.173642 29941 heartbeater.cc:499] Master 127.19.127.254:37437 was elected leader, sending a full tablet report...
I20260709 13:10:23.173899 30036 tablet_bootstrap.cc:492] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:23.184969 30037 consensus_queue.cc:260] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:23.188277 30042 raft_consensus.cc:493] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.188828 30042 raft_consensus.cc:515] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:23.191419 30042 leader_election.cc:290] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:23.210572 30034 tablet_bootstrap.cc:654] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.215489 29823 tablet_service.cc:1511] Processing CreateTablet for tablet 0cfa1a88ac2b4cacb00c00d882d4cf1b (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:23.216959 29823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cfa1a88ac2b4cacb00c00d882d4cf1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.219712 30036 tablet_bootstrap.cc:654] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:23.241307 29942 tablet.cc:2406] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:10:23.245829 30041 raft_consensus.cc:493] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.244882 29906 tablet_service.cc:1511] Processing CreateTablet for tablet 0cfa1a88ac2b4cacb00c00d882d4cf1b (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:23.245441 29971 tablet_service.cc:1511] Processing CreateTablet for tablet 0cfa1a88ac2b4cacb00c00d882d4cf1b (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:23.246495 30041 raft_consensus.cc:515] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.252168 30035 tablet_bootstrap.cc:492] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:23.251097 30041 leader_election.cc:290] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:23.268704 30034 tablet_bootstrap.cc:492] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:23.272132 30034 ts_tablet_manager.cc:1403] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.147s	user 0.013s	sys 0.000s
I20260709 13:10:23.272368 29757 tablet_service.cc:1511] Processing CreateTablet for tablet 90385323444f41d096dd3842e8b2bd6d (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:23.274349 29757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90385323444f41d096dd3842e8b2bd6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.269524 30035 ts_tablet_manager.cc:1403] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.310s	user 0.023s	sys 0.013s
I20260709 13:10:23.277298 30034 raft_consensus.cc:359] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.277823 30036 tablet_bootstrap.cc:492] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:23.278162 30034 raft_consensus.cc:385] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.278541 30036 ts_tablet_manager.cc:1403] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.105s	user 0.012s	sys 0.009s
I20260709 13:10:23.268512 29906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cfa1a88ac2b4cacb00c00d882d4cf1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.271152 30049 raft_consensus.cc:493] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.279985 30049 raft_consensus.cc:515] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:23.278514 30034 raft_consensus.cc:740] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.268740 29971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cfa1a88ac2b4cacb00c00d882d4cf1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.282635 30036 raft_consensus.cc:359] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.286986 30036 raft_consensus.cc:385] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.287619 30036 raft_consensus.cc:740] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.268013 30037 ts_tablet_manager.cc:1434] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.149s	user 0.028s	sys 0.019s
I20260709 13:10:23.294986 30049 leader_election.cc:290] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:23.297254 30037 tablet_bootstrap.cc:492] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:23.304862 29903 tablet_service.cc:1511] Processing CreateTablet for tablet 90385323444f41d096dd3842e8b2bd6d (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:23.306388 29903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90385323444f41d096dd3842e8b2bd6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.309629 30037 tablet_bootstrap.cc:654] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.288633 30036 consensus_queue.cc:260] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.316408 30034 consensus_queue.cc:260] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.321154 30034 ts_tablet_manager.cc:1434] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 13:10:23.322624 30034 tablet_bootstrap.cc:492] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:23.323625 30035 raft_consensus.cc:359] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.324481 30035 raft_consensus.cc:385] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.324776 30035 raft_consensus.cc:740] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.325562 30035 consensus_queue.cc:260] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.297411 29824 tablet_service.cc:1511] Processing CreateTablet for tablet 90385323444f41d096dd3842e8b2bd6d (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:23.343379 29824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90385323444f41d096dd3842e8b2bd6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.352138 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4a836f7bfb414eb5bba33a3be51d4f" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:23.355063 29881 leader_election.cc:343] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.384187 30036 ts_tablet_manager.cc:1434] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.000s
I20260709 13:10:23.385489 30036 tablet_bootstrap.cc:492] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:23.391116 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4a836f7bfb414eb5bba33a3be51d4f" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:23.392772 29830 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9fb82d905c441bb50d70e1ce080213 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
W20260709 13:10:23.393283 29881 leader_election.cc:343] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.394022 29881 leader_election.cc:304] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:23.394387 29830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9fb82d905c441bb50d70e1ce080213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.400202 30044 raft_consensus.cc:2749] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.401147 29979 tablet_service.cc:1511] Processing CreateTablet for tablet 5481f26841ab45efa35ded9872e310ee (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:23.402489 29979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5481f26841ab45efa35ded9872e310ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.409675 30035 ts_tablet_manager.cc:1434] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.134s	user 0.008s	sys 0.024s
I20260709 13:10:23.409502 29749 tablet_service.cc:1511] Processing CreateTablet for tablet 5481f26841ab45efa35ded9872e310ee (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:23.409996 29976 tablet_service.cc:1511] Processing CreateTablet for tablet fac962e63d404b2a89aa0afd301ce161 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:23.410831 30035 tablet_bootstrap.cc:492] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:23.411248 29976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac962e63d404b2a89aa0afd301ce161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.411666 29755 tablet_service.cc:1511] Processing CreateTablet for tablet fac962e63d404b2a89aa0afd301ce161 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:23.412112 29823 tablet_service.cc:1511] Processing CreateTablet for tablet 5481f26841ab45efa35ded9872e310ee (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:23.413441 29823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5481f26841ab45efa35ded9872e310ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.410908 29749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5481f26841ab45efa35ded9872e310ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.428067 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:23.432803 29807 leader_election.cc:343] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.432797 29755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac962e63d404b2a89aa0afd301ce161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.433727 30034 tablet_bootstrap.cc:654] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.434478 29971 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9fb82d905c441bb50d70e1ce080213 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:23.437177 29971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9fb82d905c441bb50d70e1ce080213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.445927 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543a21ae8243058d010255b40860e9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:23.447760 29955 leader_election.cc:343] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.447839 29832 tablet_service.cc:1511] Processing CreateTablet for tablet a55c176c0f054ab4ac3e880739d7c297 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:23.449328 29832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a55c176c0f054ab4ac3e880739d7c297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.454022 29828 tablet_service.cc:1511] Processing CreateTablet for tablet df180e07f62245aa9c117b1558657534 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:23.454492 29972 tablet_service.cc:1511] Processing CreateTablet for tablet df180e07f62245aa9c117b1558657534 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:23.455896 29972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df180e07f62245aa9c117b1558657534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.456270 29828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df180e07f62245aa9c117b1558657534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.460922 29904 tablet_service.cc:1511] Processing CreateTablet for tablet fac962e63d404b2a89aa0afd301ce161 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:23.462342 29904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac962e63d404b2a89aa0afd301ce161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.466141 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:23.470804 29807 leader_election.cc:343] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.471324 29807 leader_election.cc:304] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:23.472554 30049 raft_consensus.cc:2749] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.478559 29754 tablet_service.cc:1511] Processing CreateTablet for tablet df180e07f62245aa9c117b1558657534 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:23.479697 30041 raft_consensus.cc:493] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.480408 29754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df180e07f62245aa9c117b1558657534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.482466 30041 raft_consensus.cc:515] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.485419 30041 leader_election.cc:290] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:23.487758 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a7caaa2edf4bafacd9da8900ebeeb2" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:23.489413 29955 leader_election.cc:343] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.492779 29971 tablet_service.cc:1511] Processing CreateTablet for tablet af3bf2e1f12f46c59f56c808bcfd5e78 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:23.494242 29971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3bf2e1f12f46c59f56c808bcfd5e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.496264 29757 tablet_service.cc:1511] Processing CreateTablet for tablet a55c176c0f054ab4ac3e880739d7c297 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:23.496428 29753 tablet_service.cc:1511] Processing CreateTablet for tablet af3bf2e1f12f46c59f56c808bcfd5e78 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:23.497711 29757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a55c176c0f054ab4ac3e880739d7c297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.499926 29753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3bf2e1f12f46c59f56c808bcfd5e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.499625 29826 tablet_service.cc:1511] Processing CreateTablet for tablet d4e21634252d40b196da2d7ac6d5faa5 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:23.500967 29826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4e21634252d40b196da2d7ac6d5faa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.504109 30036 tablet_bootstrap.cc:654] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.508987 30062 raft_consensus.cc:493] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.509560 30062 raft_consensus.cc:515] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.520328 29897 tablet_service.cc:1511] Processing CreateTablet for tablet af3bf2e1f12f46c59f56c808bcfd5e78 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:23.518129 29906 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9fb82d905c441bb50d70e1ce080213 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:23.523904 29902 tablet_service.cc:1511] Processing CreateTablet for tablet d4e21634252d40b196da2d7ac6d5faa5 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:23.524966 29897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3bf2e1f12f46c59f56c808bcfd5e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.527016 29973 tablet_service.cc:1511] Processing CreateTablet for tablet d4e21634252d40b196da2d7ac6d5faa5 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:23.527381 29903 tablet_service.cc:1511] Processing CreateTablet for tablet a55c176c0f054ab4ac3e880739d7c297 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:23.528477 29973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4e21634252d40b196da2d7ac6d5faa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.528549 29903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a55c176c0f054ab4ac3e880739d7c297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.530949 29902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4e21634252d40b196da2d7ac6d5faa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.534981 29906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9fb82d905c441bb50d70e1ce080213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.535319 29978 tablet_service.cc:1511] Processing CreateTablet for tablet 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:23.537123 29978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ebcaf79fc9947e5a4a8e8ed8eb3dab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.537369 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543a21ae8243058d010255b40860e9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:23.538267 29916 raft_consensus.cc:2468] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:23.540548 29752 tablet_service.cc:1511] Processing CreateTablet for tablet 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:23.542419 29752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ebcaf79fc9947e5a4a8e8ed8eb3dab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.538105 29755 tablet_service.cc:1511] Processing CreateTablet for tablet 1207ccf2285f42dc901a1db9e48e39f3 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:23.549717 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a7caaa2edf4bafacd9da8900ebeeb2" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:23.542230 29828 tablet_service.cc:1511] Processing CreateTablet for tablet 1207ccf2285f42dc901a1db9e48e39f3 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:23.561772 29955 leader_election.cc:304] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:23.562278 29828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1207ccf2285f42dc901a1db9e48e39f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.562844 30041 raft_consensus.cc:2804] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:23.563320 30041 raft_consensus.cc:493] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.563719 30041 raft_consensus.cc:3060] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.564173 29755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1207ccf2285f42dc901a1db9e48e39f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.568915 29905 tablet_service.cc:1511] Processing CreateTablet for tablet 1207ccf2285f42dc901a1db9e48e39f3 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:23.570324 29905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1207ccf2285f42dc901a1db9e48e39f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.572800 30035 tablet_bootstrap.cc:654] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.575147 29980 tablet_service.cc:1511] Processing CreateTablet for tablet ce794c74850e4dc6845e02da675fa199 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:23.577121 29980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce794c74850e4dc6845e02da675fa199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.579982 30041 raft_consensus.cc:515] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.582427 30041 leader_election.cc:290] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:23.583738 29955 leader_election.cc:343] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.583977 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543a21ae8243058d010255b40860e9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:23.584262 29955 leader_election.cc:304] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
W20260709 13:10:23.585528 29955 leader_election.cc:343] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.587136 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543a21ae8243058d010255b40860e9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:23.587826 29916 raft_consensus.cc:3060] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.591670 30036 tablet_bootstrap.cc:492] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:23.592314 30036 ts_tablet_manager.cc:1403] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.207s	user 0.010s	sys 0.004s
I20260709 13:10:23.593362 29900 tablet_service.cc:1511] Processing CreateTablet for tablet ce794c74850e4dc6845e02da675fa199 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:23.595191 29900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce794c74850e4dc6845e02da675fa199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.596184 30062 leader_election.cc:290] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:23.606069 29751 tablet_service.cc:1511] Processing CreateTablet for tablet ce794c74850e4dc6845e02da675fa199 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:23.607636 29751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce794c74850e4dc6845e02da675fa199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.591588 29823 tablet_service.cc:1511] Processing CreateTablet for tablet 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:23.608860 29823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ebcaf79fc9947e5a4a8e8ed8eb3dab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.612226 30034 tablet_bootstrap.cc:492] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:23.612875 30034 ts_tablet_manager.cc:1403] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.291s	user 0.015s	sys 0.005s
I20260709 13:10:23.616209 30034 raft_consensus.cc:359] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:23.615795 30036 raft_consensus.cc:359] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.595191 30037 tablet_bootstrap.cc:492] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:23.619630 30041 raft_consensus.cc:2749] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.640619 30036 raft_consensus.cc:385] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.640985 30034 raft_consensus.cc:385] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.645347 30036 raft_consensus.cc:740] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.642889 29916 raft_consensus.cc:2468] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:23.649786 29955 leader_election.cc:304] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:23.646495 30036 consensus_queue.cc:260] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.645568 30034 raft_consensus.cc:740] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.654104 30037 ts_tablet_manager.cc:1403] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.357s	user 0.017s	sys 0.012s
I20260709 13:10:23.655319 30034 consensus_queue.cc:260] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:23.657938 30037 raft_consensus.cc:359] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.658876 30036 ts_tablet_manager.cc:1434] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.066s	user 0.004s	sys 0.006s
I20260709 13:10:23.664968 30035 tablet_bootstrap.cc:492] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:23.701916 30035 ts_tablet_manager.cc:1403] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.291s	user 0.015s	sys 0.003s
I20260709 13:10:23.703547 30036 tablet_bootstrap.cc:492] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:23.665410 30041 raft_consensus.cc:2804] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:23.669827 30034 ts_tablet_manager.cc:1434] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 13:10:23.675108 29830 tablet_service.cc:1511] Processing CreateTablet for tablet 2a52b7d43c054ac485deaf19144dced6 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:23.705513 30034 tablet_bootstrap.cc:492] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:23.705696 29830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a52b7d43c054ac485deaf19144dced6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.693220 30037 raft_consensus.cc:385] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.708920 30037 raft_consensus.cc:740] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.710005 30037 consensus_queue.cc:260] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.726154 30035 raft_consensus.cc:359] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.749719 30035 raft_consensus.cc:385] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.750080 30035 raft_consensus.cc:740] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.740947 30037 ts_tablet_manager.cc:1434] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.086s	user 0.008s	sys 0.000s
I20260709 13:10:23.750818 30035 consensus_queue.cc:260] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.723802 30041 raft_consensus.cc:697] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:23.762089 30037 tablet_bootstrap.cc:492] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
W20260709 13:10:23.722602 29955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 72925.1us
I20260709 13:10:23.767225 29972 tablet_service.cc:1511] Processing CreateTablet for tablet 2a52b7d43c054ac485deaf19144dced6 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:23.788431 29972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a52b7d43c054ac485deaf19144dced6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.768548 29900 tablet_service.cc:1511] Processing CreateTablet for tablet 2a52b7d43c054ac485deaf19144dced6 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:23.773114 29828 tablet_service.cc:1511] Processing CreateTablet for tablet 6714cbaa854d430fb09d82c5d878cff7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:23.776654 29755 tablet_service.cc:1511] Processing CreateTablet for tablet 6714cbaa854d430fb09d82c5d878cff7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:23.778543 30035 ts_tablet_manager.cc:1434] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.076s	user 0.010s	sys 0.000s
I20260709 13:10:23.779311 30041 consensus_queue.cc:237] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.773072 29905 tablet_service.cc:1511] Processing CreateTablet for tablet 6714cbaa854d430fb09d82c5d878cff7 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:23.795887 29755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6714cbaa854d430fb09d82c5d878cff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.785753 30037 tablet_bootstrap.cc:654] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.796342 29905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6714cbaa854d430fb09d82c5d878cff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.797945 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:23.798755 29842 raft_consensus.cc:2468] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:23.801297 30035 tablet_bootstrap.cc:492] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:23.796310 29828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6714cbaa854d430fb09d82c5d878cff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.809927 29973 tablet_service.cc:1511] Processing CreateTablet for tablet e86eb4af5624466bb2d9395a36632d71 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:23.811474 29732 leader_election.cc:304] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:23.812271 29973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e86eb4af5624466bb2d9395a36632d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.812533 30062 raft_consensus.cc:2804] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:23.813055 30062 raft_consensus.cc:493] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.813452 30062 raft_consensus.cc:3060] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.817878 29900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a52b7d43c054ac485deaf19144dced6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.818360 29753 tablet_service.cc:1511] Processing CreateTablet for tablet e86eb4af5624466bb2d9395a36632d71 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:23.818253 30034 tablet_bootstrap.cc:654] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.820407 29753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e86eb4af5624466bb2d9395a36632d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.825126 29832 tablet_service.cc:1511] Processing CreateTablet for tablet e86eb4af5624466bb2d9395a36632d71 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:23.827093 29832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e86eb4af5624466bb2d9395a36632d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.831259 29903 tablet_service.cc:1511] Processing CreateTablet for tablet 1b6c48a3ec224675bd35d17f96f8fb13 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:23.831918 29979 tablet_service.cc:1511] Processing CreateTablet for tablet 1b6c48a3ec224675bd35d17f96f8fb13 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:23.833164 29903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b6c48a3ec224675bd35d17f96f8fb13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.833606 29979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b6c48a3ec224675bd35d17f96f8fb13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.851640 29824 tablet_service.cc:1511] Processing CreateTablet for tablet 8170c15c8bf942b596d738aa14bb6f80 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:23.853102 29824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8170c15c8bf942b596d738aa14bb6f80. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:23.853857 29713 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:23.856021 30036 tablet_bootstrap.cc:654] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.860946 30035 tablet_bootstrap.cc:654] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.871837 29751 tablet_service.cc:1511] Processing CreateTablet for tablet 1b6c48a3ec224675bd35d17f96f8fb13 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:23.873540 29751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b6c48a3ec224675bd35d17f96f8fb13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.885190 29980 tablet_service.cc:1511] Processing CreateTablet for tablet 8807cef4b9f940359ad32d99dd28b5cd (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:23.887573 29980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8807cef4b9f940359ad32d99dd28b5cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.886096 29897 tablet_service.cc:1511] Processing CreateTablet for tablet dc233ddc2b6d41f395652a5c5010c88b (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:23.889335 29897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc233ddc2b6d41f395652a5c5010c88b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.892349 29752 tablet_service.cc:1511] Processing CreateTablet for tablet dc233ddc2b6d41f395652a5c5010c88b (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:23.893780 29752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc233ddc2b6d41f395652a5c5010c88b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.889295 29832 tablet_service.cc:1511] Processing CreateTablet for tablet dc233ddc2b6d41f395652a5c5010c88b (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:23.902697 29832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc233ddc2b6d41f395652a5c5010c88b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.909752 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:23.910562 29916 raft_consensus.cc:2468] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:23.882928 30062 raft_consensus.cc:515] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:23.913268 30062 leader_election.cc:290] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:23.913182 29978 tablet_service.cc:1511] Processing CreateTablet for tablet 8170c15c8bf942b596d738aa14bb6f80 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:23.915272 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:23.916057 29842 raft_consensus.cc:3060] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.920679 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:23.921407 29916 raft_consensus.cc:3060] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.921666 29757 tablet_service.cc:1511] Processing CreateTablet for tablet 8807cef4b9f940359ad32d99dd28b5cd (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:23.923008 29757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8807cef4b9f940359ad32d99dd28b5cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.886147 29906 tablet_service.cc:1511] Processing CreateTablet for tablet 8170c15c8bf942b596d738aa14bb6f80 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:23.926654 29906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8170c15c8bf942b596d738aa14bb6f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.926512 29902 tablet_service.cc:1511] Processing CreateTablet for tablet 96641d56c48649d993e0eec66d091ee5 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:23.928182 29902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96641d56c48649d993e0eec66d091ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.931702 29842 raft_consensus.cc:2468] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:23.933172 29732 leader_election.cc:304] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:23.935187 29978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8170c15c8bf942b596d738aa14bb6f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.935539 30036 tablet_bootstrap.cc:492] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:23.944104 30058 raft_consensus.cc:2804] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:23.947432 30036 ts_tablet_manager.cc:1403] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.244s	user 0.018s	sys 0.002s
I20260709 13:10:23.943459 30062 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.960533 30037 tablet_bootstrap.cc:492] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:23.961158 30037 ts_tablet_manager.cc:1403] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.199s	user 0.041s	sys 0.054s
I20260709 13:10:23.961470 29979 tablet_service.cc:1511] Processing CreateTablet for tablet 96641d56c48649d993e0eec66d091ee5 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:23.963624 29979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96641d56c48649d993e0eec66d091ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.964546 30037 raft_consensus.cc:359] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.965375 30037 raft_consensus.cc:385] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.965683 30037 raft_consensus.cc:740] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.965724 29824 tablet_service.cc:1511] Processing CreateTablet for tablet 8807cef4b9f940359ad32d99dd28b5cd (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:23.966580 30037 consensus_queue.cc:260] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.967134 29824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8807cef4b9f940359ad32d99dd28b5cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.972919 30037 ts_tablet_manager.cc:1434] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:10:23.990793 30049 raft_consensus.cc:493] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.991406 30049 raft_consensus.cc:515] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:23.993836 30049 leader_election.cc:290] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:24.000030 30037 tablet_bootstrap.cc:492] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:24.000070 29904 tablet_service.cc:1511] Processing CreateTablet for tablet cc1a922f421149f48209b8e771fb3ca9 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:24.001806 29904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc1a922f421149f48209b8e771fb3ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.969841 30036 raft_consensus.cc:359] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:24.006201 30036 raft_consensus.cc:385] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.956049 30062 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:24.005910 29753 tablet_service.cc:1511] Processing CreateTablet for tablet 96641d56c48649d993e0eec66d091ee5 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:24.006789 29971 tablet_service.cc:1511] Processing CreateTablet for tablet cc1a922f421149f48209b8e771fb3ca9 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:24.008075 29971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc1a922f421149f48209b8e771fb3ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.010352 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:24.008076 29753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96641d56c48649d993e0eec66d091ee5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:24.012532 29732 leader_election.cc:343] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.012377 29755 tablet_service.cc:1511] Processing CreateTablet for tablet e9a1697952144bae8ddfeab6f69d13f4 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:24.021557 29755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a1697952144bae8ddfeab6f69d13f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.014788 30036 raft_consensus.cc:740] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.024984 30058 raft_consensus.cc:697] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:24.037053 30062 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:23.974767 29662 catalog_manager.cc:5697] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:24.049710 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:24.051970 30034 tablet_bootstrap.cc:492] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:24.052570 30034 ts_tablet_manager.cc:1403] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.347s	user 0.016s	sys 0.012s
I20260709 13:10:24.058599 29905 tablet_service.cc:1511] Processing CreateTablet for tablet e9a1697952144bae8ddfeab6f69d13f4 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:24.061105 29830 tablet_service.cc:1511] Processing CreateTablet for tablet e9a1697952144bae8ddfeab6f69d13f4 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:24.062314 30041 raft_consensus.cc:493] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.069329 29830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a1697952144bae8ddfeab6f69d13f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.068208 30034 raft_consensus.cc:359] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.069998 30034 raft_consensus.cc:385] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.070474 30034 raft_consensus.cc:740] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.070889 30037 tablet_bootstrap.cc:654] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.071322 30034 consensus_queue.cc:260] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.073688 30034 ts_tablet_manager.cc:1434] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.003s
I20260709 13:10:24.074735 30034 tablet_bootstrap.cc:492] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:24.065732 29973 tablet_service.cc:1511] Processing CreateTablet for tablet 523db1e9cca74ffb938f22ae9105dbe1 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:24.068905 30041 raft_consensus.cc:515] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.064035 29754 tablet_service.cc:1511] Processing CreateTablet for tablet 523db1e9cca74ffb938f22ae9105dbe1 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:24.083432 29754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 523db1e9cca74ffb938f22ae9105dbe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.083902 29905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a1697952144bae8ddfeab6f69d13f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.084236 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543a21ae8243058d010255b40860e9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:24.078740 30041 leader_election.cc:290] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:24.081485 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8405db80d77d4b8c9737d33de3ca1f74" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:24.099443 30058 consensus_queue.cc:237] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
W20260709 13:10:24.102774 29955 leader_election.cc:343] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.068300 29824 tablet_service.cc:1511] Processing CreateTablet for tablet cc1a922f421149f48209b8e771fb3ca9 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:24.116230 29824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc1a922f421149f48209b8e771fb3ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.122061 29828 tablet_service.cc:1511] Processing CreateTablet for tablet 523db1e9cca74ffb938f22ae9105dbe1 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:24.123643 29973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 523db1e9cca74ffb938f22ae9105dbe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.124614 29828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 523db1e9cca74ffb938f22ae9105dbe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.129431 30035 tablet_bootstrap.cc:492] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:24.130263 30035 ts_tablet_manager.cc:1403] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.329s	user 0.020s	sys 0.004s
I20260709 13:10:24.140666 29916 raft_consensus.cc:2468] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:24.045881 30036 consensus_queue.cc:260] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:24.154515 29915 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:24.161176 29732 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:24.162469 30075 raft_consensus.cc:2804] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.162899 30075 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.163247 30075 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.179171 29900 tablet_service.cc:1511] Processing CreateTablet for tablet ced82bb96f8a45e790d075986ff66cce (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:24.180694 29900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced82bb96f8a45e790d075986ff66cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.185461 29751 tablet_service.cc:1511] Processing CreateTablet for tablet ced82bb96f8a45e790d075986ff66cce (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:24.185595 29980 tablet_service.cc:1511] Processing CreateTablet for tablet ced82bb96f8a45e790d075986ff66cce (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:24.188627 29897 tablet_service.cc:1511] Processing CreateTablet for tablet e02b4b0aee0c426ba054d6eac3eda348 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:24.189356 29823 tablet_service.cc:1511] Processing CreateTablet for tablet e02b4b0aee0c426ba054d6eac3eda348 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:24.190021 29897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e02b4b0aee0c426ba054d6eac3eda348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.193694 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543a21ae8243058d010255b40860e9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:24.194615 29915 raft_consensus.cc:2393] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:24.195680 29980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced82bb96f8a45e790d075986ff66cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.195888 29807 leader_election.cc:304] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:24.197312 30077 raft_consensus.cc:3060] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.199548 29823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e02b4b0aee0c426ba054d6eac3eda348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.207393 29751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced82bb96f8a45e790d075986ff66cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.211364 30041 raft_consensus.cc:493] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.211934 30041 raft_consensus.cc:515] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.214028 30041 leader_election.cc:290] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:24.216689 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4a836f7bfb414eb5bba33a3be51d4f" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:24.217465 29915 raft_consensus.cc:2468] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:24.218708 29955 leader_election.cc:304] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:24.219777 30041 raft_consensus.cc:2804] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.220167 30041 raft_consensus.cc:493] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.220508 30041 raft_consensus.cc:3060] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.229400 30075 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:24.233139 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
W20260709 13:10:24.234684 29732 leader_election.cc:343] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.241480 30077 raft_consensus.cc:2749] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.222532 30035 raft_consensus.cc:359] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.250288 30035 raft_consensus.cc:385] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.250568 30035 raft_consensus.cc:740] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.251255 30035 consensus_queue.cc:260] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.253731 30034 tablet_bootstrap.cc:654] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.253931 30035 ts_tablet_manager.cc:1434] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.123s	user 0.006s	sys 0.003s
I20260709 13:10:24.255066 30035 tablet_bootstrap.cc:492] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:24.261515 30036 ts_tablet_manager.cc:1434] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.314s	user 0.034s	sys 0.067s
I20260709 13:10:24.261858 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:24.262482 29915 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.262722 30036 tablet_bootstrap.cc:492] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:24.245054 29979 tablet_service.cc:1511] Processing CreateTablet for tablet e02b4b0aee0c426ba054d6eac3eda348 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:24.266207 29979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e02b4b0aee0c426ba054d6eac3eda348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.267908 30075 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:24.285809 29665 catalog_manager.cc:5697] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.289322 29828 tablet_service.cc:1511] Processing CreateTablet for tablet 9f721871c07c4c57b9c2a1185604e336 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:24.289669 29751 tablet_service.cc:1511] Processing CreateTablet for tablet 9f721871c07c4c57b9c2a1185604e336 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:24.290810 29828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f721871c07c4c57b9c2a1185604e336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.291663 29915 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:24.292222 29751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f721871c07c4c57b9c2a1185604e336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.294256 30069 consensus_queue.cc:1048] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:24.298796 30037 tablet_bootstrap.cc:492] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:24.299430 30037 ts_tablet_manager.cc:1403] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.300s	user 0.016s	sys 0.001s
I20260709 13:10:24.314266 30041 raft_consensus.cc:515] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.315663 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8405db80d77d4b8c9737d33de3ca1f74" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:24.316488 29732 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:24.315730 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4a836f7bfb414eb5bba33a3be51d4f" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:24.317123 30036 tablet_bootstrap.cc:654] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.317476 29766 raft_consensus.cc:2468] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:24.317813 30075 raft_consensus.cc:2804] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.318466 30075 raft_consensus.cc:697] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
W20260709 13:10:24.319417 29955 leader_election.cc:343] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.319929 29955 leader_election.cc:304] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:24.311190 30035 tablet_bootstrap.cc:654] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.321686 30069 raft_consensus.cc:2749] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.319453 30075 consensus_queue.cc:237] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:24.325309 29900 tablet_service.cc:1511] Processing CreateTablet for tablet 9f721871c07c4c57b9c2a1185604e336 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:24.327009 29900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f721871c07c4c57b9c2a1185604e336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.335722 29980 tablet_service.cc:1511] Processing CreateTablet for tablet 8a6337639944433881ce49be8f337030 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:24.336867 30034 tablet_bootstrap.cc:492] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:24.337260 29980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a6337639944433881ce49be8f337030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.336932 29754 tablet_service.cc:1511] Processing CreateTablet for tablet 8a6337639944433881ce49be8f337030 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:24.337543 30034 ts_tablet_manager.cc:1403] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.263s	user 0.000s	sys 0.018s
I20260709 13:10:24.338430 29754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a6337639944433881ce49be8f337030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.340407 30041 leader_election.cc:290] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:24.341756 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4a836f7bfb414eb5bba33a3be51d4f" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:24.343956 29915 raft_consensus.cc:3060] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.348169 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4a836f7bfb414eb5bba33a3be51d4f" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:24.348860 29766 raft_consensus.cc:3060] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.349809 30037 raft_consensus.cc:359] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.350879 30037 raft_consensus.cc:385] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.356189 29897 tablet_service.cc:1511] Processing CreateTablet for tablet 8c4c252a57bc4facbe5d2e0ceb4c80ae (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:24.358248 29897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c4c252a57bc4facbe5d2e0ceb4c80ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.351197 30034 raft_consensus.cc:359] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.363029 30034 raft_consensus.cc:385] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.351269 30037 raft_consensus.cc:740] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.365867 29915 raft_consensus.cc:2468] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:24.359938 29823 tablet_service.cc:1511] Processing CreateTablet for tablet 8a6337639944433881ce49be8f337030 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:24.368959 30034 raft_consensus.cc:740] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.370826 29978 tablet_service.cc:1511] Processing CreateTablet for tablet 8c4c252a57bc4facbe5d2e0ceb4c80ae (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:24.383564 29823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a6337639944433881ce49be8f337030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.383947 29978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c4c252a57bc4facbe5d2e0ceb4c80ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.387184 29955 leader_election.cc:304] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:24.388237 30041 raft_consensus.cc:2804] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.388876 30041 raft_consensus.cc:697] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:24.389914 30041 consensus_queue.cc:237] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.395478 29828 tablet_service.cc:1511] Processing CreateTablet for tablet d182f1fe9db6430d8ff9a0c29b11c3f3 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:24.371924 30037 consensus_queue.cc:260] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.398880 29828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d182f1fe9db6430d8ff9a0c29b11c3f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.399901 30037 ts_tablet_manager.cc:1434] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.100s	user 0.008s	sys 0.001s
I20260709 13:10:24.400992 30037 tablet_bootstrap.cc:492] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:24.376474 29755 tablet_service.cc:1511] Processing CreateTablet for tablet 8c4c252a57bc4facbe5d2e0ceb4c80ae (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:24.404464 29755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c4c252a57bc4facbe5d2e0ceb4c80ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.407490 30037 tablet_bootstrap.cc:654] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.408792 29905 tablet_service.cc:1511] Processing CreateTablet for tablet d182f1fe9db6430d8ff9a0c29b11c3f3 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:24.410194 29905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d182f1fe9db6430d8ff9a0c29b11c3f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.371206 29766 raft_consensus.cc:2468] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:24.428098 29979 tablet_service.cc:1511] Processing CreateTablet for tablet d182f1fe9db6430d8ff9a0c29b11c3f3 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:24.429596 29979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d182f1fe9db6430d8ff9a0c29b11c3f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.438254 29753 tablet_service.cc:1511] Processing CreateTablet for tablet d9f49c45a4a5479d8609fe95f8486b97 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:24.439769 29753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9f49c45a4a5479d8609fe95f8486b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.442027 30069 consensus_queue.cc:1048] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:24.446650 30077 raft_consensus.cc:493] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.451596 30077 raft_consensus.cc:515] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.453763 30077 leader_election.cc:290] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:24.469295 30036 tablet_bootstrap.cc:492] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:24.469846 30036 ts_tablet_manager.cc:1403] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.207s	user 0.019s	sys 0.006s
I20260709 13:10:24.468541 29897 tablet_service.cc:1511] Processing CreateTablet for tablet d9f49c45a4a5479d8609fe95f8486b97 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:24.472937 29900 tablet_service.cc:1511] Processing CreateTablet for tablet 20470bec53744d58b8955d49e2e7ea2a (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:24.473152 29897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9f49c45a4a5479d8609fe95f8486b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.478843 29900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20470bec53744d58b8955d49e2e7ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.483824 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:24.484164 29752 tablet_service.cc:1511] Processing CreateTablet for tablet 20470bec53744d58b8955d49e2e7ea2a (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:24.415047 30034 consensus_queue.cc:260] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.485651 29752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20470bec53744d58b8955d49e2e7ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.494234 29751 tablet_service.cc:1511] Processing CreateTablet for tablet 70e5a681df5b42e98dac6b49a96773fe (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260709 13:10:24.491500 29807 leader_election.cc:343] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.500401 29979 tablet_service.cc:1511] Processing CreateTablet for tablet 70e5a681df5b42e98dac6b49a96773fe (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:24.501247 30037 tablet_bootstrap.cc:492] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:24.500877 29980 tablet_service.cc:1511] Processing CreateTablet for tablet 20470bec53744d58b8955d49e2e7ea2a (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:24.501747 30037 ts_tablet_manager.cc:1403] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.101s	user 0.013s	sys 0.021s
I20260709 13:10:24.501909 29979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e5a681df5b42e98dac6b49a96773fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.504241 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:24.509914 29807 leader_election.cc:343] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.510442 29807 leader_election.cc:304] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:24.510726 29832 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1bb074fa4f46b48be16325eabd6d7d (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:24.511780 30077 raft_consensus.cc:2749] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.512157 29832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1bb074fa4f46b48be16325eabd6d7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.512457 30034 ts_tablet_manager.cc:1434] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.174s	user 0.007s	sys 0.004s
I20260709 13:10:24.513679 30034 tablet_bootstrap.cc:492] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:24.515514 29980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20470bec53744d58b8955d49e2e7ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.516240 30036 raft_consensus.cc:359] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.517112 30036 raft_consensus.cc:385] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.517453 30036 raft_consensus.cc:740] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.519436 30037 raft_consensus.cc:359] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:24.520289 30037 raft_consensus.cc:385] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.520653 30037 raft_consensus.cc:740] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.500937 29823 tablet_service.cc:1511] Processing CreateTablet for tablet d9f49c45a4a5479d8609fe95f8486b97 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:24.522476 29823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9f49c45a4a5479d8609fe95f8486b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.523983 30036 consensus_queue.cc:260] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.525046 29904 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1bb074fa4f46b48be16325eabd6d7d (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:24.526391 29904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1bb074fa4f46b48be16325eabd6d7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.518879 29663 catalog_manager.cc:5697] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.532526 29828 tablet_service.cc:1511] Processing CreateTablet for tablet 70e5a681df5b42e98dac6b49a96773fe (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:24.532886 30037 consensus_queue.cc:260] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:24.533829 29828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e5a681df5b42e98dac6b49a96773fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.535868 29751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e5a681df5b42e98dac6b49a96773fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.544305 30036 ts_tablet_manager.cc:1434] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.003s
I20260709 13:10:24.549911 29978 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1bb074fa4f46b48be16325eabd6d7d (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:24.551582 29978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1bb074fa4f46b48be16325eabd6d7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.555984 30075 consensus_queue.cc:1048] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:24.561332 30036 tablet_bootstrap.cc:492] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:24.562256 30037 ts_tablet_manager.cc:1434] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.003s
I20260709 13:10:24.579121 30034 tablet_bootstrap.cc:654] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.580444 29661 catalog_manager.cc:5697] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.594065 29902 tablet_service.cc:1511] Processing CreateTablet for tablet 9fecd61828184c7e9cba33adb56ccd51 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:24.595491 29902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fecd61828184c7e9cba33adb56ccd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.597143 30037 tablet_bootstrap.cc:492] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:24.598238 29824 tablet_service.cc:1511] Processing CreateTablet for tablet 9fecd61828184c7e9cba33adb56ccd51 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:24.599566 29824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fecd61828184c7e9cba33adb56ccd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.605192 29755 tablet_service.cc:1511] Processing CreateTablet for tablet 9fecd61828184c7e9cba33adb56ccd51 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:24.606675 29755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fecd61828184c7e9cba33adb56ccd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.607753 30035 tablet_bootstrap.cc:492] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:24.608355 30035 ts_tablet_manager.cc:1403] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.354s	user 0.030s	sys 0.000s
I20260709 13:10:24.614380 29973 tablet_service.cc:1511] Processing CreateTablet for tablet 244d599ec58f4e51ac20679e26a3c2b1 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:24.612921 30035 raft_consensus.cc:359] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.619663 30035 raft_consensus.cc:385] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.625382 29973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 244d599ec58f4e51ac20679e26a3c2b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.628140 29754 tablet_service.cc:1511] Processing CreateTablet for tablet 244d599ec58f4e51ac20679e26a3c2b1 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:24.633651 30037 tablet_bootstrap.cc:654] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.643347 29754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 244d599ec58f4e51ac20679e26a3c2b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.636615 30035 raft_consensus.cc:740] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.664496 30041 raft_consensus.cc:493] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.666793 30041 raft_consensus.cc:515] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.676545 30041 leader_election.cc:290] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:24.678574 30088 raft_consensus.cc:493] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.679266 30088 raft_consensus.cc:515] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.681430 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:24.682894 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:24.683382 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8405db80d77d4b8c9737d33de3ca1f74" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:24.684672 29955 leader_election.cc:343] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.685271 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8405db80d77d4b8c9737d33de3ca1f74" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:24.681474 30088 leader_election.cc:290] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:24.685973 29990 raft_consensus.cc:2468] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:24.686440 29979 tablet_service.cc:1511] Processing CreateTablet for tablet 74e8c425db9e4e479c00e6f131fbafe2 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:24.676699 30036 tablet_bootstrap.cc:654] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.688066 29881 leader_election.cc:304] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:24.689160 29979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e8c425db9e4e479c00e6f131fbafe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.688931 29753 tablet_service.cc:1511] Processing CreateTablet for tablet 74e8c425db9e4e479c00e6f131fbafe2 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
W20260709 13:10:24.685417 29955 leader_election.cc:343] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:24.689013 29881 leader_election.cc:343] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.691890 29753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e8c425db9e4e479c00e6f131fbafe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.689778 29955 leader_election.cc:304] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:24.692598 29823 tablet_service.cc:1511] Processing CreateTablet for tablet 244d599ec58f4e51ac20679e26a3c2b1 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:24.693737 29823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 244d599ec58f4e51ac20679e26a3c2b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.695901 30075 consensus_queue.cc:1048] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:24.695901 30041 raft_consensus.cc:2749] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.694898 30088 raft_consensus.cc:2804] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.706638 30088 raft_consensus.cc:493] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.724148 29900 tablet_service.cc:1511] Processing CreateTablet for tablet 74e8c425db9e4e479c00e6f131fbafe2 (DEFAULT_TABLE table=test-workload [id=6e4c54e7339f4c2f98a59d5946456b58]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:24.727028 30088 raft_consensus.cc:3060] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.736862 30034 tablet_bootstrap.cc:492] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:24.737473 30034 ts_tablet_manager.cc:1403] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.224s	user 0.020s	sys 0.011s
I20260709 13:10:24.743778 29900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e8c425db9e4e479c00e6f131fbafe2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:24.750131 29732 consensus_peers.cc:597] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:24.714500 30035 consensus_queue.cc:260] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.757241 30035 ts_tablet_manager.cc:1434] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.149s	user 0.009s	sys 0.001s
I20260709 13:10:24.758428 30035 tablet_bootstrap.cc:492] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:24.773032 30034 raft_consensus.cc:359] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.773792 30034 raft_consensus.cc:385] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.774083 30034 raft_consensus.cc:740] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.774816 30034 consensus_queue.cc:260] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.782078 30037 tablet_bootstrap.cc:492] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:24.785109 30037 ts_tablet_manager.cc:1403] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.188s	user 0.037s	sys 0.005s
I20260709 13:10:24.799433 30088 raft_consensus.cc:515] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.799537 30037 raft_consensus.cc:359] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:24.800576 30037 raft_consensus.cc:385] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.800904 30037 raft_consensus.cc:740] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.803273 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8405db80d77d4b8c9737d33de3ca1f74" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:24.803411 30088 leader_election.cc:290] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:24.803835 30035 tablet_bootstrap.cc:654] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.803949 29990 raft_consensus.cc:3060] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.808183 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8405db80d77d4b8c9737d33de3ca1f74" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:24.808315 30037 consensus_queue.cc:260] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:24.809607 30036 tablet_bootstrap.cc:492] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
W20260709 13:10:24.809911 29881 leader_election.cc:343] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.810168 30036 ts_tablet_manager.cc:1403] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.252s	user 0.008s	sys 0.013s
I20260709 13:10:24.810648 30037 ts_tablet_manager.cc:1434] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.001s
I20260709 13:10:24.813670 30037 tablet_bootstrap.cc:492] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:24.813357 30036 raft_consensus.cc:359] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:24.814159 30036 raft_consensus.cc:385] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.814458 30036 raft_consensus.cc:740] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.816004 30034 ts_tablet_manager.cc:1434] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.078s	user 0.009s	sys 0.000s
I20260709 13:10:24.817094 30034 tablet_bootstrap.cc:492] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:24.818185 30035 tablet_bootstrap.cc:492] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:24.818811 30035 ts_tablet_manager.cc:1403] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.001s
I20260709 13:10:24.821427 30037 tablet_bootstrap.cc:654] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.824005 30034 tablet_bootstrap.cc:654] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.815234 30036 consensus_queue.cc:260] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:24.829653 30036 ts_tablet_manager.cc:1434] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:10:24.831341 30036 tablet_bootstrap.cc:492] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:24.837368 30035 raft_consensus.cc:359] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.838560 30036 tablet_bootstrap.cc:654] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.838946 30035 raft_consensus.cc:385] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.839520 30035 raft_consensus.cc:740] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.847214 29990 raft_consensus.cc:2468] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:24.848779 29881 leader_election.cc:304] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:24.848779 30036 tablet_bootstrap.cc:492] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:24.850543 30036 ts_tablet_manager.cc:1403] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.002s
I20260709 13:10:24.850451 30088 raft_consensus.cc:2804] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.851159 30088 raft_consensus.cc:697] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:24.840801 30035 consensus_queue.cc:260] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.852254 30088 consensus_queue.cc:237] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.853998 30035 ts_tablet_manager.cc:1434] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.002s
I20260709 13:10:24.857697 30037 tablet_bootstrap.cc:492] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:24.858315 30037 ts_tablet_manager.cc:1403] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.024s
I20260709 13:10:24.858865 30034 tablet_bootstrap.cc:492] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:24.859700 30034 ts_tablet_manager.cc:1403] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.009s
I20260709 13:10:24.864805 30035 tablet_bootstrap.cc:492] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
W20260709 13:10:24.868912 29713 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:24.864127 30034 raft_consensus.cc:359] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.869743 30034 raft_consensus.cc:385] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.870193 30034 raft_consensus.cc:740] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.869001 30036 raft_consensus.cc:359] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:24.872172 30036 raft_consensus.cc:385] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.872589 30036 raft_consensus.cc:740] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.872753 30037 raft_consensus.cc:359] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.873930 30037 raft_consensus.cc:385] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.874388 30037 raft_consensus.cc:740] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.873710 30036 consensus_queue.cc:260] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:24.875008 30041 raft_consensus.cc:493] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.875268 30037 consensus_queue.cc:260] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.875715 30041 raft_consensus.cc:515] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.878115 30035 tablet_bootstrap.cc:654] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.878132 30037 ts_tablet_manager.cc:1434] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:10:24.878834 30058 raft_consensus.cc:493] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.879413 30058 raft_consensus.cc:515] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.880620 30037 tablet_bootstrap.cc:492] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:24.881560 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db6009685e4a3d88d098ea07105ad7" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:24.881822 30058 leader_election.cc:290] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:24.882306 29841 raft_consensus.cc:2468] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:24.880658 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db6009685e4a3d88d098ea07105ad7" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:24.878074 30041 leader_election.cc:290] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:24.875301 30034 consensus_queue.cc:260] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:24.888067 30037 tablet_bootstrap.cc:654] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.897157 29663 catalog_manager.cc:5697] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.898335 30034 ts_tablet_manager.cc:1434] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:10:24.899447 30034 tablet_bootstrap.cc:492] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:24.904917 30036 ts_tablet_manager.cc:1434] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 13:10:24.906067 30036 tablet_bootstrap.cc:492] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:24.913473 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:24.914244 29841 raft_consensus.cc:2468] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:24.917407 29732 leader_election.cc:304] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:24.918599 30058 raft_consensus.cc:2804] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.919111 30058 raft_consensus.cc:493] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.919579 30058 raft_consensus.cc:3060] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:24.922477 29955 leader_election.cc:343] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.923388 29955 leader_election.cc:304] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:24.924281 30041 raft_consensus.cc:2804] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.924660 30041 raft_consensus.cc:493] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.924983 30041 raft_consensus.cc:3060] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.927673 30058 raft_consensus.cc:515] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.929804 30058 leader_election.cc:290] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:24.932590 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:24.933404 29841 raft_consensus.cc:3060] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.932271 30041 raft_consensus.cc:515] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.940778 30041 leader_election.cc:290] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:24.941087 29841 raft_consensus.cc:2468] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:24.942449 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db6009685e4a3d88d098ea07105ad7" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:24.943995 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db6009685e4a3d88d098ea07105ad7" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:24.955524 29842 raft_consensus.cc:3060] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.955229 30058 consensus_queue.cc:1048] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:24.960330 30034 tablet_bootstrap.cc:654] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.948841 30036 tablet_bootstrap.cc:654] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.964407 29732 leader_election.cc:304] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:24.965059 29842 raft_consensus.cc:2468] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:24.965567 30058 raft_consensus.cc:2804] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.966282 30058 raft_consensus.cc:697] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:24.967728 29955 leader_election.cc:304] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:24.967157 30058 consensus_queue.cc:237] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.971171 30041 raft_consensus.cc:2804] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.973518 30041 raft_consensus.cc:697] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:24.974545 30107 raft_consensus.cc:493] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.975816 30107 raft_consensus.cc:515] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
W20260709 13:10:24.975231 29955 leader_election.cc:343] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.980348 30037 tablet_bootstrap.cc:492] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:24.980962 30037 ts_tablet_manager.cc:1403] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.101s	user 0.042s	sys 0.031s
I20260709 13:10:24.981482 30107 leader_election.cc:290] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:24.984138 30037 raft_consensus.cc:359] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:24.984915 30037 raft_consensus.cc:385] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.985236 30037 raft_consensus.cc:740] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.985991 30037 consensus_queue.cc:260] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:24.986830 30035 tablet_bootstrap.cc:492] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:24.987417 30035 ts_tablet_manager.cc:1403] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.123s	user 0.027s	sys 0.054s
I20260709 13:10:24.988646 30037 ts_tablet_manager.cc:1434] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:24.989964 30037 tablet_bootstrap.cc:492] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:24.974416 30041 consensus_queue.cc:237] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:24.991532 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f2cb09b0a48b1987133f213820765" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:24.988770 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f2cb09b0a48b1987133f213820765" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.000984 30069 consensus_queue.cc:1048] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:25.001652 29807 leader_election.cc:343] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.004530 30036 tablet_bootstrap.cc:492] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
W20260709 13:10:25.005079 29807 leader_election.cc:343] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.005618 29807 leader_election.cc:304] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:25.006403 30107 raft_consensus.cc:2749] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.001631 29661 catalog_manager.cc:5697] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.005067 30036 ts_tablet_manager.cc:1403] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.099s	user 0.011s	sys 0.007s
I20260709 13:10:25.017809 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.003681 30035 raft_consensus.cc:359] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.019852 30035 raft_consensus.cc:385] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.020327 30035 raft_consensus.cc:740] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.021621 30035 consensus_queue.cc:260] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.026939 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:25.027958 30034 tablet_bootstrap.cc:492] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:25.028496 30034 ts_tablet_manager.cc:1403] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.129s	user 0.019s	sys 0.003s
I20260709 13:10:25.029047 30035 ts_tablet_manager.cc:1434] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:10:25.030133 30035 tablet_bootstrap.cc:492] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:25.032392 30037 tablet_bootstrap.cc:654] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.035387 30034 raft_consensus.cc:359] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.036144 30034 raft_consensus.cc:385] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.036393 30034 raft_consensus.cc:740] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.037025 30034 consensus_queue.cc:260] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.037319 30036 raft_consensus.cc:359] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.037979 30036 raft_consensus.cc:385] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.038159 30036 raft_consensus.cc:740] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.038570 30036 consensus_queue.cc:260] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.040249 30036 ts_tablet_manager.cc:1434] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:10:25.041071 30036 tablet_bootstrap.cc:492] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06: Bootstrap starting.
W20260709 13:10:25.044333 29732 leader_election.cc:343] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:25.045756 29732 leader_election.cc:343] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.049240 30035 tablet_bootstrap.cc:654] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.049640 30036 tablet_bootstrap.cc:654] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.050275 29661 catalog_manager.cc:5697] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.054778 30034 ts_tablet_manager.cc:1434] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:10:25.049190 30041 consensus_queue.cc:1048] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:10:25.060282 30034 tablet_bootstrap.cc:492] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:25.077497 30034 tablet_bootstrap.cc:654] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.091171 30069 raft_consensus.cc:493] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.091791 30069 raft_consensus.cc:515] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.092947 30107 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.093474 30107 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.094152 30069 leader_election.cc:290] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:25.095677 30107 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:25.096679 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.097949 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.100171 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.100073 29807 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:25.101277 30107 raft_consensus.cc:2749] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.103036 30037 tablet_bootstrap.cc:492] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:25.103771 30037 ts_tablet_manager.cc:1403] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.114s	user 0.018s	sys 0.008s
I20260709 13:10:25.104090 30036 tablet_bootstrap.cc:492] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:25.104643 30036 ts_tablet_manager.cc:1403] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.008s
I20260709 13:10:25.107740 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:25.107833 30037 raft_consensus.cc:359] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.108659 29842 raft_consensus.cc:2393] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75162022240a4fe1a113333e77964ca3 in current term 1: Already voted for candidate 2ba28923c4534c78903f759f0a4e1265 in this term.
I20260709 13:10:25.109918 29955 leader_election.cc:304] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:25.110775 30069 raft_consensus.cc:3060] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.110371 30036 raft_consensus.cc:359] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.114705 30036 raft_consensus.cc:385] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.115118 30036 raft_consensus.cc:740] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.111711 30035 tablet_bootstrap.cc:492] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:25.116365 30035 ts_tablet_manager.cc:1403] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.086s	user 0.020s	sys 0.016s
I20260709 13:10:25.113106 30037 raft_consensus.cc:385] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.117588 30037 raft_consensus.cc:740] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.117610 30069 raft_consensus.cc:2749] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.119623 30037 consensus_queue.cc:260] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.120942 30036 consensus_queue.cc:260] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.122335 30075 raft_consensus.cc:493] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.123018 30075 raft_consensus.cc:515] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.124850 30037 ts_tablet_manager.cc:1434] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:10:25.125849 30037 tablet_bootstrap.cc:492] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:25.125922 30075 leader_election.cc:290] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.123085 30036 ts_tablet_manager.cc:1434] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:10:25.132010 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:25.133153 30036 tablet_bootstrap.cc:492] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:25.133105 29842 raft_consensus.cc:2468] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:25.135639 29732 leader_election.cc:304] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:25.137149 30058 raft_consensus.cc:2804] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.137678 30058 raft_consensus.cc:493] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.138154 30058 raft_consensus.cc:3060] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.142680 30107 raft_consensus.cc:493] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.143227 30107 raft_consensus.cc:515] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.144428 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.145406 30107 leader_election.cc:290] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:25.146090 29732 leader_election.cc:343] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.149313 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.149914 30037 tablet_bootstrap.cc:654] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:25.150908 29807 leader_election.cc:343] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.150938 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.151787 29915 raft_consensus.cc:2468] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:25.152930 30036 tablet_bootstrap.cc:654] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.153121 29807 leader_election.cc:304] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:25.154053 30107 raft_consensus.cc:2804] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.154445 30107 raft_consensus.cc:493] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.154588 30034 tablet_bootstrap.cc:492] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:25.154798 30107 raft_consensus.cc:3060] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.155114 30034 ts_tablet_manager.cc:1403] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.099s	user 0.026s	sys 0.027s
I20260709 13:10:25.158089 30034 raft_consensus.cc:359] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.158738 30034 raft_consensus.cc:385] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.158990 30034 raft_consensus.cc:740] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.159844 30034 consensus_queue.cc:260] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.160251 30035 raft_consensus.cc:359] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.160976 30035 raft_consensus.cc:385] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.161288 30035 raft_consensus.cc:740] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.162442 30034 ts_tablet_manager.cc:1434] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:25.159502 30058 raft_consensus.cc:515] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.163627 30034 tablet_bootstrap.cc:492] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:25.170444 30034 tablet_bootstrap.cc:654] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.170898 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:25.171728 29842 raft_consensus.cc:3060] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.175800 30107 raft_consensus.cc:515] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.178916 30088 raft_consensus.cc:493] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.179642 30088 raft_consensus.cc:515] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.181115 29842 raft_consensus.cc:2468] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:25.182199 30034 tablet_bootstrap.cc:492] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:25.182423 29732 leader_election.cc:304] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:25.182708 30034 ts_tablet_manager.cc:1403] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.011s
I20260709 13:10:25.183357 30075 raft_consensus.cc:2804] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.183854 30075 raft_consensus.cc:697] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:25.184701 30075 consensus_queue.cc:237] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.186043 30069 raft_consensus.cc:493] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.186543 30069 raft_consensus.cc:515] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.188829 30069 leader_election.cc:290] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:25.185655 30034 raft_consensus.cc:359] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.190725 30034 raft_consensus.cc:385] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.191004 30034 raft_consensus.cc:740] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.162067 30035 consensus_queue.cc:260] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.191823 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac2292232bc434b88b2786ed56f7adb" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.191903 30034 consensus_queue.cc:260] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.193308 30035 ts_tablet_manager.cc:1434] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.001s
W20260709 13:10:25.194069 29955 leader_election.cc:343] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.194417 30035 tablet_bootstrap.cc:492] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:25.194612 30034 ts_tablet_manager.cc:1434] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:10:25.191123 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac2292232bc434b88b2786ed56f7adb" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.195647 29915 raft_consensus.cc:2468] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:25.195685 30034 tablet_bootstrap.cc:492] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:25.196844 29955 leader_election.cc:304] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:25.197008 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:25.198555 30069 raft_consensus.cc:2804] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.198948 30069 raft_consensus.cc:493] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.198755 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:25.199412 30069 raft_consensus.cc:3060] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.199676 29990 raft_consensus.cc:3060] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.203751 30034 tablet_bootstrap.cc:654] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.205140 30088 leader_election.cc:290] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.207587 30069 raft_consensus.cc:515] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.211541 30058 leader_election.cc:290] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.212227 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac2292232bc434b88b2786ed56f7adb" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:25.212992 29915 raft_consensus.cc:3060] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.218261 30069 leader_election.cc:290] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:25.220839 29915 raft_consensus.cc:2468] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:25.222326 29955 leader_election.cc:304] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:25.223227 30069 raft_consensus.cc:2804] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.224903 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac2292232bc434b88b2786ed56f7adb" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:25.228185 29661 catalog_manager.cc:5697] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.236517 30069 raft_consensus.cc:697] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
W20260709 13:10:25.239466 29955 leader_election.cc:343] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.240550 30069 consensus_queue.cc:237] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.247527 30075 raft_consensus.cc:493] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.249629 30075 raft_consensus.cc:515] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.259035 30035 tablet_bootstrap.cc:654] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.260308 30036 tablet_bootstrap.cc:492] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:25.260452 30058 raft_consensus.cc:493] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.261379 30036 ts_tablet_manager.cc:1403] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.128s	user 0.026s	sys 0.035s
I20260709 13:10:25.261404 30058 raft_consensus.cc:515] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.263017 30075 leader_election.cc:290] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.263850 30058 leader_election.cc:290] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:25.264209 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8405db80d77d4b8c9737d33de3ca1f74" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.266479 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8405db80d77d4b8c9737d33de3ca1f74" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.267318 29990 raft_consensus.cc:2468] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:25.267448 30036 raft_consensus.cc:359] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.268364 30036 raft_consensus.cc:385] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.268678 30036 raft_consensus.cc:740] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.261047 30107 leader_election.cc:290] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:25.269575 30036 consensus_queue.cc:260] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.270368 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:25.275506 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.276288 29915 raft_consensus.cc:2468] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:25.267391 29989 raft_consensus.cc:2393] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:25.277014 30036 ts_tablet_manager.cc:1434] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
W20260709 13:10:25.281796 29732 leader_election.cc:343] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:25.282661 29807 leader_election.cc:343] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.283041 29732 leader_election.cc:304] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:25.283748 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:25.284399 29915 raft_consensus.cc:3060] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.284538 29732 leader_election.cc:304] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:25.285490 30075 raft_consensus.cc:2804] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.285897 30075 raft_consensus.cc:493] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.286511 30075 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.287503 30058 raft_consensus.cc:3060] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.291574 30037 tablet_bootstrap.cc:492] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:25.291893 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a7caaa2edf4bafacd9da8900ebeeb2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.292670 29990 raft_consensus.cc:2468] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:25.292171 30037 ts_tablet_manager.cc:1403] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.167s	user 0.005s	sys 0.051s
I20260709 13:10:25.294850 29881 leader_election.cc:304] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:25.299496 30088 raft_consensus.cc:2804] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.299625 30036 tablet_bootstrap.cc:492] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:25.299986 30088 raft_consensus.cc:493] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.300416 30088 raft_consensus.cc:3060] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.301851 30037 raft_consensus.cc:359] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.302614 30037 raft_consensus.cc:385] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.302906 30037 raft_consensus.cc:740] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.302207 29663 catalog_manager.cc:5697] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.304988 30075 raft_consensus.cc:515] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.307233 30036 tablet_bootstrap.cc:654] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.307034 30037 consensus_queue.cc:260] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.317728 30037 ts_tablet_manager.cc:1434] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 13:10:25.310873 29915 raft_consensus.cc:2468] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:25.311306 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a7caaa2edf4bafacd9da8900ebeeb2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:25.318425 29842 raft_consensus.cc:2468] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:25.319267 30037 tablet_bootstrap.cc:492] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:25.321462 29807 leader_election.cc:304] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:25.322355 30107 raft_consensus.cc:2804] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.325112 30037 tablet_bootstrap.cc:654] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.326310 29842 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.326591 30034 tablet_bootstrap.cc:492] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:25.327135 30034 ts_tablet_manager.cc:1403] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.132s	user 0.027s	sys 0.019s
I20260709 13:10:25.330289 30034 raft_consensus.cc:359] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.330976 30034 raft_consensus.cc:385] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.331578 30058 raft_consensus.cc:2749] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.325114 30088 raft_consensus.cc:515] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.336334 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a7caaa2edf4bafacd9da8900ebeeb2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:25.337141 29990 raft_consensus.cc:3060] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.337500 30088 leader_election.cc:290] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.337700 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:25.338302 29841 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.339066 30075 leader_election.cc:290] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:25.339635 30034 raft_consensus.cc:740] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.340319 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:25.341128 29915 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.341567 30034 consensus_queue.cc:260] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.343750 30034 ts_tablet_manager.cc:1434] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:10:25.344761 30034 tablet_bootstrap.cc:492] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:25.346544 29841 raft_consensus.cc:2468] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:25.351155 30034 tablet_bootstrap.cc:654] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.356709 30069 raft_consensus.cc:493] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.357414 30035 tablet_bootstrap.cc:492] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:25.357352 30069 raft_consensus.cc:515] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.357770 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a7caaa2edf4bafacd9da8900ebeeb2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:25.358141 30035 ts_tablet_manager.cc:1403] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.164s	user 0.007s	sys 0.015s
I20260709 13:10:25.358345 29840 raft_consensus.cc:3060] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.359696 30069 leader_election.cc:290] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:25.361334 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "277299b9ea1b456eb943ddf7be26fe05" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.364137 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "277299b9ea1b456eb943ddf7be26fe05" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
W20260709 13:10:25.366127 29955 leader_election.cc:343] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:25.367098 29955 leader_election.cc:343] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.367690 29955 leader_election.cc:304] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:25.369035 30069 raft_consensus.cc:2749] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.372668 30088 consensus_queue.cc:1048] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.376715 29732 leader_election.cc:304] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:25.377385 30036 tablet_bootstrap.cc:492] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:25.378034 30036 ts_tablet_manager.cc:1403] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.079s	user 0.015s	sys 0.004s
I20260709 13:10:25.381286 30036 raft_consensus.cc:359] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.384521 29840 raft_consensus.cc:2468] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:25.387025 30075 raft_consensus.cc:2804] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.396649 30075 raft_consensus.cc:697] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:25.397153 29881 leader_election.cc:304] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:25.397603 30075 consensus_queue.cc:237] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.401715 30035 raft_consensus.cc:359] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.402475 30035 raft_consensus.cc:385] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.402745 30035 raft_consensus.cc:740] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.382109 30036 raft_consensus.cc:385] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.388356 30107 raft_consensus.cc:697] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:25.388789 29990 raft_consensus.cc:2468] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:25.406039 30036 raft_consensus.cc:740] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.395725 29915 raft_consensus.cc:2468] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:25.406596 30107 consensus_queue.cc:237] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.406806 30036 consensus_queue.cc:260] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.409834 30088 raft_consensus.cc:2804] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.433126 30088 raft_consensus.cc:697] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:25.434041 30088 consensus_queue.cc:237] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.414675 30034 tablet_bootstrap.cc:492] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
W20260709 13:10:25.427619 29955 consensus_peers.cc:597] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:25.429631 29661 catalog_manager.cc:5697] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.408553 30035 consensus_queue.cc:260] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.436568 30147 consensus_queue.cc:1048] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.437831 30035 ts_tablet_manager.cc:1434] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.004s
I20260709 13:10:25.439631 30036 ts_tablet_manager.cc:1434] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.008s
I20260709 13:10:25.441071 30036 tablet_bootstrap.cc:492] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:25.430754 30037 tablet_bootstrap.cc:492] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:25.447575 30037 ts_tablet_manager.cc:1403] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.129s	user 0.033s	sys 0.062s
I20260709 13:10:25.449438 30036 tablet_bootstrap.cc:654] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.449759 30035 tablet_bootstrap.cc:492] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:25.455091 30036 tablet_bootstrap.cc:492] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:25.435322 30034 ts_tablet_manager.cc:1403] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.091s	user 0.021s	sys 0.013s
I20260709 13:10:25.431609 29664 catalog_manager.cc:5697] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.456661 30069 consensus_queue.cc:1048] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.457475 30126 raft_consensus.cc:493] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.457964 30126 raft_consensus.cc:515] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.458158 30034 raft_consensus.cc:359] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.459100 30034 raft_consensus.cc:385] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.459452 30034 raft_consensus.cc:740] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.460093 30126 leader_election.cc:290] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:25.460202 30034 consensus_queue.cc:260] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.460947 30035 tablet_bootstrap.cc:654] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.462559 30034 ts_tablet_manager.cc:1434] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:25.462683 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f2cb09b0a48b1987133f213820765" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.463471 29915 raft_consensus.cc:2468] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:25.463932 30034 tablet_bootstrap.cc:492] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:25.465469 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f2cb09b0a48b1987133f213820765" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:25.466362 29840 raft_consensus.cc:2468] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:25.454730 30037 raft_consensus.cc:359] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.467378 30037 raft_consensus.cc:385] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.468068 30037 raft_consensus.cc:740] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.468874 30037 consensus_queue.cc:260] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.473845 30037 ts_tablet_manager.cc:1434] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:10:25.467725 30036 ts_tablet_manager.cc:1403] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.001s
I20260709 13:10:25.469091 30075 consensus_queue.cc:1048] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.470389 29955 leader_election.cc:304] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:25.476740 30126 raft_consensus.cc:2804] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.477342 30126 raft_consensus.cc:493] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.477840 30126 raft_consensus.cc:3060] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.477989 30037 tablet_bootstrap.cc:492] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:25.478592 30088 raft_consensus.cc:493] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.479090 30088 raft_consensus.cc:515] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.481262 30088 leader_election.cc:290] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:25.476992 30036 raft_consensus.cc:359] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.482482 30036 raft_consensus.cc:385] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.482740 30036 raft_consensus.cc:740] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.483405 30036 consensus_queue.cc:260] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.485821 30036 ts_tablet_manager.cc:1434] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:10:25.486593 30075 consensus_queue.cc:1048] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.490415 30037 tablet_bootstrap.cc:654] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.490345 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.486971 30036 tablet_bootstrap.cc:492] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:25.491537 30126 raft_consensus.cc:515] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.493906 30126 leader_election.cc:290] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:25.494071 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:25.496464 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f2cb09b0a48b1987133f213820765" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:25.497072 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f2cb09b0a48b1987133f213820765" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:25.497421 29841 raft_consensus.cc:3060] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.497756 29915 raft_consensus.cc:3060] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.472168 30034 tablet_bootstrap.cc:654] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:25.516748 29881 leader_election.cc:343] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:25.519253 29881 leader_election.cc:343] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.520044 29915 raft_consensus.cc:2468] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:25.530561 30036 tablet_bootstrap.cc:654] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.532457 29955 leader_election.cc:304] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:25.534963 30037 tablet_bootstrap.cc:492] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:25.535562 30037 ts_tablet_manager.cc:1403] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.032s
I20260709 13:10:25.528640 29881 leader_election.cc:304] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:25.537437 30088 raft_consensus.cc:2749] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.539170 30037 raft_consensus.cc:359] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.540133 30037 raft_consensus.cc:385] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.540474 30037 raft_consensus.cc:740] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.541222 30037 consensus_queue.cc:260] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.543725 30037 ts_tablet_manager.cc:1434] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:25.525535 29841 raft_consensus.cc:2468] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:25.544847 29664 catalog_manager.cc:5697] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.552377 30126 raft_consensus.cc:2804] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.553917 30126 raft_consensus.cc:697] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:25.554795 30126 consensus_queue.cc:237] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.557186 30037 tablet_bootstrap.cc:492] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:25.563735 30037 tablet_bootstrap.cc:654] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.572430 30036 tablet_bootstrap.cc:492] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:25.573091 30036 ts_tablet_manager.cc:1403] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.086s	user 0.010s	sys 0.003s
I20260709 13:10:25.575899 30036 raft_consensus.cc:359] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.576625 30036 raft_consensus.cc:385] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.576898 30036 raft_consensus.cc:740] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.577692 30036 consensus_queue.cc:260] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.594054 30034 tablet_bootstrap.cc:492] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:25.594121 30035 tablet_bootstrap.cc:492] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:25.595054 30035 ts_tablet_manager.cc:1403] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.146s	user 0.020s	sys 0.036s
I20260709 13:10:25.595178 30034 ts_tablet_manager.cc:1403] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.131s	user 0.010s	sys 0.004s
I20260709 13:10:25.596803 30075 consensus_queue.cc:1048] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.597958 30035 raft_consensus.cc:359] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.598673 30035 raft_consensus.cc:385] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.598961 30035 raft_consensus.cc:740] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.599828 30035 consensus_queue.cc:260] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.591557 29662 catalog_manager.cc:5697] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.602016 30035 ts_tablet_manager.cc:1434] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:25.603012 30035 tablet_bootstrap.cc:492] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:25.609370 30036 ts_tablet_manager.cc:1434] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:10:25.609901 30088 raft_consensus.cc:493] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.610531 30036 tablet_bootstrap.cc:492] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:25.610415 30088 raft_consensus.cc:515] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.599014 30034 raft_consensus.cc:359] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.612597 30088 leader_election.cc:290] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.612764 30034 raft_consensus.cc:385] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.612860 30035 tablet_bootstrap.cc:654] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.613093 30034 raft_consensus.cc:740] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.614593 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125fc2e410394efba24979999a6785b7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:25.615020 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125fc2e410394efba24979999a6785b7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:25.616055 29881 leader_election.cc:343] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.617606 30037 tablet_bootstrap.cc:492] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:25.618150 30037 ts_tablet_manager.cc:1403] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.071s	user 0.033s	sys 0.015s
I20260709 13:10:25.618842 30034 consensus_queue.cc:260] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.619941 29990 raft_consensus.cc:2468] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:25.621004 30037 raft_consensus.cc:359] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.622090 30037 raft_consensus.cc:385] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.622427 30037 raft_consensus.cc:740] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.622745 29881 leader_election.cc:304] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:25.623174 30037 consensus_queue.cc:260] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.624223 30088 raft_consensus.cc:2804] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.624614 30088 raft_consensus.cc:493] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.624975 30088 raft_consensus.cc:3060] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.625164 30034 ts_tablet_manager.cc:1434] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:10:25.625699 30037 ts_tablet_manager.cc:1434] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:25.626291 30034 tablet_bootstrap.cc:492] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:25.626704 30037 tablet_bootstrap.cc:492] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:25.634073 30036 tablet_bootstrap.cc:654] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.619421 30035 tablet_bootstrap.cc:492] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:25.634634 30037 tablet_bootstrap.cc:654] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.633630 30088 raft_consensus.cc:515] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.637408 30034 tablet_bootstrap.cc:654] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.637382 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125fc2e410394efba24979999a6785b7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
W20260709 13:10:25.639179 29881 leader_election.cc:343] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.641464 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125fc2e410394efba24979999a6785b7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:25.642081 29990 raft_consensus.cc:3060] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.644034 30058 consensus_queue.cc:1048] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.649118 30088 leader_election.cc:290] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.650887 29990 raft_consensus.cc:2468] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:25.651747 30036 tablet_bootstrap.cc:492] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:25.652678 30036 ts_tablet_manager.cc:1403] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.042s	user 0.005s	sys 0.010s
I20260709 13:10:25.653226 29881 leader_election.cc:304] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:25.654721 30088 raft_consensus.cc:2804] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.634829 30035 ts_tablet_manager.cc:1403] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.002s
I20260709 13:10:25.657213 30036 raft_consensus.cc:359] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.658322 30036 raft_consensus.cc:385] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.658707 30036 raft_consensus.cc:740] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.648877 30034 tablet_bootstrap.cc:492] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:25.659816 30034 ts_tablet_manager.cc:1403] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.000s
I20260709 13:10:25.661388 30035 raft_consensus.cc:359] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.662220 30035 raft_consensus.cc:385] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.662709 30088 raft_consensus.cc:697] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:25.664227 30088 consensus_queue.cc:237] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.666134 30034 raft_consensus.cc:359] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.666777 30034 raft_consensus.cc:385] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.667026 30034 raft_consensus.cc:740] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.664978 30037 tablet_bootstrap.cc:492] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:25.663084 30035 raft_consensus.cc:740] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.668166 30035 consensus_queue.cc:260] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.670817 30037 ts_tablet_manager.cc:1403] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.044s	user 0.023s	sys 0.012s
I20260709 13:10:25.670513 30036 consensus_queue.cc:260] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.672817 30036 ts_tablet_manager.cc:1434] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:10:25.673839 30036 tablet_bootstrap.cc:492] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:25.674364 30037 raft_consensus.cc:359] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.675174 30037 raft_consensus.cc:385] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.675578 30037 raft_consensus.cc:740] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.676493 30037 consensus_queue.cc:260] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.678985 30037 ts_tablet_manager.cc:1434] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:25.670547 30035 ts_tablet_manager.cc:1434] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:10:25.680166 30037 tablet_bootstrap.cc:492] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:25.680269 30035 tablet_bootstrap.cc:492] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:25.667788 30034 consensus_queue.cc:260] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.684048 29664 catalog_manager.cc:5697] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.689049 30034 ts_tablet_manager.cc:1434] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.004s
I20260709 13:10:25.689405 30035 tablet_bootstrap.cc:654] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.690265 30034 tablet_bootstrap.cc:492] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:25.686879 30036 tablet_bootstrap.cc:654] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.697893 30037 tablet_bootstrap.cc:654] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.697990 30034 tablet_bootstrap.cc:654] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.699777 30036 tablet_bootstrap.cc:492] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:25.700327 30036 ts_tablet_manager.cc:1403] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.000s
I20260709 13:10:25.702903 30035 tablet_bootstrap.cc:492] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:25.703626 30035 ts_tablet_manager.cc:1403] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.006s
I20260709 13:10:25.703318 30036 raft_consensus.cc:359] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.704107 30036 raft_consensus.cc:385] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.704396 30036 raft_consensus.cc:740] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.705137 30036 consensus_queue.cc:260] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.707224 30035 raft_consensus.cc:359] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.708282 30035 raft_consensus.cc:385] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.708693 30035 raft_consensus.cc:740] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.708103 30036 ts_tablet_manager.cc:1434] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:25.710778 30036 tablet_bootstrap.cc:492] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:25.711634 30037 tablet_bootstrap.cc:492] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:25.712201 30037 ts_tablet_manager.cc:1403] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.004s
I20260709 13:10:25.713096 30035 consensus_queue.cc:260] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.715060 30037 raft_consensus.cc:359] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.715502 30035 ts_tablet_manager.cc:1434] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:10:25.715898 30037 raft_consensus.cc:385] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.716295 30037 raft_consensus.cc:740] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.716629 30035 tablet_bootstrap.cc:492] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:25.717180 30037 consensus_queue.cc:260] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.719651 30037 ts_tablet_manager.cc:1434] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:25.720464 30036 tablet_bootstrap.cc:654] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.721275 30037 tablet_bootstrap.cc:492] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:25.728073 30037 tablet_bootstrap.cc:654] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.729658 30035 tablet_bootstrap.cc:654] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.733183 30034 tablet_bootstrap.cc:492] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:25.733906 30034 ts_tablet_manager.cc:1403] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.008s
I20260709 13:10:25.737711 30034 raft_consensus.cc:359] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.738620 30034 raft_consensus.cc:385] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.738973 30034 raft_consensus.cc:740] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.740039 30034 consensus_queue.cc:260] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.743221 30034 ts_tablet_manager.cc:1434] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:10:25.744549 30034 tablet_bootstrap.cc:492] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:25.755864 30075 raft_consensus.cc:493] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.756697 30075 raft_consensus.cc:515] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.761886 30034 tablet_bootstrap.cc:654] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.762053 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:25.763526 30037 tablet_bootstrap.cc:492] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:25.763722 30075 leader_election.cc:290] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:25.764214 30037 ts_tablet_manager.cc:1403] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.019s
I20260709 13:10:25.766078 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.766903 29915 raft_consensus.cc:2393] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:25.768118 29767 raft_consensus.cc:3060] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.768280 30037 raft_consensus.cc:359] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.769467 30037 raft_consensus.cc:385] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.769878 30037 raft_consensus.cc:740] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.771118 30107 raft_consensus.cc:493] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.771818 30107 raft_consensus.cc:515] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.774621 29732 leader_election.cc:304] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:25.775827 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388e95cfde842949825ab8dbdc03e91" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.777704 30088 raft_consensus.cc:493] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:25.778093 29807 leader_election.cc:343] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.778244 30088 raft_consensus.cc:515] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.776150 30075 raft_consensus.cc:3060] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.780753 30088 leader_election.cc:290] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.782204 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388e95cfde842949825ab8dbdc03e91" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.783116 29915 raft_consensus.cc:2468] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:25.783411 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.784999 29807 leader_election.cc:304] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:25.786962 30143 raft_consensus.cc:493] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.779978 30107 leader_election.cc:290] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:25.787492 30143 raft_consensus.cc:515] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.788038 30107 raft_consensus.cc:2804] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.788695 30107 raft_consensus.cc:493] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.789222 30107 raft_consensus.cc:3060] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.789790 30143 leader_election.cc:290] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
W20260709 13:10:25.785027 29881 leader_election.cc:343] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.785595 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:25.790889 29840 raft_consensus.cc:2468] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:25.795490 30037 consensus_queue.cc:260] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.800911 30035 tablet_bootstrap.cc:492] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:25.801489 30035 ts_tablet_manager.cc:1403] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.085s	user 0.012s	sys 0.004s
I20260709 13:10:25.801787 30037 ts_tablet_manager.cc:1434] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:10:25.802148 30036 tablet_bootstrap.cc:492] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:25.802696 30036 ts_tablet_manager.cc:1403] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.092s	user 0.014s	sys 0.032s
I20260709 13:10:25.803063 30126 consensus_queue.cc:1048] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.804618 30035 raft_consensus.cc:359] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.805326 30035 raft_consensus.cc:385] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.801638 30075 raft_consensus.cc:2749] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.805621 30035 raft_consensus.cc:740] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.806555 30036 raft_consensus.cc:359] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.806963 30035 consensus_queue.cc:260] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.807631 30036 raft_consensus.cc:385] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.808025 30036 raft_consensus.cc:740] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.809126 30035 ts_tablet_manager.cc:1434] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:25.808732 30036 consensus_queue.cc:260] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.810082 30035 tablet_bootstrap.cc:492] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:25.815218 30037 tablet_bootstrap.cc:492] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:25.821447 30107 raft_consensus.cc:515] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:25.823518 30126 consensus_queue.cc:1048] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.824652 30036 ts_tablet_manager.cc:1434] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 13:10:25.826053 30036 tablet_bootstrap.cc:492] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:25.835832 30126 raft_consensus.cc:493] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.836408 30126 raft_consensus.cc:515] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:25.836464 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.837484 29916 raft_consensus.cc:2468] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:25.838563 30126 leader_election.cc:290] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:25.840242 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.840359 29881 leader_election.cc:304] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 75162022240a4fe1a113333e77964ca3
I20260709 13:10:25.840507 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ef88786080e40e1a8a8b623e9c30106" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:25.841576 30088 raft_consensus.cc:2804] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.841660 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ef88786080e40e1a8a8b623e9c30106" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.842206 30088 raft_consensus.cc:493] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.842720 30088 raft_consensus.cc:3060] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:25.845319 29955 leader_election.cc:343] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.846580 30035 tablet_bootstrap.cc:654] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.848315 30075 consensus_queue.cc:1048] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.850628 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388e95cfde842949825ab8dbdc03e91" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:25.823609 30107 leader_election.cc:290] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:25.851080 30037 tablet_bootstrap.cc:654] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.855646 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388e95cfde842949825ab8dbdc03e91" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
W20260709 13:10:25.855648 29955 leader_election.cc:343] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.862656 29916 raft_consensus.cc:3060] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.862890 29955 leader_election.cc:304] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:25.863250 30143 raft_consensus.cc:493] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.863776 30126 raft_consensus.cc:2749] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.864049 30143 raft_consensus.cc:515] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.866886 30143 leader_election.cc:290] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:25.856635 30107 raft_consensus.cc:493] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.868832 30107 raft_consensus.cc:515] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.871600 29916 raft_consensus.cc:2468] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:25.871776 30107 leader_election.cc:290] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
W20260709 13:10:25.856133 29807 leader_election.cc:343] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:25.873924 29807 leader_election.cc:343] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.862241 30036 tablet_bootstrap.cc:654] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.883214 30037 tablet_bootstrap.cc:492] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:25.883853 30037 ts_tablet_manager.cc:1403] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.002s
I20260709 13:10:25.886045 30088 raft_consensus.cc:515] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.892148 30088 leader_election.cc:290] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.892715 29807 leader_election.cc:304] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 75162022240a4fe1a113333e77964ca3
I20260709 13:10:25.893960 29807 leader_election.cc:304] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:25.896802 30180 raft_consensus.cc:2804] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.897225 30180 raft_consensus.cc:493] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.897567 30180 raft_consensus.cc:3060] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.899684 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.900594 29766 raft_consensus.cc:2468] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:25.901623 30075 raft_consensus.cc:493] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.902279 30075 raft_consensus.cc:515] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.905001 30075 leader_election.cc:290] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:25.905231 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:25.905958 29767 raft_consensus.cc:2468] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:25.907640 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.908318 29990 raft_consensus.cc:2468] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:25.909332 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.909610 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:25.909986 29713 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:25.910524 29840 raft_consensus.cc:2468] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:25.912513 30075 raft_consensus.cc:493] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.912760 30037 raft_consensus.cc:359] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.913091 30107 raft_consensus.cc:2804] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.915938 30037 raft_consensus.cc:385] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.916364 30037 raft_consensus.cc:740] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.910010 29989 raft_consensus.cc:2468] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:25.918854 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:25.919339 30034 tablet_bootstrap.cc:492] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:25.920015 30034 ts_tablet_manager.cc:1403] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.176s	user 0.034s	sys 0.025s
I20260709 13:10:25.920233 30088 consensus_queue.cc:1048] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.922832 30107 raft_consensus.cc:697] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:25.923712 30037 consensus_queue.cc:260] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.923794 30107 consensus_queue.cc:237] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
W20260709 13:10:25.926458 29881 leader_election.cc:343] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.930696 30075 raft_consensus.cc:515] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:25.934228 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:25.934154 29807 leader_election.cc:304] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:25.935849 30143 raft_consensus.cc:2804] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.936308 30143 raft_consensus.cc:493] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.936780 30143 raft_consensus.cc:3060] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.936560 29732 leader_election.cc:304] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:25.939230 29807 leader_election.cc:304] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:25.940783 30121 raft_consensus.cc:2804] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.941248 30121 raft_consensus.cc:493] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.941572 30121 raft_consensus.cc:3060] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.941740 29767 raft_consensus.cc:3060] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.944342 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:25.945192 29990 raft_consensus.cc:2468] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:25.968920 30034 raft_consensus.cc:359] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.969705 30034 raft_consensus.cc:385] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.970009 30034 raft_consensus.cc:740] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.970741 30034 consensus_queue.cc:260] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:25.988137 30107 raft_consensus.cc:2804] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.988582 30107 raft_consensus.cc:493] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.988936 30107 raft_consensus.cc:3060] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.991855 30037 ts_tablet_manager.cc:1434] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.108s	user 0.009s	sys 0.003s
I20260709 13:10:25.993212 30037 tablet_bootstrap.cc:492] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:25.968962 30075 leader_election.cc:290] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:25.996919 30034 ts_tablet_manager.cc:1434] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.002s
I20260709 13:10:25.970310 30180 raft_consensus.cc:515] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.003849 30180 leader_election.cc:290] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:25.997905 30034 tablet_bootstrap.cc:492] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:26.000608 30121 raft_consensus.cc:515] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.009370 29840 raft_consensus.cc:2393] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 74191176370841578d396cc01f230f06 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:26.009357 30129 raft_consensus.cc:493] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.023053 30129 raft_consensus.cc:515] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.025442 30129 leader_election.cc:290] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:26.026209 30128 raft_consensus.cc:493] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.026957 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:26.027827 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:26.028487 29840 raft_consensus.cc:2468] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
W20260709 13:10:26.028895 29955 leader_election.cc:343] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.018623 30186 raft_consensus.cc:493] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.029230 30180 raft_consensus.cc:493] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.029699 29955 leader_election.cc:304] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:26.029585 30186 raft_consensus.cc:515] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.029790 30180 raft_consensus.cc:515] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.031473 30126 raft_consensus.cc:2804] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.031883 30126 raft_consensus.cc:493] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.032016 30186 leader_election.cc:290] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.032291 30126 raft_consensus.cc:3060] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.032384 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db6009685e4a3d88d098ea07105ad7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:26.035708 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbc73ad9e6848228c0d2cae81843b7a" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:26.036881 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbc73ad9e6848228c0d2cae81843b7a" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:26.038983 29881 leader_election.cc:343] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.039791 30034 tablet_bootstrap.cc:654] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.012931 30035 tablet_bootstrap.cc:492] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:26.040551 30035 ts_tablet_manager.cc:1403] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.231s	user 0.007s	sys 0.009s
I20260709 13:10:26.015996 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db6009685e4a3d88d098ea07105ad7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:26.026779 30128 raft_consensus.cc:515] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
W20260709 13:10:26.039748 29881 leader_election.cc:343] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.052348 30035 raft_consensus.cc:359] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.053413 30035 raft_consensus.cc:385] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.053781 30035 raft_consensus.cc:740] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.054733 30035 consensus_queue.cc:260] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.032027 30180 leader_election.cc:290] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:26.047827 29881 leader_election.cc:304] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:26.070443 30129 consensus_queue.cc:1048] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:26.075858 30069 consensus_queue.cc:1048] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:26.076153 30186 raft_consensus.cc:2749] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.074162 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:26.074901 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:26.071843 30036 tablet_bootstrap.cc:492] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:26.083913 30036 ts_tablet_manager.cc:1403] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.258s	user 0.019s	sys 0.021s
I20260709 13:10:26.080165 30035 ts_tablet_manager.cc:1434] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.002s
I20260709 13:10:26.066847 30128 leader_election.cc:290] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:26.046577 29807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:35599, user_credentials={real_user=slave}} blocked reactor thread for 107466us
I20260709 13:10:26.085295 30035 tablet_bootstrap.cc:492] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:26.047415 30143 raft_consensus.cc:515] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.094319 30147 consensus_queue.cc:1048] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:26.098281 29915 raft_consensus.cc:2393] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75162022240a4fe1a113333e77964ca3 in current term 1: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:26.099992 30037 tablet_bootstrap.cc:654] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.102232 30036 raft_consensus.cc:359] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.102972 30036 raft_consensus.cc:385] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.103339 29840 raft_consensus.cc:2393] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75162022240a4fe1a113333e77964ca3 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:26.103227 30036 raft_consensus.cc:740] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.104354 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:26.104787 30036 consensus_queue.cc:260] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.105473 29915 raft_consensus.cc:2393] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 1: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:26.105741 30147 consensus_queue.cc:1048] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:26.107007 30036 ts_tablet_manager.cc:1434] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.000s
I20260709 13:10:26.108269 30121 leader_election.cc:290] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:26.108577 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:26.109531 30036 tablet_bootstrap.cc:492] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:26.109449 29990 raft_consensus.cc:3060] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.124266 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:26.127575 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:26.128371 29914 raft_consensus.cc:2468] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:26.128683 29955 leader_election.cc:304] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:26.132864 30126 raft_consensus.cc:515] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.122788 29881 leader_election.cc:304] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:26.158428 30186 raft_consensus.cc:2749] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.157435 30143 leader_election.cc:290] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.167891 30126 leader_election.cc:290] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:26.170670 29732 leader_election.cc:304] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:26.173002 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:26.174903 30035 tablet_bootstrap.cc:654] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.174786 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:26.175727 30036 tablet_bootstrap.cc:654] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.121547 30107 raft_consensus.cc:515] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.192670 30186 raft_consensus.cc:493] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.197172 30186 raft_consensus.cc:515] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.200803 30186 leader_election.cc:290] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.202421 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ef88786080e40e1a8a8b623e9c30106" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:26.203218 29989 raft_consensus.cc:2468] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:26.203919 30107 leader_election.cc:290] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.204608 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ef88786080e40e1a8a8b623e9c30106" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:26.204739 29881 leader_election.cc:304] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:26.205364 29766 raft_consensus.cc:2468] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:26.205680 30186 raft_consensus.cc:2804] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.206089 30186 raft_consensus.cc:493] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.206436 30186 raft_consensus.cc:3060] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.193117 29807 leader_election.cc:304] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:26.212368 29841 raft_consensus.cc:3060] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.213119 30121 raft_consensus.cc:2749] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.228227 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:26.228946 29988 raft_consensus.cc:3060] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.222500 30088 raft_consensus.cc:493] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.231400 30088 raft_consensus.cc:515] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.234220 30088 leader_election.cc:290] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.240586 30037 tablet_bootstrap.cc:492] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:26.241101 30037 ts_tablet_manager.cc:1403] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.248s	user 0.035s	sys 0.034s
I20260709 13:10:26.222903 30107 raft_consensus.cc:2804] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.243646 30034 tablet_bootstrap.cc:492] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:26.244863 30034 ts_tablet_manager.cc:1403] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.247s	user 0.003s	sys 0.016s
I20260709 13:10:26.245332 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "277299b9ea1b456eb943ddf7be26fe05" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:26.235297 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "277299b9ea1b456eb943ddf7be26fe05" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:26.220571 29732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39603, user_credentials={real_user=slave}} blocked reactor thread for 49987.8us
I20260709 13:10:26.246606 29987 raft_consensus.cc:2468] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:26.248052 30192 consensus_queue.cc:1048] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:26.224274 30143 consensus_queue.cc:1048] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
W20260709 13:10:26.248091 29881 leader_election.cc:343] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.250458 29881 leader_election.cc:304] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
W20260709 13:10:26.252913 29955 leader_election.cc:343] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.244593 30107 raft_consensus.cc:493] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.253898 30107 raft_consensus.cc:3060] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.255332 30147 raft_consensus.cc:2804] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.255774 30147 raft_consensus.cc:493] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.256138 30147 raft_consensus.cc:3060] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.258210 30034 raft_consensus.cc:359] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.258982 30034 raft_consensus.cc:385] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.259271 30034 raft_consensus.cc:740] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.260053 30034 consensus_queue.cc:260] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.265892 29990 raft_consensus.cc:2468] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:26.267730 30128 raft_consensus.cc:2749] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.268262 29841 raft_consensus.cc:2468] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:26.268544 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:26.269181 29767 raft_consensus.cc:3060] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.269567 30034 ts_tablet_manager.cc:1434] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:10:26.270750 30034 tablet_bootstrap.cc:492] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:26.272948 30186 raft_consensus.cc:515] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.275694 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:26.276834 30186 leader_election.cc:290] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.273625 29841 raft_consensus.cc:3060] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:26.278311 30199 log.cc:927] Time spent T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:10:26.279836 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ef88786080e40e1a8a8b623e9c30106" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:26.280577 29765 raft_consensus.cc:3060] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.274181 29955 leader_election.cc:304] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:26.283968 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ef88786080e40e1a8a8b623e9c30106" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:26.284089 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:26.276913 29766 raft_consensus.cc:2393] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 1: Already voted for candidate 2ba28923c4534c78903f759f0a4e1265 in this term.
I20260709 13:10:26.284930 29990 raft_consensus.cc:3060] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.286039 30128 raft_consensus.cc:2804] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.286815 30128 raft_consensus.cc:697] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:26.287781 30128 consensus_queue.cc:237] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.291325 30107 raft_consensus.cc:515] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.287889 30037 raft_consensus.cc:359] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.293644 30037 raft_consensus.cc:385] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.293753 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:26.293977 30037 raft_consensus.cc:740] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.294087 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:26.295079 29839 raft_consensus.cc:2393] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:26.294689 30037 consensus_queue.cc:260] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
W20260709 13:10:26.273449 29807 leader_election.cc:343] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.300570 30143 consensus_queue.cc:1048] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:26.284708 29989 raft_consensus.cc:3060] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.306010 30034 tablet_bootstrap.cc:654] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.314636 30037 ts_tablet_manager.cc:1434] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.000s
I20260709 13:10:26.293398 30107 leader_election.cc:290] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:26.316169 30037 tablet_bootstrap.cc:492] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:26.319175 29807 leader_election.cc:304] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06
I20260709 13:10:26.326365 29990 raft_consensus.cc:2468] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:26.336035 29807 leader_election.cc:304] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:26.337296 30188 raft_consensus.cc:2804] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.320919 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:26.339075 30188 raft_consensus.cc:697] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:26.340090 30188 consensus_queue.cc:237] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.320393 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:26.342016 29914 raft_consensus.cc:3060] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.346496 29767 raft_consensus.cc:2468] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:26.343017 29988 raft_consensus.cc:2468] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:26.329396 30107 raft_consensus.cc:2804] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.351608 29665 catalog_manager.cc:5697] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.359018 29807 leader_election.cc:304] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:26.359853 30037 tablet_bootstrap.cc:654] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.347389 29765 raft_consensus.cc:2468] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:26.361589 30143 raft_consensus.cc:493] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.364144 30035 tablet_bootstrap.cc:492] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:26.365175 30107 raft_consensus.cc:697] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:26.366911 30035 ts_tablet_manager.cc:1403] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.282s	user 0.018s	sys 0.016s
I20260709 13:10:26.367905 30180 raft_consensus.cc:2804] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.328038 29732 leader_election.cc:304] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:26.367808 30143 raft_consensus.cc:515] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.368503 30180 raft_consensus.cc:697] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:26.368847 30121 raft_consensus.cc:2749] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.369398 30180 consensus_queue.cc:237] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.370812 30107 consensus_queue.cc:237] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.366628 29914 raft_consensus.cc:2468] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:26.375080 30035 raft_consensus.cc:359] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.376194 30035 raft_consensus.cc:385] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.376556 30035 raft_consensus.cc:740] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.364702 29989 raft_consensus.cc:2468] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:26.320307 30147 raft_consensus.cc:515] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.386794 29881 leader_election.cc:304] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:26.395506 30088 raft_consensus.cc:2804] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.396046 30088 raft_consensus.cc:697] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:26.396886 30088 consensus_queue.cc:237] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.398252 30036 tablet_bootstrap.cc:492] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:26.398802 30036 ts_tablet_manager.cc:1403] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.289s	user 0.023s	sys 0.029s
I20260709 13:10:26.412444 30034 tablet_bootstrap.cc:492] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:26.413283 30034 ts_tablet_manager.cc:1403] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.143s	user 0.008s	sys 0.010s
I20260709 13:10:26.418524 29664 catalog_manager.cc:5697] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.422595 30147 leader_election.cc:290] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.424346 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "277299b9ea1b456eb943ddf7be26fe05" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:26.408844 30143 leader_election.cc:290] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:26.426802 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "277299b9ea1b456eb943ddf7be26fe05" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:26.427733 29988 raft_consensus.cc:3060] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.428150 30036 raft_consensus.cc:359] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.429106 30036 raft_consensus.cc:385] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.429488 30036 raft_consensus.cc:740] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.430384 30036 consensus_queue.cc:260] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.447664 30036 ts_tablet_manager.cc:1434] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 13:10:26.448527 30037 tablet_bootstrap.cc:492] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:26.449067 30037 ts_tablet_manager.cc:1403] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.133s	user 0.014s	sys 0.000s
I20260709 13:10:26.451807 30037 raft_consensus.cc:359] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.452466 29988 raft_consensus.cc:2468] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:26.452541 30037 raft_consensus.cc:385] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.453354 30037 raft_consensus.cc:740] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
W20260709 13:10:26.439733 29807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:46707, user_credentials={real_user=slave}} blocked reactor thread for 80807.8us
I20260709 13:10:26.454010 30036 tablet_bootstrap.cc:492] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:26.454619 30147 raft_consensus.cc:493] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.458194 29881 leader_election.cc:304] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:26.458411 30147 raft_consensus.cc:515] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.461617 30186 raft_consensus.cc:2804] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.463702 30186 raft_consensus.cc:697] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:26.464623 30186 consensus_queue.cc:237] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.470273 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:26.470993 30147 leader_election.cc:290] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
W20260709 13:10:26.462226 29807 leader_election.cc:343] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.474879 30037 consensus_queue.cc:260] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.479166 30034 raft_consensus.cc:359] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.487366 30034 raft_consensus.cc:385] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.487764 30034 raft_consensus.cc:740] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.488752 30034 consensus_queue.cc:260] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.480775 30035 consensus_queue.cc:260] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.493533 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:26.494968 30035 ts_tablet_manager.cc:1434] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.127s	user 0.011s	sys 0.000s
I20260709 13:10:26.496266 30037 ts_tablet_manager.cc:1434] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.000s
I20260709 13:10:26.496641 30035 tablet_bootstrap.cc:492] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:26.497615 30037 tablet_bootstrap.cc:492] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:26.497814 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125fc2e410394efba24979999a6785b7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:26.501847 30034 ts_tablet_manager.cc:1434] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.088s	user 0.006s	sys 0.005s
I20260709 13:10:26.503747 30034 tablet_bootstrap.cc:492] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
W20260709 13:10:26.481341 29881 leader_election.cc:343] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.493448 29807 leader_election.cc:304] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
W20260709 13:10:26.513525 29881 leader_election.cc:343] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.514072 30216 raft_consensus.cc:493] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.514600 30216 raft_consensus.cc:515] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.507665 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125fc2e410394efba24979999a6785b7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:26.515466 30147 consensus_queue.cc:1048] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:26.521063 30211 raft_consensus.cc:2804] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.531502 30216 leader_election.cc:290] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.535458 30211 raft_consensus.cc:697] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
W20260709 13:10:26.526832 29881 leader_election.cc:343] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.539529 30035 tablet_bootstrap.cc:654] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.541376 30036 tablet_bootstrap.cc:654] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.542857 30034 tablet_bootstrap.cc:654] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.539988 29881 leader_election.cc:304] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:26.542713 29661 catalog_manager.cc:5697] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.544289 30147 raft_consensus.cc:2749] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.544953 29661 catalog_manager.cc:5697] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.545848 30211 consensus_queue.cc:237] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.554491 29807 leader_election.cc:304] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:26.554903 30037 tablet_bootstrap.cc:654] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.560768 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:26.562471 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:26.566888 30216 raft_consensus.cc:2749] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:26.569545 30034 tablet_bootstrap.cc:492] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:26.570120 30034 ts_tablet_manager.cc:1403] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.067s	user 0.007s	sys 0.006s
W20260709 13:10:26.576845 29807 consensus_peers.cc:597] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.577927 29807 leader_election.cc:343] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:26.581425 29807 leader_election.cc:343] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.582049 29807 leader_election.cc:304] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:26.583108 30216 raft_consensus.cc:2749] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.588644 30036 tablet_bootstrap.cc:492] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:26.589226 30036 ts_tablet_manager.cc:1403] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.135s	user 0.014s	sys 0.002s
I20260709 13:10:26.594018 30036 raft_consensus.cc:359] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.594734 30036 raft_consensus.cc:385] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.595005 30036 raft_consensus.cc:740] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.595839 30036 consensus_queue.cc:260] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.597905 30036 ts_tablet_manager.cc:1434] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:26.598053 30069 raft_consensus.cc:493] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.598564 30069 raft_consensus.cc:515] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.598855 30036 tablet_bootstrap.cc:492] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:26.600785 30216 consensus_queue.cc:1048] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:26.601712 30069 leader_election.cc:290] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.602952 30075 raft_consensus.cc:493] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.603238 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbc73ad9e6848228c0d2cae81843b7a" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:26.603376 30034 raft_consensus.cc:359] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.603484 30075 raft_consensus.cc:515] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.604275 30034 raft_consensus.cc:385] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.604588 30034 raft_consensus.cc:740] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
W20260709 13:10:26.604995 29955 leader_election.cc:343] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.605242 30034 consensus_queue.cc:260] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.605705 30075 leader_election.cc:290] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:26.607462 30034 ts_tablet_manager.cc:1434] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:10:26.607993 30035 tablet_bootstrap.cc:492] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:26.608536 30035 ts_tablet_manager.cc:1403] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.112s	user 0.022s	sys 0.000s
I20260709 13:10:26.607620 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:26.609421 30034 tablet_bootstrap.cc:492] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:26.611665 29840 raft_consensus.cc:2393] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:26.615150 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbc73ad9e6848228c0d2cae81843b7a" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:26.616034 29914 raft_consensus.cc:2468] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:26.616205 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:26.623198 30036 tablet_bootstrap.cc:654] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.625771 29955 leader_election.cc:304] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:26.626586 29732 leader_election.cc:304] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:26.628543 30075 raft_consensus.cc:3060] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.629164 30069 raft_consensus.cc:2804] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.629571 30069 raft_consensus.cc:493] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.629901 30069 raft_consensus.cc:3060] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.621735 30035 raft_consensus.cc:359] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.634183 30035 raft_consensus.cc:385] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.634565 30035 raft_consensus.cc:740] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.638518 30037 tablet_bootstrap.cc:492] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:26.639079 30037 ts_tablet_manager.cc:1403] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.142s	user 0.025s	sys 0.000s
I20260709 13:10:26.615646 30034 tablet_bootstrap.cc:654] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.641959 30069 raft_consensus.cc:515] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.642449 30037 raft_consensus.cc:359] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.643222 30037 raft_consensus.cc:385] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.643580 30037 raft_consensus.cc:740] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.643805 30035 consensus_queue.cc:260] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.644356 30037 consensus_queue.cc:260] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.647598 30037 ts_tablet_manager.cc:1434] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:26.647753 30035 ts_tablet_manager.cc:1434] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:10:26.648595 30037 tablet_bootstrap.cc:492] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:26.638960 30075 raft_consensus.cc:2749] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.649607 30035 tablet_bootstrap.cc:492] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:26.655212 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbc73ad9e6848228c0d2cae81843b7a" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:26.656424 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbc73ad9e6848228c0d2cae81843b7a" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:26.665736 29914 raft_consensus.cc:3060] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.667915 30037 tablet_bootstrap.cc:654] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.668262 29664 catalog_manager.cc:5697] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:26.673209 30036 tablet_bootstrap.cc:492] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:26.672775 29664 catalog_manager.cc:5697] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.673732 30036 ts_tablet_manager.cc:1403] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.075s	user 0.022s	sys 0.001s
I20260709 13:10:26.674615 29664 catalog_manager.cc:5697] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:26.675660 30035 tablet_bootstrap.cc:654] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:26.676378 29955 leader_election.cc:343] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.676561 30036 raft_consensus.cc:359] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.677248 30036 raft_consensus.cc:385] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.677577 30036 raft_consensus.cc:740] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.678308 30036 consensus_queue.cc:260] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.679164 30034 tablet_bootstrap.cc:492] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:26.679700 30034 ts_tablet_manager.cc:1403] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.071s	user 0.007s	sys 0.004s
I20260709 13:10:26.680394 30036 ts_tablet_manager.cc:1434] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:26.680073 29664 catalog_manager.cc:5697] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.681494 30036 tablet_bootstrap.cc:492] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:26.682382 30034 raft_consensus.cc:359] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.683106 30034 raft_consensus.cc:385] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.683400 30034 raft_consensus.cc:740] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.688294 30034 consensus_queue.cc:260] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.699977 30034 ts_tablet_manager.cc:1434] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:10:26.700942 30034 tablet_bootstrap.cc:492] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:26.654068 30069 leader_election.cc:290] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.703369 30037 tablet_bootstrap.cc:492] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:26.703925 30037 ts_tablet_manager.cc:1403] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.001s
I20260709 13:10:26.687810 30036 tablet_bootstrap.cc:654] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.687724 29914 raft_consensus.cc:2468] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:26.708982 30180 raft_consensus.cc:493] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.713353 30180 raft_consensus.cc:515] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.719478 29955 leader_election.cc:304] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:26.720940 30128 raft_consensus.cc:2804] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.721444 30128 raft_consensus.cc:697] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:26.722234 30128 consensus_queue.cc:237] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.733703 30037 raft_consensus.cc:359] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.734795 30037 raft_consensus.cc:385] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.736503 30180 leader_election.cc:290] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:26.739408 30034 tablet_bootstrap.cc:654] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.742106 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:26.742728 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:26.735133 30037 raft_consensus.cc:740] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.742789 29988 raft_consensus.cc:2468] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:26.743695 29914 raft_consensus.cc:2468] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:26.744148 30037 consensus_queue.cc:260] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.744717 29807 leader_election.cc:304] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:26.745807 30180 raft_consensus.cc:2804] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.746201 30180 raft_consensus.cc:493] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.746583 30180 raft_consensus.cc:3060] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.748451 30037 ts_tablet_manager.cc:1434] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.001s
I20260709 13:10:26.749882 30035 tablet_bootstrap.cc:492] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:26.752665 30035 ts_tablet_manager.cc:1403] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.103s	user 0.021s	sys 0.003s
I20260709 13:10:26.756297 30180 raft_consensus.cc:515] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.756948 30035 raft_consensus.cc:359] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.758003 30035 raft_consensus.cc:385] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.758335 30035 raft_consensus.cc:740] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.759534 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:26.759315 30035 consensus_queue.cc:260] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.761164 30180 leader_election.cc:290] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:26.760138 29914 raft_consensus.cc:3060] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.765043 29665 catalog_manager.cc:5697] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.766050 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:26.766700 29988 raft_consensus.cc:3060] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.774923 29988 raft_consensus.cc:2468] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:26.776649 29807 leader_election.cc:304] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:26.776963 30037 tablet_bootstrap.cc:492] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:26.777608 30180 raft_consensus.cc:2804] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.778306 30180 raft_consensus.cc:697] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:26.779379 30035 ts_tablet_manager.cc:1434] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.001s
I20260709 13:10:26.779171 30180 consensus_queue.cc:237] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:26.750882 30034 tablet_bootstrap.cc:492] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:26.783082 30034 ts_tablet_manager.cc:1403] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.000s
I20260709 13:10:26.784758 30216 consensus_queue.cc:1048] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:26.786976 30034 raft_consensus.cc:359] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.788075 30034 raft_consensus.cc:385] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.788443 30034 raft_consensus.cc:740] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.780853 30035 tablet_bootstrap.cc:492] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:26.788586 30147 consensus_queue.cc:1048] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:26.794243 29914 raft_consensus.cc:2468] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:26.789345 30034 consensus_queue.cc:260] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.805547 30216 consensus_queue.cc:1048] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:26.818562 30037 tablet_bootstrap.cc:654] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.813315 30216 raft_consensus.cc:493] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.819324 30035 tablet_bootstrap.cc:654] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.822109 30216 raft_consensus.cc:515] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:26.832275 30216 leader_election.cc:290] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:26.837539 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:26.838585 30069 consensus_queue.cc:1048] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:26.834300 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:26.847163 30193 consensus_queue.cc:1048] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:10:26.854169 30088 consensus_queue.cc:1048] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:26.863445 30034 ts_tablet_manager.cc:1434] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.080s	user 0.009s	sys 0.000s
I20260709 13:10:26.864564 30034 tablet_bootstrap.cc:492] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:26.878062 29664 catalog_manager.cc:5697] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:26.864580 29807 leader_election.cc:343] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:26.889063 29807 leader_election.cc:343] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.889732 29807 leader_election.cc:304] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:26.891541 30211 raft_consensus.cc:2749] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.902415 30036 tablet_bootstrap.cc:492] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:26.903218 30036 ts_tablet_manager.cc:1403] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.222s	user 0.045s	sys 0.063s
I20260709 13:10:26.915207 30216 consensus_queue.cc:1048] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:26.922228 30212 consensus_queue.cc:1048] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:26.925616 30036 raft_consensus.cc:359] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.926678 30036 raft_consensus.cc:385] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.927038 30036 raft_consensus.cc:740] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.934701 30037 tablet_bootstrap.cc:492] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:26.935379 30037 ts_tablet_manager.cc:1403] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.182s	user 0.020s	sys 0.004s
I20260709 13:10:26.921710 30216 consensus_queue.cc:1048] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:10:26.961192 30036 consensus_queue.cc:260] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.964769 30036 ts_tablet_manager.cc:1434] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.061s	user 0.003s	sys 0.004s
I20260709 13:10:26.966598 30036 tablet_bootstrap.cc:492] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06: Bootstrap starting.
W20260709 13:10:26.967808 30243 log.cc:927] Time spent T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:10:26.970770 30035 tablet_bootstrap.cc:492] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:26.971390 30035 ts_tablet_manager.cc:1403] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.191s	user 0.017s	sys 0.009s
I20260709 13:10:26.973131 30036 tablet_bootstrap.cc:654] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.976274 30037 raft_consensus.cc:359] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.977228 30037 raft_consensus.cc:385] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.977547 30037 raft_consensus.cc:740] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.974278 30035 raft_consensus.cc:359] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.978341 30037 consensus_queue.cc:260] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.983840 30035 raft_consensus.cc:385] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.984283 30035 raft_consensus.cc:740] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.985795 30034 tablet_bootstrap.cc:654] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.987264 30037 ts_tablet_manager.cc:1434] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 13:10:26.989048 30037 tablet_bootstrap.cc:492] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:26.987563 30035 consensus_queue.cc:260] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:26.984860 30180 raft_consensus.cc:493] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.993444 30180 raft_consensus.cc:515] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:26.995940 29766 raft_consensus.cc:3060] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.006132 30035 ts_tablet_manager.cc:1434] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:10:27.007650 30035 tablet_bootstrap.cc:492] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:27.014227 30037 tablet_bootstrap.cc:654] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.015329 30035 tablet_bootstrap.cc:654] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.019251 29765 raft_consensus.cc:3060] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.029384 30211 consensus_queue.cc:1048] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:27.037091 30192 raft_consensus.cc:493] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.039908 30192 raft_consensus.cc:515] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.053195 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:27.053381 30180 leader_election.cc:290] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:27.054867 29807 leader_election.cc:343] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.057476 30147 raft_consensus.cc:493] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:27.059175 29881 consensus_peers.cc:597] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:27.058727 30147 raft_consensus.cc:515] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.062503 30147 leader_election.cc:290] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:27.062603 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.063756 29914 raft_consensus.cc:2468] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:27.048560 30211 consensus_queue.cc:1048] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:10:27.068632 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:27.067848 30180 consensus_queue.cc:1048] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 13:10:27.080312 29713 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:27.083583 29807 leader_election.cc:304] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:27.084411 30180 raft_consensus.cc:2804] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.084802 30069 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.084836 30180 raft_consensus.cc:493] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.085407 30069 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.085906 30180 raft_consensus.cc:3060] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.088841 30069 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:27.076823 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:27.099298 30180 raft_consensus.cc:515] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.100575 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:27.101444 30192 leader_election.cc:290] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:27.111136 30147 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.112689 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.112684 30147 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
W20260709 13:10:27.114282 29955 leader_election.cc:343] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.115373 30147 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:27.116274 29914 raft_consensus.cc:2468] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:27.117847 30034 tablet_bootstrap.cc:492] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:27.118745 30034 ts_tablet_manager.cc:1403] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.254s	user 0.013s	sys 0.009s
I20260709 13:10:27.107553 30193 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.119752 30193 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.122814 30193 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:27.102628 29881 leader_election.cc:343] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.127542 30147 consensus_queue.cc:1048] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.129199 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:27.129696 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.130606 29987 raft_consensus.cc:2468] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
W20260709 13:10:27.130784 29881 leader_election.cc:343] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.131939 29881 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:27.132823 30147 raft_consensus.cc:2804] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.133199 30147 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.133551 30147 raft_consensus.cc:3060] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.133843 29955 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:27.136492 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:27.138286 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.142339 30034 raft_consensus.cc:359] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.138330 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:27.147269 30128 raft_consensus.cc:2804] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.148052 30128 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.148535 30128 raft_consensus.cc:3060] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.149698 30036 tablet_bootstrap.cc:492] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:27.150185 30036 ts_tablet_manager.cc:1403] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.184s	user 0.016s	sys 0.034s
I20260709 13:10:27.153014 30036 raft_consensus.cc:359] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.153695 30036 raft_consensus.cc:385] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.153966 30036 raft_consensus.cc:740] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.154667 30036 consensus_queue.cc:260] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.143666 30034 raft_consensus.cc:385] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.156982 30037 tablet_bootstrap.cc:492] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:27.158113 30034 raft_consensus.cc:740] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.158483 30037 ts_tablet_manager.cc:1403] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.170s	user 0.017s	sys 0.004s
I20260709 13:10:27.159400 30034 consensus_queue.cc:260] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
W20260709 13:10:27.169082 29807 leader_election.cc:343] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.173981 30147 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.177096 30147 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
W20260709 13:10:27.157954 29732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:46707, user_credentials={real_user=slave}} blocked reactor thread for 84617.6us
W20260709 13:10:27.180559 29881 leader_election.cc:343] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.180686 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:27.181208 29881 leader_election.cc:304] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:27.181666 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:27.181973 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.183614 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:27.187114 29732 leader_election.cc:304] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:27.188954 30180 leader_election.cc:290] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:27.190619 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:27.191512 29914 raft_consensus.cc:3060] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.192198 30036 ts_tablet_manager.cc:1434] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:10:27.193506 30036 tablet_bootstrap.cc:492] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:27.200906 30034 ts_tablet_manager.cc:1434] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.082s	user 0.006s	sys 0.003s
I20260709 13:10:27.202366 30034 tablet_bootstrap.cc:492] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:27.203562 30193 raft_consensus.cc:493] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.204300 30193 raft_consensus.cc:515] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.207621 30088 raft_consensus.cc:2749] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:27.210649 29955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 76950.9us
W20260709 13:10:27.214546 29881 leader_election.cc:343] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.187896 30128 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.235028 30211 raft_consensus.cc:493] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.235733 30211 raft_consensus.cc:515] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.248605 30180 consensus_queue.cc:1048] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:27.259752 30257 raft_consensus.cc:493] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.260499 30257 raft_consensus.cc:515] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
W20260709 13:10:27.264303 29955 leader_election.cc:343] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:27.280998 29955 leader_election.cc:343] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.282905 29988 raft_consensus.cc:2393] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 74191176370841578d396cc01f230f06 in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:27.285154 29881 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:27.287147 30147 raft_consensus.cc:2749] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.281780 29955 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:27.293488 30261 raft_consensus.cc:2749] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.198558 30037 raft_consensus.cc:359] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.302045 30037 raft_consensus.cc:385] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.302547 30037 raft_consensus.cc:740] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.310770 29914 raft_consensus.cc:2468] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:27.311514 30257 leader_election.cc:290] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:27.312343 30035 tablet_bootstrap.cc:492] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:27.312892 30035 ts_tablet_manager.cc:1403] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.306s	user 0.037s	sys 0.023s
I20260709 13:10:27.313330 30193 leader_election.cc:290] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:27.314148 29807 leader_election.cc:304] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:27.314539 30211 leader_election.cc:290] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:27.315076 30180 raft_consensus.cc:2804] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:27.315784 30180 raft_consensus.cc:697] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:27.316048 30035 raft_consensus.cc:359] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.316879 30035 raft_consensus.cc:385] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.317186 30035 raft_consensus.cc:740] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.317071 30180 consensus_queue.cc:237] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.320436 30121 raft_consensus.cc:493] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:27.321234 30121 raft_consensus.cc:515] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.320358 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbc73ad9e6848228c0d2cae81843b7a" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.322472 29916 raft_consensus.cc:2393] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:27.323165 30035 consensus_queue.cc:260] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.324071 30121 leader_election.cc:290] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:27.324504 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e21634252d40b196da2d7ac6d5faa5" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.325238 30128 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:27.325686 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388e95cfde842949825ab8dbdc03e91" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:27.327448 29807 leader_election.cc:343] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.327312 30037 consensus_queue.cc:260] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.327795 30035 ts_tablet_manager.cc:1434] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
W20260709 13:10:27.327360 29955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 61655.2us
I20260709 13:10:27.328894 30035 tablet_bootstrap.cc:492] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:27.333081 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388e95cfde842949825ab8dbdc03e91" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.333899 30036 tablet_bootstrap.cc:654] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.336020 30035 tablet_bootstrap.cc:654] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.340915 29732 leader_election.cc:304] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:27.343179 30037 ts_tablet_manager.cc:1434] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.184s	user 0.007s	sys 0.000s
I20260709 13:10:27.339056 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e21634252d40b196da2d7ac6d5faa5" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
W20260709 13:10:27.345916 29807 leader_election.cc:343] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.346525 29807 leader_election.cc:304] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:27.347450 30107 raft_consensus.cc:2749] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.347898 30035 tablet_bootstrap.cc:492] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:27.348464 30035 ts_tablet_manager.cc:1403] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.003s
I20260709 13:10:27.348387 30075 raft_consensus.cc:2749] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:27.350675 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:27.351188 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3bf2e1f12f46c59f56c808bcfd5e78" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.351732 29916 raft_consensus.cc:2393] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 75162022240a4fe1a113333e77964ca3 in current term 1: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:27.351826 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3bf2e1f12f46c59f56c808bcfd5e78" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:27.353518 30257 raft_consensus.cc:2749] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:27.358385 30034 tablet_bootstrap.cc:654] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.358095 30107 consensus_queue.cc:1048] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:27.372462 29955 leader_election.cc:343] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.362195 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:27.363510 30037 tablet_bootstrap.cc:492] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:27.367128 30036 tablet_bootstrap.cc:492] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:27.376544 30036 ts_tablet_manager.cc:1403] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.183s	user 0.010s	sys 0.004s
I20260709 13:10:27.379678 30036 raft_consensus.cc:359] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.371455 30035 raft_consensus.cc:359] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.382006 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bbc73ad9e6848228c0d2cae81843b7a" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.382423 30036 raft_consensus.cc:385] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.382467 30035 raft_consensus.cc:385] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.382853 30036 raft_consensus.cc:740] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.382992 30035 raft_consensus.cc:740] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.383872 30035 consensus_queue.cc:260] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.379953 30193 consensus_queue.cc:1048] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:27.381124 29955 leader_election.cc:343] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.385157 29955 leader_election.cc:304] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:27.387197 30260 raft_consensus.cc:2749] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:27.389045 29955 leader_election.cc:343] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.389575 29955 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:27.390311 30069 raft_consensus.cc:2749] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.391072 29732 leader_election.cc:304] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:27.392320 30075 raft_consensus.cc:3060] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.391062 29664 catalog_manager.cc:5697] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.394205 30035 ts_tablet_manager.cc:1434] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:10:27.395653 30035 tablet_bootstrap.cc:492] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:27.400120 30088 raft_consensus.cc:493] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.400693 30088 raft_consensus.cc:515] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.402935 30088 leader_election.cc:290] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:27.406245 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce794c74850e4dc6845e02da675fa199" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.406926 29988 raft_consensus.cc:2468] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:27.410260 30075 raft_consensus.cc:2749] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.412293 30036 consensus_queue.cc:260] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.414003 30035 tablet_bootstrap.cc:654] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.414646 30036 ts_tablet_manager.cc:1434] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.038s	user 0.003s	sys 0.002s
I20260709 13:10:27.412307 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce794c74850e4dc6845e02da675fa199" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:27.426133 30263 log.cc:927] Time spent T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:10:27.429167 29881 leader_election.cc:343] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.430012 29881 leader_election.cc:304] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:27.434789 30147 raft_consensus.cc:2804] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.435303 30147 raft_consensus.cc:493] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.435722 30147 raft_consensus.cc:3060] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.446174 30037 tablet_bootstrap.cc:654] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.449519 30036 tablet_bootstrap.cc:492] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:27.450121 30211 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.450371 30035 tablet_bootstrap.cc:492] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:27.450964 30035 ts_tablet_manager.cc:1403] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.002s
I20260709 13:10:27.450696 30211 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.452963 30211 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:27.454864 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.454977 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.457640 29989 raft_consensus.cc:2468] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
W20260709 13:10:27.458765 29807 leader_election.cc:343] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.459659 29807 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06
I20260709 13:10:27.461740 30211 raft_consensus.cc:2804] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.462110 30035 raft_consensus.cc:359] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.463070 30211 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.463306 30035 raft_consensus.cc:385] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.463550 30211 raft_consensus.cc:3060] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.463802 30035 raft_consensus.cc:740] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.464808 30035 consensus_queue.cc:260] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.467257 30035 ts_tablet_manager.cc:1434] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:10:27.467923 30121 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.468291 30035 tablet_bootstrap.cc:492] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:27.468461 30121 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.470546 30121 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:27.472416 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.473239 29914 raft_consensus.cc:2393] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:27.480324 30035 tablet_bootstrap.cc:654] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.480484 30211 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.488574 30037 tablet_bootstrap.cc:492] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:27.487725 30034 tablet_bootstrap.cc:492] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:27.489238 30034 ts_tablet_manager.cc:1403] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.287s	user 0.039s	sys 0.055s
I20260709 13:10:27.491132 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:27.491528 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:27.489120 30069 consensus_queue.cc:1048] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:27.492092 29989 raft_consensus.cc:3060] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.490422 30211 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:27.489106 30037 ts_tablet_manager.cc:1403] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.001s
I20260709 13:10:27.493433 30034 raft_consensus.cc:359] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.494167 30034 raft_consensus.cc:385] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.494448 30034 raft_consensus.cc:740] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.495069 30034 consensus_queue.cc:260] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.485215 30036 tablet_bootstrap.cc:654] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.496357 30037 raft_consensus.cc:359] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
W20260709 13:10:27.496832 29807 leader_election.cc:343] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.497339 30037 raft_consensus.cc:385] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.497840 30037 raft_consensus.cc:740] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.498169 30075 raft_consensus.cc:493] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.498733 30075 raft_consensus.cc:515] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.500762 30075 leader_election.cc:290] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:27.502071 30034 ts_tablet_manager.cc:1434] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:10:27.502605 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.502717 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:27.503119 30034 tablet_bootstrap.cc:492] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:27.503561 29915 raft_consensus.cc:2393] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:27.504868 29732 leader_election.cc:304] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:27.506074 30075 raft_consensus.cc:3060] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.508790 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.510010 29987 raft_consensus.cc:2393] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:27.514654 30075 raft_consensus.cc:2749] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.518934 30037 consensus_queue.cc:260] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.516176 30034 tablet_bootstrap.cc:654] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.521525 29732 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:27.522795 30075 raft_consensus.cc:3060] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.515673 29989 raft_consensus.cc:2468] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:27.524950 29807 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06
I20260709 13:10:27.526142 30212 raft_consensus.cc:2804] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:27.526618 30212 raft_consensus.cc:697] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:27.527468 30212 consensus_queue.cc:237] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.486572 30147 raft_consensus.cc:515] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.532351 30075 raft_consensus.cc:2749] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.533850 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce794c74850e4dc6845e02da675fa199" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:27.534617 29987 raft_consensus.cc:3060] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.538053 30036 tablet_bootstrap.cc:492] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:27.538617 30036 ts_tablet_manager.cc:1403] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.010s
I20260709 13:10:27.540074 30035 tablet_bootstrap.cc:492] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:27.540781 30035 ts_tablet_manager.cc:1403] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.003s
I20260709 13:10:27.542198 30088 raft_consensus.cc:493] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.544212 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce794c74850e4dc6845e02da675fa199" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:27.544777 30036 raft_consensus.cc:359] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.545469 30036 raft_consensus.cc:385] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.545178 30035 raft_consensus.cc:359] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.545776 30036 raft_consensus.cc:740] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.542845 30088 raft_consensus.cc:515] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.546505 30036 consensus_queue.cc:260] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
W20260709 13:10:27.547435 29881 leader_election.cc:343] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.547302 30147 leader_election.cc:290] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:27.548995 30088 leader_election.cc:290] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:27.545909 30035 raft_consensus.cc:385] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.551860 30035 raft_consensus.cc:740] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.546833 29987 raft_consensus.cc:2468] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:27.552429 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3bf2e1f12f46c59f56c808bcfd5e78" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:27.552621 30035 consensus_queue.cc:260] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.549816 30037 ts_tablet_manager.cc:1434] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.000s
W20260709 13:10:27.554116 29881 leader_election.cc:343] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.554304 30037 tablet_bootstrap.cc:492] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:27.554991 30036 ts_tablet_manager.cc:1434] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:10:27.558033 30036 tablet_bootstrap.cc:492] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:27.555823 29881 leader_election.cc:304] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:27.563117 30037 tablet_bootstrap.cc:654] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.564617 30088 raft_consensus.cc:2804] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:27.565200 30088 raft_consensus.cc:697] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:27.566057 30088 consensus_queue.cc:237] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.568972 30035 ts_tablet_manager.cc:1434] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:10:27.569993 30035 tablet_bootstrap.cc:492] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:27.570724 30036 tablet_bootstrap.cc:654] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.578014 30035 tablet_bootstrap.cc:654] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.549453 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3bf2e1f12f46c59f56c808bcfd5e78" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.584146 29989 raft_consensus.cc:2468] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:27.586598 29881 leader_election.cc:304] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:27.588290 30147 raft_consensus.cc:2804] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.588775 30147 raft_consensus.cc:493] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.589184 30147 raft_consensus.cc:3060] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.616523 29664 catalog_manager.cc:5697] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.621286 29764 raft_consensus.cc:3060] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.627048 30037 tablet_bootstrap.cc:492] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:27.623766 30035 tablet_bootstrap.cc:492] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:27.627610 30037 ts_tablet_manager.cc:1403] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.003s
I20260709 13:10:27.627796 30035 ts_tablet_manager.cc:1403] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.000s
I20260709 13:10:27.630460 30035 raft_consensus.cc:359] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.631157 30035 raft_consensus.cc:385] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.631458 30035 raft_consensus.cc:740] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.632123 30035 consensus_queue.cc:260] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.634620 30035 ts_tablet_manager.cc:1434] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:27.635977 30035 tablet_bootstrap.cc:492] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:27.635876 30037 raft_consensus.cc:359] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.636742 30037 raft_consensus.cc:385] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.637238 30037 raft_consensus.cc:740] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.636780 30036 tablet_bootstrap.cc:492] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:27.639726 30036 ts_tablet_manager.cc:1403] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.009s
I20260709 13:10:27.643534 30035 tablet_bootstrap.cc:654] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.646250 30147 raft_consensus.cc:515] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.645049 30034 tablet_bootstrap.cc:492] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:27.650507 30034 ts_tablet_manager.cc:1403] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.148s	user 0.027s	sys 0.019s
I20260709 13:10:27.648033 30036 raft_consensus.cc:359] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.650923 30036 raft_consensus.cc:385] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.652635 30036 raft_consensus.cc:740] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.653487 30036 consensus_queue.cc:260] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.652171 30147 leader_election.cc:290] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:27.653807 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3bf2e1f12f46c59f56c808bcfd5e78" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:27.656104 30036 ts_tablet_manager.cc:1434] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
W20260709 13:10:27.656523 29881 leader_election.cc:343] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:27.640004 30037 consensus_queue.cc:260] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.658646 30036 tablet_bootstrap.cc:492] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:27.657423 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3bf2e1f12f46c59f56c808bcfd5e78" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:27.665922 29989 raft_consensus.cc:3060] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.672521 30036 tablet_bootstrap.cc:654] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.677204 30037 ts_tablet_manager.cc:1434] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 13:10:27.679216 29665 catalog_manager.cc:5697] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.674984 30034 raft_consensus.cc:359] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.685114 30034 raft_consensus.cc:385] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.685596 30034 raft_consensus.cc:740] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.686764 30034 consensus_queue.cc:260] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.687764 30121 raft_consensus.cc:493] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.690289 30121 raft_consensus.cc:515] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.693145 30121 leader_election.cc:290] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:27.696085 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:27.696730 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.697137 29841 raft_consensus.cc:2468] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:27.698943 29732 leader_election.cc:304] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:27.701697 30121 raft_consensus.cc:2804] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.702167 30121 raft_consensus.cc:493] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.702574 30121 raft_consensus.cc:3060] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.706005 30037 tablet_bootstrap.cc:492] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:27.724027 29987 raft_consensus.cc:2468] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:27.728207 30037 tablet_bootstrap.cc:654] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.729403 30035 tablet_bootstrap.cc:492] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:27.729980 30035 ts_tablet_manager.cc:1403] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.094s	user 0.021s	sys 0.065s
I20260709 13:10:27.733651 30035 raft_consensus.cc:359] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.734706 30035 raft_consensus.cc:385] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.735054 30035 raft_consensus.cc:740] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.736379 30035 consensus_queue.cc:260] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.737291 30037 tablet_bootstrap.cc:492] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:27.737854 30037 ts_tablet_manager.cc:1403] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.006s
I20260709 13:10:27.740162 30035 ts_tablet_manager.cc:1434] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:10:27.751093 30035 tablet_bootstrap.cc:492] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:27.750654 30121 raft_consensus.cc:515] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.752817 30036 tablet_bootstrap.cc:492] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:27.753283 30036 ts_tablet_manager.cc:1403] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.095s	user 0.021s	sys 0.026s
I20260709 13:10:27.733669 30034 ts_tablet_manager.cc:1434] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.083s	user 0.005s	sys 0.002s
I20260709 13:10:27.749670 30037 raft_consensus.cc:359] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.757393 30037 raft_consensus.cc:385] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.757576 30034 tablet_bootstrap.cc:492] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:27.757772 30037 raft_consensus.cc:740] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.758703 30037 consensus_queue.cc:260] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.749325 29989 raft_consensus.cc:2468] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:27.755959 30036 raft_consensus.cc:359] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.756906 30147 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.760342 30121 leader_election.cc:290] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:27.760545 30035 tablet_bootstrap.cc:654] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.760708 30036 raft_consensus.cc:385] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.761163 30036 raft_consensus.cc:740] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.760867 30147 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.761844 30036 consensus_queue.cc:260] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.763044 30147 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:27.764092 30036 ts_tablet_manager.cc:1434] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:10:27.765864 30036 tablet_bootstrap.cc:492] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:27.766070 30034 tablet_bootstrap.cc:654] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.766892 30037 ts_tablet_manager.cc:1434] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.005s
I20260709 13:10:27.768326 30037 tablet_bootstrap.cc:492] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:27.771963 30036 tablet_bootstrap.cc:654] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.774981 30035 tablet_bootstrap.cc:492] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:27.775697 30035 ts_tablet_manager.cc:1403] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.000s
I20260709 13:10:27.827073 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:27.828850 29841 raft_consensus.cc:3060] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.842006 29881 leader_election.cc:304] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:27.844081 30037 tablet_bootstrap.cc:654] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.845314 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:27.849314 30035 raft_consensus.cc:359] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.853551 30035 raft_consensus.cc:385] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.855367 30035 raft_consensus.cc:740] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.842105 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:27.852759 30075 raft_consensus.cc:493] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.859381 29989 raft_consensus.cc:3060] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.864450 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.865952 29987 raft_consensus.cc:2393] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74191176370841578d396cc01f230f06 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:27.868914 30075 raft_consensus.cc:515] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.871769 30147 raft_consensus.cc:2804] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:27.872603 30147 raft_consensus.cc:697] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:27.870745 30035 consensus_queue.cc:260] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:27.892132 30212 consensus_queue.cc:1048] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:27.898455 30107 raft_consensus.cc:493] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.899749 30035 ts_tablet_manager.cc:1434] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.124s	user 0.009s	sys 0.006s
I20260709 13:10:27.899271 30107 raft_consensus.cc:515] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.902000 30035 tablet_bootstrap.cc:492] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
W20260709 13:10:27.905997 29881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:46707, user_credentials={real_user=slave}} blocked reactor thread for 64586us
I20260709 13:10:27.906473 30107 leader_election.cc:290] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:27.875046 30147 consensus_queue.cc:237] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:27.911646 30035 tablet_bootstrap.cc:654] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.913669 30075 leader_election.cc:290] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:27.915680 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:27.916718 29840 raft_consensus.cc:2468] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:27.920852 29986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:27.924023 29986 raft_consensus.cc:2468] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:27.924928 30121 raft_consensus.cc:493] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.925691 30121 raft_consensus.cc:515] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.927995 30121 leader_election.cc:290] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:27.919729 29732 leader_election.cc:304] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:27.928469 29881 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:27.929291 30121 raft_consensus.cc:2804] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.932085 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:27.932288 30088 raft_consensus.cc:3060] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.933027 29842 raft_consensus.cc:2468] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:27.935962 29732 leader_election.cc:304] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:27.929806 30121 raft_consensus.cc:493] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.938771 30121 raft_consensus.cc:3060] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.939642 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:27.940258 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.941046 29914 raft_consensus.cc:2468] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:27.942466 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:27.943388 29915 raft_consensus.cc:2468] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:27.943440 30257 raft_consensus.cc:2804] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.944048 30257 raft_consensus.cc:493] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.944494 30257 raft_consensus.cc:3060] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.945007 30034 tablet_bootstrap.cc:492] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:27.945654 30034 ts_tablet_manager.cc:1403] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.188s	user 0.017s	sys 0.003s
I20260709 13:10:27.957834 30034 raft_consensus.cc:359] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.958794 30034 raft_consensus.cc:385] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.960237 30034 raft_consensus.cc:740] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.959270 30036 tablet_bootstrap.cc:492] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:27.962188 30212 consensus_queue.cc:1048] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:10:27.968461 30034 consensus_queue.cc:260] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:27.973237 30034 ts_tablet_manager.cc:1434] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 13:10:27.966672 30088 raft_consensus.cc:2749] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.981072 30036 ts_tablet_manager.cc:1403] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.215s	user 0.014s	sys 0.000s
I20260709 13:10:27.980756 29807 leader_election.cc:304] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:27.988102 30212 raft_consensus.cc:2804] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.988587 30212 raft_consensus.cc:493] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.988966 30212 raft_consensus.cc:3060] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.979031 30257 raft_consensus.cc:515] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:27.997921 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:27.998649 29842 raft_consensus.cc:3060] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.000138 30147 raft_consensus.cc:493] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.003598 30147 raft_consensus.cc:515] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.013672 30034 tablet_bootstrap.cc:492] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:28.017902 30147 leader_election.cc:290] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:27.985456 29841 raft_consensus.cc:2468] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:28.031211 30037 tablet_bootstrap.cc:492] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:27.992676 30036 raft_consensus.cc:359] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.030927 29989 raft_consensus.cc:2468] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:28.032003 30037 ts_tablet_manager.cc:1403] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.264s	user 0.005s	sys 0.015s
I20260709 13:10:28.031944 30036 raft_consensus.cc:385] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.033739 30036 raft_consensus.cc:740] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.038110 30036 consensus_queue.cc:260] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.038208 30212 raft_consensus.cc:515] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
W20260709 13:10:28.029021 29881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.196:46707, user_credentials={real_user=slave}} blocked reactor thread for 100690us
I20260709 13:10:28.044292 30035 tablet_bootstrap.cc:492] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:28.044799 30035 ts_tablet_manager.cc:1403] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.143s	user 0.011s	sys 0.023s
W20260709 13:10:28.037770 29807 leader_election.cc:343] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.046926 30036 ts_tablet_manager.cc:1434] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 13:10:28.040939 30257 leader_election.cc:290] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:28.042303 30037 raft_consensus.cc:359] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.050701 30037 raft_consensus.cc:385] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.050908 30037 raft_consensus.cc:740] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.059522 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:28.060261 29915 raft_consensus.cc:3060] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.061194 30035 raft_consensus.cc:359] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:28.063918 30036 tablet_bootstrap.cc:492] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:28.065207 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e21634252d40b196da2d7ac6d5faa5" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:28.066816 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e21634252d40b196da2d7ac6d5faa5" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:28.067680 29990 raft_consensus.cc:2468] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:28.066038 29841 raft_consensus.cc:2468] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:28.070952 30180 consensus_queue.cc:1048] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:28.062003 30035 raft_consensus.cc:385] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.071872 30035 raft_consensus.cc:740] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.072700 30035 consensus_queue.cc:260] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:28.076819 30035 ts_tablet_manager.cc:1434] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:10:28.082252 30037 consensus_queue.cc:260] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.061641 30121 raft_consensus.cc:515] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.079232 29842 raft_consensus.cc:2468] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:28.092540 29732 leader_election.cc:304] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:28.089986 30037 ts_tablet_manager.cc:1434] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.058s	user 0.000s	sys 0.007s
I20260709 13:10:28.097421 30075 raft_consensus.cc:2804] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:28.097997 30075 raft_consensus.cc:697] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:28.098969 30075 consensus_queue.cc:237] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.099732 30212 leader_election.cc:290] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:28.100700 30121 leader_election.cc:290] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:28.090828 30034 tablet_bootstrap.cc:654] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.103641 30036 tablet_bootstrap.cc:654] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.098439 30037 tablet_bootstrap.cc:492] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:28.099061 29732 leader_election.cc:304] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:28.108282 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:28.109012 29988 raft_consensus.cc:3060] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:28.111948 29807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 46479us
I20260709 13:10:28.120920 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:28.121714 29764 raft_consensus.cc:3060] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.112764 30257 raft_consensus.cc:2804] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:28.120959 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:28.125609 29914 raft_consensus.cc:3060] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.128082 30180 raft_consensus.cc:493] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.128702 30180 raft_consensus.cc:515] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.130561 30180 leader_election.cc:290] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:28.132608 29881 leader_election.cc:304] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:28.133836 30088 raft_consensus.cc:2804] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:28.134346 30088 raft_consensus.cc:493] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.134776 30088 raft_consensus.cc:3060] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.137569 30257 raft_consensus.cc:697] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:28.138947 30257 consensus_queue.cc:237] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:28.140831 29914 raft_consensus.cc:2468] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:28.143692 30186 consensus_queue.cc:1048] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:28.147647 29988 raft_consensus.cc:2468] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:28.148522 30035 tablet_bootstrap.cc:492] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:28.148582 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:28.149549 29842 raft_consensus.cc:3060] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:28.151600 29881 consensus_peers.cc:597] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:28.153256 29732 leader_election.cc:304] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:28.154094 30121 raft_consensus.cc:2804] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:28.154608 30121 raft_consensus.cc:697] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:28.155583 30121 consensus_queue.cc:237] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.158546 29842 raft_consensus.cc:2468] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:28.160709 29764 raft_consensus.cc:2468] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:28.168709 29807 leader_election.cc:304] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:28.175580 30212 raft_consensus.cc:2804] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:28.182858 30037 tablet_bootstrap.cc:654] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.203836 30298 consensus_queue.cc:1048] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:10:28.207444 30088 raft_consensus.cc:515] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.209812 30088 leader_election.cc:290] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:28.210923 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e21634252d40b196da2d7ac6d5faa5" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:28.212172 29989 raft_consensus.cc:3060] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.213912 30034 tablet_bootstrap.cc:492] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:28.214637 30034 ts_tablet_manager.cc:1403] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.206s	user 0.016s	sys 0.000s
I20260709 13:10:28.218662 30034 raft_consensus.cc:359] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.219949 30034 raft_consensus.cc:385] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.220285 30034 raft_consensus.cc:740] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.185173 29915 raft_consensus.cc:2468] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:28.210407 29661 catalog_manager.cc:5697] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.239395 30212 raft_consensus.cc:697] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
W20260709 13:10:28.241219 29807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 53116.9us
I20260709 13:10:28.245942 30034 consensus_queue.cc:260] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.248953 30257 raft_consensus.cc:493] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.249567 30257 raft_consensus.cc:515] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.253489 30034 ts_tablet_manager.cc:1434] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:10:28.256150 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e21634252d40b196da2d7ac6d5faa5" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:28.256959 29842 raft_consensus.cc:3060] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.259158 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:28.261387 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:28.262126 29987 raft_consensus.cc:2468] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:28.264765 30257 leader_election.cc:290] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:28.266796 29764 raft_consensus.cc:2468] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:28.274260 29989 raft_consensus.cc:2468] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:28.277129 29986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:28.277484 29881 leader_election.cc:304] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:28.277843 29986 raft_consensus.cc:2468] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:28.278389 30186 raft_consensus.cc:2804] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:28.278867 30186 raft_consensus.cc:697] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:28.279894 30034 tablet_bootstrap.cc:492] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:28.279901 30186 consensus_queue.cc:237] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.284916 29732 leader_election.cc:304] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:28.285717 29665 catalog_manager.cc:5697] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.287683 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:28.288398 29840 raft_consensus.cc:2468] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:28.273301 30212 consensus_queue.cc:237] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:28.291666 30035 tablet_bootstrap.cc:654] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.291095 29665 catalog_manager.cc:5697] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.294344 30257 raft_consensus.cc:2804] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:28.295048 30257 raft_consensus.cc:493] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.295574 30257 raft_consensus.cc:3060] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.302627 30034 tablet_bootstrap.cc:654] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.317421 29807 leader_election.cc:304] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:28.327237 30309 raft_consensus.cc:2804] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:28.327756 30309 raft_consensus.cc:493] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.328109 30037 tablet_bootstrap.cc:492] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:28.328094 30309 raft_consensus.cc:3060] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.331508 30036 tablet_bootstrap.cc:492] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:28.332080 30036 ts_tablet_manager.cc:1403] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.269s	user 0.014s	sys 0.009s
I20260709 13:10:28.333683 30037 ts_tablet_manager.cc:1403] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.236s	user 0.011s	sys 0.006s
I20260709 13:10:28.342190 30036 raft_consensus.cc:359] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.343365 30036 raft_consensus.cc:385] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.343658 30036 raft_consensus.cc:740] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.344355 30036 consensus_queue.cc:260] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.321969 29842 raft_consensus.cc:2468] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:28.347095 30036 ts_tablet_manager.cc:1434] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:10:28.348491 30036 tablet_bootstrap.cc:492] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:28.342008 30309 raft_consensus.cc:515] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
W20260709 13:10:28.358134 29713 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:28.363570 30037 raft_consensus.cc:359] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.364920 30037 raft_consensus.cc:385] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.365181 30037 raft_consensus.cc:740] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.376773 30035 tablet_bootstrap.cc:492] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:28.379220 30035 ts_tablet_manager.cc:1403] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.257s	user 0.019s	sys 0.000s
I20260709 13:10:28.372007 30309 leader_election.cc:290] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:28.374161 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:28.382650 30037 consensus_queue.cc:260] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.388339 30037 ts_tablet_manager.cc:1434] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 13:10:28.384666 30257 raft_consensus.cc:515] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.391443 30037 tablet_bootstrap.cc:492] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:28.392951 30035 raft_consensus.cc:359] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.393795 30035 raft_consensus.cc:385] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.393287 30257 leader_election.cc:290] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:28.397029 30186 consensus_queue.cc:1048] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:28.404454 29767 raft_consensus.cc:2393] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 1: Already voted for candidate 2ba28923c4534c78903f759f0a4e1265 in this term.
I20260709 13:10:28.404840 29986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:28.410672 29986 raft_consensus.cc:3060] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.405284 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:28.394124 30035 raft_consensus.cc:740] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.409188 30036 tablet_bootstrap.cc:654] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.414884 29841 raft_consensus.cc:2393] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:28.414747 30035 consensus_queue.cc:260] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.427536 29664 catalog_manager.cc:5697] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.431406 30180 raft_consensus.cc:493] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.432056 30180 raft_consensus.cc:515] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.433439 30035 ts_tablet_manager.cc:1434] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:10:28.434234 30180 leader_election.cc:290] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:28.434720 30035 tablet_bootstrap.cc:492] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:28.436064 29986 raft_consensus.cc:2468] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:28.436641 29764 raft_consensus.cc:3060] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.437675 29732 leader_election.cc:304] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:28.443895 29665 catalog_manager.cc:5697] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.439851 30147 raft_consensus.cc:493] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.448263 30257 raft_consensus.cc:2804] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:28.461426 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86eb4af5624466bb2d9395a36632d71" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:28.457558 30147 raft_consensus.cc:515] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.468328 30147 leader_election.cc:290] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:28.468987 30257 raft_consensus.cc:697] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:28.470026 30257 consensus_queue.cc:237] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.470250 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:28.471616 29990 raft_consensus.cc:2393] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 1: Already voted for candidate 2ba28923c4534c78903f759f0a4e1265 in this term.
I20260709 13:10:28.468149 29986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86eb4af5624466bb2d9395a36632d71" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:28.474679 29807 leader_election.cc:343] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:28.481993 29807 leader_election.cc:343] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.482366 29807 leader_election.cc:304] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:28.485195 29807 leader_election.cc:304] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:28.493137 30036 tablet_bootstrap.cc:492] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:28.497277 30036 ts_tablet_manager.cc:1403] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.149s	user 0.012s	sys 0.003s
I20260709 13:10:28.485076 30180 raft_consensus.cc:2749] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:28.511227 30107 raft_consensus.cc:2749] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:28.512488 30037 tablet_bootstrap.cc:654] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.503248 30036 raft_consensus.cc:359] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:28.514556 30036 raft_consensus.cc:385] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.515651 30036 raft_consensus.cc:740] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.516609 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:28.517140 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:28.530467 29665 catalog_manager.cc:5697] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.535092 30107 raft_consensus.cc:493] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.535709 30107 raft_consensus.cc:515] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.538295 30107 leader_election.cc:290] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:28.538997 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:28.529211 30036 consensus_queue.cc:260] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
W20260709 13:10:28.542374 29807 leader_election.cc:343] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.547079 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:28.550316 30034 tablet_bootstrap.cc:492] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:28.551056 30034 ts_tablet_manager.cc:1403] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.271s	user 0.012s	sys 0.003s
W20260709 13:10:28.552178 29807 leader_election.cc:343] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.552662 29807 leader_election.cc:304] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:28.553336 30180 raft_consensus.cc:2749] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:28.554154 30295 consensus_queue.cc:1048] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:28.556882 30036 ts_tablet_manager.cc:1434] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.004s
I20260709 13:10:28.554364 30034 raft_consensus.cc:359] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.573740 30034 raft_consensus.cc:385] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.574034 30034 raft_consensus.cc:740] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.579957 30034 consensus_queue.cc:260] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.591141 30036 tablet_bootstrap.cc:492] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:28.600641 30257 consensus_queue.cc:1048] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:28.607522 30034 ts_tablet_manager.cc:1434] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.001s
W20260709 13:10:28.608351 29881 leader_election.cc:343] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.608759 30034 tablet_bootstrap.cc:492] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
W20260709 13:10:28.611270 29881 consensus_peers.cc:597] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:28.615656 29881 leader_election.cc:343] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.619807 30035 tablet_bootstrap.cc:654] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.622383 30257 consensus_queue.cc:1048] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:28.630223 29661 catalog_manager.cc:5697] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.618721 29881 leader_election.cc:304] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:28.631974 30186 raft_consensus.cc:2749] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:28.637537 30037 tablet_bootstrap.cc:492] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:28.638094 30037 ts_tablet_manager.cc:1403] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.247s	user 0.014s	sys 0.000s
I20260709 13:10:28.641304 30034 tablet_bootstrap.cc:654] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.648188 30036 tablet_bootstrap.cc:654] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.649953 30295 consensus_queue.cc:1048] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:28.675841 30121 consensus_queue.cc:1048] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:28.694226 30186 raft_consensus.cc:493] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.694957 30186 raft_consensus.cc:515] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:28.697957 30186 leader_election.cc:290] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:28.701611 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:28.707433 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:28.709993 29881 leader_election.cc:343] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:28.711001 29881 leader_election.cc:343] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.712487 29881 leader_election.cc:304] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:28.714601 30037 raft_consensus.cc:359] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.715371 30037 raft_consensus.cc:385] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.715659 30037 raft_consensus.cc:740] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.716549 30035 tablet_bootstrap.cc:492] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:28.716888 30186 raft_consensus.cc:2749] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:28.717631 30035 ts_tablet_manager.cc:1403] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.283s	user 0.020s	sys 0.000s
I20260709 13:10:28.718012 30322 raft_consensus.cc:493] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.718516 30322 raft_consensus.cc:515] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.722847 30036 tablet_bootstrap.cc:492] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:28.723565 30036 ts_tablet_manager.cc:1403] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.160s	user 0.024s	sys 0.008s
I20260709 13:10:28.721794 30035 raft_consensus.cc:359] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.723758 30186 consensus_queue.cc:1048] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:28.724185 30035 raft_consensus.cc:385] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.724655 30035 raft_consensus.cc:740] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.722865 30322 leader_election.cc:290] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:28.729741 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8170c15c8bf942b596d738aa14bb6f80" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:28.731596 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8170c15c8bf942b596d738aa14bb6f80" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:28.725410 30035 consensus_queue.cc:260] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.734885 30035 ts_tablet_manager.cc:1434] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:10:28.736337 30035 tablet_bootstrap.cc:492] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:28.741736 30295 consensus_queue.cc:1048] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:28.737759 30036 raft_consensus.cc:359] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.739193 30180 consensus_queue.cc:1048] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:28.742955 29881 leader_election.cc:343] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.716349 30037 consensus_queue.cc:260] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.747025 29841 raft_consensus.cc:2468] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:28.749327 29881 leader_election.cc:304] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 75162022240a4fe1a113333e77964ca3
I20260709 13:10:28.742996 30036 raft_consensus.cc:385] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.750558 30036 raft_consensus.cc:740] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.747171 30037 ts_tablet_manager.cc:1434] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.109s	user 0.008s	sys 0.000s
I20260709 13:10:28.751148 30036 consensus_queue.cc:260] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.756800 30323 raft_consensus.cc:2804] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:28.757189 30034 tablet_bootstrap.cc:492] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:28.757707 30034 ts_tablet_manager.cc:1403] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.149s	user 0.007s	sys 0.005s
I20260709 13:10:28.757252 30323 raft_consensus.cc:493] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.758435 30323 raft_consensus.cc:3060] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.761308 30036 ts_tablet_manager.cc:1434] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 13:10:28.760975 30034 raft_consensus.cc:359] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.761926 30034 raft_consensus.cc:385] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.762267 30034 raft_consensus.cc:740] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.765534 30035 tablet_bootstrap.cc:654] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.763155 30034 consensus_queue.cc:260] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.768858 30034 ts_tablet_manager.cc:1434] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:10:28.772432 30088 raft_consensus.cc:493] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.773072 30088 raft_consensus.cc:515] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.773430 30036 tablet_bootstrap.cc:492] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:28.775750 30034 tablet_bootstrap.cc:492] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:28.780906 30323 raft_consensus.cc:515] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.783778 30088 leader_election.cc:290] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:28.785137 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96641d56c48649d993e0eec66d091ee5" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:28.787971 29881 leader_election.cc:343] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.798868 30306 raft_consensus.cc:493] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.799726 30306 raft_consensus.cc:515] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.803694 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:28.804417 29915 raft_consensus.cc:2468] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:28.804796 30306 leader_election.cc:290] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:28.805022 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:28.806005 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96641d56c48649d993e0eec66d091ee5" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:28.807096 29955 leader_election.cc:343] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:28.808750 29881 leader_election.cc:343] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.809303 29881 leader_election.cc:304] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:28.810166 30186 raft_consensus.cc:2749] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:28.811968 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8170c15c8bf942b596d738aa14bb6f80" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:28.812827 29841 raft_consensus.cc:3060] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.814864 29955 leader_election.cc:304] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:28.815749 30306 raft_consensus.cc:2804] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:28.816126 30306 raft_consensus.cc:493] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.816475 30306 raft_consensus.cc:3060] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.817529 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8170c15c8bf942b596d738aa14bb6f80" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:28.824714 30034 tablet_bootstrap.cc:654] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:28.825052 29881 leader_election.cc:343] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.826048 30036 tablet_bootstrap.cc:654] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.828743 30037 tablet_bootstrap.cc:492] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:28.811182 30323 leader_election.cc:290] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:28.836776 30322 consensus_queue.cc:1048] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:28.837224 29841 raft_consensus.cc:2468] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:28.840967 30035 tablet_bootstrap.cc:492] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:28.841518 30035 ts_tablet_manager.cc:1403] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.105s	user 0.020s	sys 0.022s
I20260709 13:10:28.845393 30037 tablet_bootstrap.cc:654] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.849089 29881 leader_election.cc:304] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 75162022240a4fe1a113333e77964ca3
I20260709 13:10:28.851810 30295 raft_consensus.cc:493] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.852385 30295 raft_consensus.cc:515] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.851965 30323 raft_consensus.cc:2804] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:28.853506 30323 raft_consensus.cc:697] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:28.853590 30306 raft_consensus.cc:515] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.854543 30295 leader_election.cc:290] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:28.857645 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3bf2e1f12f46c59f56c808bcfd5e78" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:28.858148 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3bf2e1f12f46c59f56c808bcfd5e78" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:28.859738 30035 raft_consensus.cc:359] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:28.866036 30323 consensus_queue.cc:237] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.867422 30298 consensus_queue.cc:1048] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:28.873180 30035 raft_consensus.cc:385] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.873594 30035 raft_consensus.cc:740] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.874547 30035 consensus_queue.cc:260] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:28.888105 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:28.888736 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:28.889485 29915 raft_consensus.cc:3060] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:28.889698 29955 leader_election.cc:343] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:28.888131 29732 leader_election.cc:304] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:28.898546 30257 raft_consensus.cc:2749] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:28.891592 30306 leader_election.cc:290] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:28.909317 30035 ts_tablet_manager.cc:1434] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:10:28.911223 30035 tablet_bootstrap.cc:492] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:28.911540 30036 tablet_bootstrap.cc:492] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:28.913252 30036 ts_tablet_manager.cc:1403] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.140s	user 0.017s	sys 0.011s
I20260709 13:10:28.916301 30036 raft_consensus.cc:359] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.918298 30036 raft_consensus.cc:385] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.919667 30037 tablet_bootstrap.cc:492] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:28.920375 30037 ts_tablet_manager.cc:1403] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.124s	user 0.016s	sys 0.006s
I20260709 13:10:28.909523 29915 raft_consensus.cc:2468] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:28.919126 30036 raft_consensus.cc:740] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.922169 29955 leader_election.cc:304] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:28.924928 30036 consensus_queue.cc:260] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.925612 30307 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.926270 30307 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:28.927753 30036 ts_tablet_manager.cc:1434] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:10:28.927584 30037 raft_consensus.cc:359] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.928416 30037 raft_consensus.cc:385] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.928551 30307 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:28.928733 30037 raft_consensus.cc:740] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.929105 30036 tablet_bootstrap.cc:492] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:28.931496 30306 raft_consensus.cc:2804] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:28.931973 30257 consensus_queue.cc:1048] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:28.932164 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:28.936942 30035 tablet_bootstrap.cc:654] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.938966 29765 raft_consensus.cc:3060] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:28.939420 30034 tablet_bootstrap.cc:492] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:28.939921 30034 ts_tablet_manager.cc:1403] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.166s	user 0.011s	sys 0.009s
I20260709 13:10:28.940778 29764 raft_consensus.cc:2468] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:28.942839 29955 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:28.943230 30036 tablet_bootstrap.cc:654] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:28.943667 30307 raft_consensus.cc:2804] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:28.944094 30307 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.944455 30307 raft_consensus.cc:3060] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:28.946372 30034 raft_consensus.cc:359] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.947106 30034 raft_consensus.cc:385] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:28.947386 30034 raft_consensus.cc:740] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:28.932756 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:28.934152 30037 consensus_queue.cc:260] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:28.948093 30034 consensus_queue.cc:260] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:28.949970 29915 raft_consensus.cc:2468] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:28.951310 30037 ts_tablet_manager.cc:1434] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.005s
I20260709 13:10:28.967219 30037 tablet_bootstrap.cc:492] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:28.986717 29663 catalog_manager.cc:5697] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.996171 30037 tablet_bootstrap.cc:654] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.014173 30034 ts_tablet_manager.cc:1434] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.074s	user 0.014s	sys 0.008s
I20260709 13:10:29.007349 30306 raft_consensus.cc:697] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:29.021003 30036 tablet_bootstrap.cc:492] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:29.022764 30036 ts_tablet_manager.cc:1403] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.094s	user 0.010s	sys 0.004s
I20260709 13:10:29.003263 30307 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.021389 30306 consensus_queue.cc:237] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.032677 30034 tablet_bootstrap.cc:492] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:29.040115 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:29.040853 29913 raft_consensus.cc:3060] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:29.054950 30295 consensus_queue.cc:1048] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:29.059316 30036 raft_consensus.cc:359] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.061177 29913 raft_consensus.cc:2468] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:29.061647 30036 raft_consensus.cc:385] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.062311 30036 raft_consensus.cc:740] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.063066 30307 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:29.063900 30036 consensus_queue.cc:260] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.068506 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:29.069342 29767 raft_consensus.cc:3060] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:29.072952 30035 tablet_bootstrap.cc:492] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:29.073468 30035 ts_tablet_manager.cc:1403] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.163s	user 0.007s	sys 0.008s
I20260709 13:10:29.076973 29955 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:29.079304 30034 tablet_bootstrap.cc:654] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.081851 30354 raft_consensus.cc:2804] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:29.082428 30354 raft_consensus.cc:697] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:29.083376 30354 consensus_queue.cc:237] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.085794 30036 ts_tablet_manager.cc:1434] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.063s	user 0.005s	sys 0.001s
I20260709 13:10:29.092608 30036 tablet_bootstrap.cc:492] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:29.103758 29767 raft_consensus.cc:2468] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:29.105772 30035 raft_consensus.cc:359] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.107362 30035 raft_consensus.cc:385] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.107944 30035 raft_consensus.cc:740] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.109283 30035 consensus_queue.cc:260] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.119537 30035 ts_tablet_manager.cc:1434] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.004s
I20260709 13:10:29.122869 30257 raft_consensus.cc:493] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.123433 30257 raft_consensus.cc:515] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.133636 30035 tablet_bootstrap.cc:492] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:29.135510 30037 tablet_bootstrap.cc:492] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:29.132192 29663 catalog_manager.cc:5697] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:29.136040 30037 ts_tablet_manager.cc:1403] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.172s	user 0.036s	sys 0.019s
I20260709 13:10:29.119602 30354 raft_consensus.cc:493] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.143038 30037 raft_consensus.cc:359] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.143916 30037 raft_consensus.cc:385] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.144225 30037 raft_consensus.cc:740] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.142987 30354 raft_consensus.cc:515] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.146432 30037 consensus_queue.cc:260] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.151960 30257 leader_election.cc:290] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.156692 30037 ts_tablet_manager.cc:1434] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:10:29.158577 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:29.159261 29915 raft_consensus.cc:2468] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:29.152402 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:29.160075 29913 raft_consensus.cc:2468] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:29.159297 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:29.153251 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:29.162690 29841 raft_consensus.cc:2468] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:29.162824 29840 raft_consensus.cc:2468] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:29.153810 30036 tablet_bootstrap.cc:654] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.168527 29955 leader_election.cc:304] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:29.169273 30307 raft_consensus.cc:2804] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:29.169668 30307 raft_consensus.cc:493] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.170008 30307 raft_consensus.cc:3060] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.170585 29732 leader_election.cc:304] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:29.171687 30075 raft_consensus.cc:2804] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:29.172595 30075 raft_consensus.cc:493] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.173313 30075 raft_consensus.cc:3060] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.173686 30322 raft_consensus.cc:493] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.174772 30322 raft_consensus.cc:515] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.178212 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:29.178695 30322 leader_election.cc:290] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:29.179242 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:29.180256 30037 tablet_bootstrap.cc:492] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
W20260709 13:10:29.180172 29881 leader_election.cc:343] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:29.181253 29881 leader_election.cc:343] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.181675 29881 leader_election.cc:304] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:29.183192 30322 raft_consensus.cc:2749] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:29.183982 30354 leader_election.cc:290] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.185211 30034 tablet_bootstrap.cc:492] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:29.185734 30034 ts_tablet_manager.cc:1403] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.153s	user 0.008s	sys 0.006s
I20260709 13:10:29.187366 30037 tablet_bootstrap.cc:654] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.192708 30307 raft_consensus.cc:515] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.195151 30035 tablet_bootstrap.cc:654] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.194828 30034 raft_consensus.cc:359] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.195758 30034 raft_consensus.cc:385] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.196110 30034 raft_consensus.cc:740] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.196965 30034 consensus_queue.cc:260] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.197355 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:29.198000 29913 raft_consensus.cc:3060] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.199331 30034 ts_tablet_manager.cc:1434] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 13:10:29.200682 30034 tablet_bootstrap.cc:492] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:29.196981 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:29.203451 29665 catalog_manager.cc:5697] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 2, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 2 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:29.204442 29841 raft_consensus.cc:3060] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.205953 30036 tablet_bootstrap.cc:492] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:29.206450 30036 ts_tablet_manager.cc:1403] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.114s	user 0.008s	sys 0.012s
I20260709 13:10:29.209250 30036 raft_consensus.cc:359] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.209878 30036 raft_consensus.cc:385] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.210130 30036 raft_consensus.cc:740] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.212275 30035 tablet_bootstrap.cc:492] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:29.213574 30075 raft_consensus.cc:515] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.214538 30036 consensus_queue.cc:260] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.219877 30036 ts_tablet_manager.cc:1434] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:10:29.215298 30035 ts_tablet_manager.cc:1403] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.001s
I20260709 13:10:29.194864 30307 leader_election.cc:290] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.224905 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:29.231220 30034 tablet_bootstrap.cc:654] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.228986 30036 tablet_bootstrap.cc:492] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:29.231920 30075 leader_election.cc:290] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.233150 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:29.229099 30037 tablet_bootstrap.cc:492] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:29.224088 29841 raft_consensus.cc:2468] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:29.234274 30037 ts_tablet_manager.cc:1403] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.001s
I20260709 13:10:29.225253 29913 raft_consensus.cc:2468] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:29.230321 30035 raft_consensus.cc:359] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.235349 29955 leader_election.cc:304] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:29.233731 29915 raft_consensus.cc:3060] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.236182 30307 raft_consensus.cc:2804] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:29.236660 30307 raft_consensus.cc:697] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:29.237553 30307 consensus_queue.cc:237] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.222048 30354 raft_consensus.cc:493] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.239498 30354 raft_consensus.cc:515] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.244612 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86eb4af5624466bb2d9395a36632d71" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:29.244894 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86eb4af5624466bb2d9395a36632d71" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:29.245540 29839 raft_consensus.cc:2468] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:29.245574 29767 raft_consensus.cc:2468] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:29.247758 29955 leader_election.cc:304] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:29.248565 30306 raft_consensus.cc:2804] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:29.248955 30306 raft_consensus.cc:493] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.242352 30354 leader_election.cc:290] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:29.249297 30306 raft_consensus.cc:3060] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.231678 29840 raft_consensus.cc:3060] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.235527 30035 raft_consensus.cc:385] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.252194 30037 raft_consensus.cc:359] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.253285 30037 raft_consensus.cc:385] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.253567 30037 raft_consensus.cc:740] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.245818 30036 tablet_bootstrap.cc:654] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.255892 29915 raft_consensus.cc:2468] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:29.252147 30035 raft_consensus.cc:740] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.256675 30037 consensus_queue.cc:260] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.257522 29732 leader_election.cc:304] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:29.260253 30035 consensus_queue.cc:260] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.260762 30075 raft_consensus.cc:2804] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:29.262534 30037 ts_tablet_manager.cc:1434] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:10:29.265605 30037 tablet_bootstrap.cc:492] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:29.272121 30075 raft_consensus.cc:697] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:29.273121 30075 consensus_queue.cc:237] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.279364 30354 raft_consensus.cc:493] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.281813 30354 raft_consensus.cc:515] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.284260 30354 leader_election.cc:290] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.284754 30035 ts_tablet_manager.cc:1434] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.000s
I20260709 13:10:29.286369 30035 tablet_bootstrap.cc:492] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:29.287999 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:29.288153 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:29.288244 29840 raft_consensus.cc:2468] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:29.288856 29915 raft_consensus.cc:2468] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
W20260709 13:10:29.291023 29955 leader_election.cc:343] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.291676 29955 leader_election.cc:304] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:29.292398 30354 raft_consensus.cc:2804] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:29.292768 30354 raft_consensus.cc:493] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.293104 30354 raft_consensus.cc:3060] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.297863 30306 raft_consensus.cc:515] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.301095 30034 tablet_bootstrap.cc:492] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:29.301699 30034 ts_tablet_manager.cc:1403] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.101s	user 0.009s	sys 0.008s
I20260709 13:10:29.307606 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86eb4af5624466bb2d9395a36632d71" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:29.308344 29767 raft_consensus.cc:3060] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.308706 30354 raft_consensus.cc:515] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.311559 30354 leader_election.cc:290] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.314213 30037 tablet_bootstrap.cc:654] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.314404 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:29.315672 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:29.316264 29915 raft_consensus.cc:3060] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:29.318856 29955 leader_election.cc:343] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.321813 30035 tablet_bootstrap.cc:654] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.333421 30036 tablet_bootstrap.cc:492] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:29.334319 29767 raft_consensus.cc:2468] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:29.334964 30036 ts_tablet_manager.cc:1403] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.106s	user 0.015s	sys 0.015s
I20260709 13:10:29.336901 29915 raft_consensus.cc:2468] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:29.337680 30306 leader_election.cc:290] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:29.337082 29665 catalog_manager.cc:5697] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:29.352505 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86eb4af5624466bb2d9395a36632d71" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:29.353216 29841 raft_consensus.cc:3060] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.354090 30034 raft_consensus.cc:359] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.354791 30034 raft_consensus.cc:385] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.355074 30034 raft_consensus.cc:740] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.357311 30298 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.357838 30298 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.360707 30036 raft_consensus.cc:359] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.361554 30036 raft_consensus.cc:385] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.360129 30034 consensus_queue.cc:260] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.361826 30036 raft_consensus.cc:740] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.365836 30036 consensus_queue.cc:260] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.366747 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:29.367084 29955 leader_election.cc:304] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
W20260709 13:10:29.368701 29807 leader_election.cc:343] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.370173 30306 raft_consensus.cc:2804] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:29.374511 30298 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.375360 30034 ts_tablet_manager.cc:1434] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.000s
W20260709 13:10:29.377707 29881 consensus_peers.cc:597] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 -> Peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Couldn't send request to peer 75162022240a4fe1a113333e77964ca3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:29.381808 29661 catalog_manager.cc:5697] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:29.384235 30034 tablet_bootstrap.cc:492] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:29.387532 30036 ts_tablet_manager.cc:1434] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.003s
I20260709 13:10:29.389063 30037 tablet_bootstrap.cc:492] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:29.389494 30036 tablet_bootstrap.cc:492] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:29.389583 30037 ts_tablet_manager.cc:1403] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.124s	user 0.021s	sys 0.018s
I20260709 13:10:29.385601 30309 raft_consensus.cc:493] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.392575 30309 raft_consensus.cc:515] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.394662 30309 leader_election.cc:290] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:29.395004 30037 raft_consensus.cc:359] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.395768 30037 raft_consensus.cc:385] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.396034 30037 raft_consensus.cc:740] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.396715 30037 consensus_queue.cc:260] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.398193 29955 leader_election.cc:304] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:29.384209 30306 raft_consensus.cc:697] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:29.400704 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:29.401494 29988 raft_consensus.cc:2468] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:29.402912 29807 leader_election.cc:304] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:29.403357 30354 raft_consensus.cc:2804] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:29.403766 30309 raft_consensus.cc:2804] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:29.403964 30354 raft_consensus.cc:697] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:29.404227 30309 raft_consensus.cc:493] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.404605 30309 raft_consensus.cc:3060] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.405222 30354 consensus_queue.cc:237] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.409581 30306 consensus_queue.cc:237] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.410668 30036 tablet_bootstrap.cc:654] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.412580 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:29.413331 29915 raft_consensus.cc:2468] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:29.416625 29807 leader_election.cc:304] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:29.419459 30034 tablet_bootstrap.cc:654] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.419796 30037 ts_tablet_manager.cc:1434] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:10:29.424305 30298 raft_consensus.cc:2804] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:29.433162 30298 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.433584 30298 raft_consensus.cc:3060] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.432228 30121 raft_consensus.cc:493] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.434815 30121 raft_consensus.cc:515] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.437268 30121 leader_election.cc:290] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:29.439254 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:29.440017 29987 raft_consensus.cc:2468] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:29.442149 29732 leader_election.cc:304] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:29.426734 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:29.442965 30257 raft_consensus.cc:2804] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:29.443465 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:29.445637 30147 consensus_queue.cc:1048] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:29.428743 29841 raft_consensus.cc:2468] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:29.458880 30037 tablet_bootstrap.cc:492] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:29.443529 29767 raft_consensus.cc:2468] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:29.466071 30035 tablet_bootstrap.cc:492] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:29.467334 30035 ts_tablet_manager.cc:1403] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.181s	user 0.017s	sys 0.011s
I20260709 13:10:29.466885 30257 raft_consensus.cc:493] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.475073 30257 raft_consensus.cc:3060] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.481688 30035 raft_consensus.cc:359] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.482542 30035 raft_consensus.cc:385] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.482856 30035 raft_consensus.cc:740] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.483270 30257 raft_consensus.cc:515] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.486791 30309 raft_consensus.cc:515] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.488454 29840 raft_consensus.cc:2393] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:29.483649 30035 consensus_queue.cc:260] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.501683 30147 raft_consensus.cc:493] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.502458 30035 ts_tablet_manager.cc:1434] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.002s
I20260709 13:10:29.502307 30147 raft_consensus.cc:515] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.504715 30147 leader_election.cc:290] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:29.505868 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:29.507656 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:29.507575 29987 raft_consensus.cc:3060] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:29.509166 29881 leader_election.cc:343] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.510272 30037 tablet_bootstrap.cc:654] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.514482 30298 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.526737 30036 tablet_bootstrap.cc:492] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:29.527503 30036 ts_tablet_manager.cc:1403] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.004s
I20260709 13:10:29.530381 30035 tablet_bootstrap.cc:492] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:29.526083 30309 leader_election.cc:290] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:29.536978 29987 raft_consensus.cc:2468] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:29.538877 30298 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.539368 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:29.542240 30257 leader_election.cc:290] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:29.542356 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:29.545513 29915 raft_consensus.cc:1275] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:29.543229 30036 raft_consensus.cc:359] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.546572 30036 raft_consensus.cc:385] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.546885 30036 raft_consensus.cc:740] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.547683 30354 consensus_queue.cc:1048] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:29.547612 30036 consensus_queue.cc:260] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.548568 30037 tablet_bootstrap.cc:492] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:29.549033 30037 ts_tablet_manager.cc:1403] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.121s	user 0.026s	sys 0.004s
I20260709 13:10:29.550599 30036 ts_tablet_manager.cc:1434] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.003s
I20260709 13:10:29.544260 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:29.552409 30036 tablet_bootstrap.cc:492] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:29.552830 29839 raft_consensus.cc:2393] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:29.553344 30037 raft_consensus.cc:359] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.554040 30037 raft_consensus.cc:385] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.554297 30037 raft_consensus.cc:740] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.555015 30037 consensus_queue.cc:260] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
W20260709 13:10:29.557152 29713 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:29.543387 29987 raft_consensus.cc:2393] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:29.560549 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:29.562497 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:29.563314 29767 raft_consensus.cc:2393] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 1: Already voted for candidate 2ba28923c4534c78903f759f0a4e1265 in this term.
I20260709 13:10:29.564563 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:29.565071 29915 raft_consensus.cc:3060] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:29.565131 29881 leader_election.cc:343] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.565629 29881 leader_election.cc:304] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:29.556756 29665 catalog_manager.cc:5697] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:29.570161 29732 leader_election.cc:304] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:29.570854 30323 raft_consensus.cc:2749] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:29.571782 30037 ts_tablet_manager.cc:1434] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:10:29.572089 30298 raft_consensus.cc:493] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.572993 30075 raft_consensus.cc:2749] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:29.567852 29665 catalog_manager.cc:5697] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:29.577531 30037 tablet_bootstrap.cc:492] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:29.577703 29807 leader_election.cc:304] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:29.582615 30180 raft_consensus.cc:2804] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:29.572578 30298 raft_consensus.cc:515] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.585363 30036 tablet_bootstrap.cc:654] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.587625 30035 tablet_bootstrap.cc:654] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.592350 30298 leader_election.cc:290] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.583231 30354 consensus_queue.cc:1048] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:10:29.601644 29767 raft_consensus.cc:3060] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.613895 30034 tablet_bootstrap.cc:492] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:29.621197 30034 ts_tablet_manager.cc:1403] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.237s	user 0.008s	sys 0.009s
I20260709 13:10:29.626698 29915 raft_consensus.cc:2468] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:29.626873 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
W20260709 13:10:29.629848 29807 leader_election.cc:343] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.632692 29807 leader_election.cc:304] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:29.627671 29916 raft_consensus.cc:2468] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:29.639120 30034 raft_consensus.cc:359] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.639964 30034 raft_consensus.cc:385] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.633572 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:29.640317 30034 raft_consensus.cc:740] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.641062 30034 consensus_queue.cc:260] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.643149 30034 ts_tablet_manager.cc:1434] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.022s	user 0.001s	sys 0.003s
I20260709 13:10:29.633508 30180 raft_consensus.cc:697] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:29.644241 30180 consensus_queue.cc:237] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.648952 30034 tablet_bootstrap.cc:492] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:29.656127 30034 tablet_bootstrap.cc:654] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.667057 30037 tablet_bootstrap.cc:654] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.668372 30307 consensus_queue.cc:1048] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:29.670382 30307 raft_consensus.cc:493] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.670853 30307 raft_consensus.cc:515] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.674737 30307 leader_election.cc:290] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:29.676697 30382 raft_consensus.cc:2804] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:29.676558 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96641d56c48649d993e0eec66d091ee5" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:29.677093 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96641d56c48649d993e0eec66d091ee5" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:29.679481 29807 leader_election.cc:343] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.688516 30257 raft_consensus.cc:493] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.689123 30383 raft_consensus.cc:493] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.690449 30036 tablet_bootstrap.cc:492] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:29.703990 29807 leader_election.cc:304] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:29.708617 30036 ts_tablet_manager.cc:1403] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.156s	user 0.016s	sys 0.016s
I20260709 13:10:29.708678 29913 raft_consensus.cc:2468] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:29.713779 29766 raft_consensus.cc:1275] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:29.708727 30257 raft_consensus.cc:515] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.716331 30257 leader_election.cc:290] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:29.711798 30383 raft_consensus.cc:515] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.716897 30036 raft_consensus.cc:359] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.717701 30036 raft_consensus.cc:385] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.718011 30036 raft_consensus.cc:740] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.718135 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:29.718907 30383 leader_election.cc:290] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:29.718703 30036 consensus_queue.cc:260] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.721172 30036 ts_tablet_manager.cc:1434] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:10:29.708181 30382 raft_consensus.cc:697] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:29.716194 29955 leader_election.cc:304] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:29.722486 30382 consensus_queue.cc:237] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.722946 30075 consensus_queue.cc:1048] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:29.724960 30349 raft_consensus.cc:2804] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:29.725553 30349 raft_consensus.cc:493] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.725981 30349 raft_consensus.cc:3060] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.726245 30309 raft_consensus.cc:2804] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:29.726682 30309 raft_consensus.cc:493] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.727020 30309 raft_consensus.cc:3060] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:29.723377 29955 leader_election.cc:343] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.733129 30306 consensus_queue.cc:1048] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:29.735073 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:29.739256 30035 tablet_bootstrap.cc:492] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:29.740021 30035 ts_tablet_manager.cc:1403] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.210s	user 0.016s	sys 0.015s
I20260709 13:10:29.744283 29732 leader_election.cc:304] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:29.745246 30035 raft_consensus.cc:359] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.746969 30035 raft_consensus.cc:385] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.747355 30035 raft_consensus.cc:740] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.753047 30035 consensus_queue.cc:260] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.756008 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:29.756928 29914 raft_consensus.cc:2393] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:29.756045 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:29.759888 30035 ts_tablet_manager.cc:1434] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:10:29.763900 30035 tablet_bootstrap.cc:492] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:29.766336 29665 catalog_manager.cc:5697] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:29.769003 29807 leader_election.cc:304] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:29.792198 30036 tablet_bootstrap.cc:492] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:29.778405 30298 raft_consensus.cc:3060] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.794692 30075 raft_consensus.cc:493] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.795493 30075 raft_consensus.cc:515] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.806542 30075 leader_election.cc:290] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.809921 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:29.810619 30309 raft_consensus.cc:515] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.811884 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:29.828200 29915 raft_consensus.cc:2468] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:29.813941 30034 tablet_bootstrap.cc:492] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:29.829865 30034 ts_tablet_manager.cc:1403] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.181s	user 0.021s	sys 0.000s
I20260709 13:10:29.817360 30354 consensus_queue.cc:1048] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:29.819183 30037 tablet_bootstrap.cc:492] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:29.832630 29732 leader_election.cc:304] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:29.833040 30037 ts_tablet_manager.cc:1403] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.260s	user 0.024s	sys 0.014s
I20260709 13:10:29.833388 30309 leader_election.cc:290] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:29.834213 29842 raft_consensus.cc:2393] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:29.835920 30037 raft_consensus.cc:359] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.836655 30037 raft_consensus.cc:385] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.836763 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:29.836942 30037 raft_consensus.cc:740] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.836972 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:29.837361 29915 raft_consensus.cc:3060] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.837745 29764 raft_consensus.cc:3060] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.805236 30295 raft_consensus.cc:2749] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:29.826313 30349 raft_consensus.cc:515] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:29.853760 30037 consensus_queue.cc:260] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.858655 30034 raft_consensus.cc:359] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.859263 30034 raft_consensus.cc:385] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.859579 30034 raft_consensus.cc:740] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.862074 30034 consensus_queue.cc:260] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:29.880216 30035 tablet_bootstrap.cc:654] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.883682 30037 ts_tablet_manager.cc:1434] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 13:10:29.895314 29665 catalog_manager.cc:5697] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:29.903791 29915 raft_consensus.cc:2468] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:29.897003 30034 ts_tablet_manager.cc:1434] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.067s	user 0.018s	sys 0.000s
I20260709 13:10:29.905218 30034 tablet_bootstrap.cc:492] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:29.892023 30298 raft_consensus.cc:2749] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:29.912740 29807 leader_election.cc:304] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:29.927700 29764 raft_consensus.cc:2468] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:29.932073 30309 raft_consensus.cc:2804] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:29.932190 30034 tablet_bootstrap.cc:654] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.932129 30350 consensus_queue.cc:1048] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:29.934079 30350 consensus_queue.cc:1048] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:29.936831 30257 raft_consensus.cc:2764] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:10:29.937813 30350 consensus_queue.cc:1048] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:29.944911 30309 raft_consensus.cc:697] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:29.945761 30036 tablet_bootstrap.cc:654] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:29.953606 30349 leader_election.cc:290] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:29.956584 30309 consensus_queue.cc:237] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:29.945169 30350 consensus_queue.cc:1048] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:10:29.968230 30147 raft_consensus.cc:493] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.968881 30147 raft_consensus.cc:515] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.971019 30147 leader_election.cc:290] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:29.974344 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:29.976727 29881 leader_election.cc:343] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:29.981370 30075 consensus_queue.cc:1048] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:29.988672 30034 tablet_bootstrap.cc:492] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:29.989651 30034 ts_tablet_manager.cc:1403] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.085s	user 0.020s	sys 0.000s
I20260709 13:10:29.989651 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96641d56c48649d993e0eec66d091ee5" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:29.990468 29916 raft_consensus.cc:3060] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:29.993193 30034 raft_consensus.cc:359] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:29.993950 30034 raft_consensus.cc:385] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:29.994218 30034 raft_consensus.cc:740] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:29.995987 29838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:29.998260 30037 tablet_bootstrap.cc:492] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:29.998996 30034 consensus_queue.cc:260] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.006687 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96641d56c48649d993e0eec66d091ee5" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
W20260709 13:10:30.007799 29881 leader_election.cc:343] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.008312 29881 leader_election.cc:304] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:30.009992 30323 raft_consensus.cc:2749] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:30.012993 29955 leader_election.cc:343] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.012892 30350 consensus_queue.cc:1048] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:30.015491 30323 raft_consensus.cc:493] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.016124 30323 raft_consensus.cc:515] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.016695 30034 ts_tablet_manager.cc:1434] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:10:30.017853 30034 tablet_bootstrap.cc:492] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:30.018743 30323 leader_election.cc:290] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:30.021068 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:30.021373 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:30.023954 29881 leader_election.cc:343] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:30.024763 29881 leader_election.cc:343] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.025216 29881 leader_election.cc:304] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:30.026849 30147 raft_consensus.cc:2749] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:30.035009 30037 tablet_bootstrap.cc:654] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.044763 29916 raft_consensus.cc:2468] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:30.087761 30298 raft_consensus.cc:493] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.088382 30298 raft_consensus.cc:515] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.089080 30383 consensus_queue.cc:1048] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:30.090423 30298 leader_election.cc:290] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:30.093182 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523db1e9cca74ffb938f22ae9105dbe1" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:30.096279 30035 tablet_bootstrap.cc:492] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:30.096851 30035 ts_tablet_manager.cc:1403] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.333s	user 0.018s	sys 0.008s
I20260709 13:10:30.096293 29955 leader_election.cc:304] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
W20260709 13:10:30.098479 29807 leader_election.cc:343] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.110304 30034 tablet_bootstrap.cc:654] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.113680 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523db1e9cca74ffb938f22ae9105dbe1" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:30.106235 30354 raft_consensus.cc:2804] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:30.118651 29987 raft_consensus.cc:2468] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:30.133239 29807 leader_election.cc:304] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:30.135238 30382 raft_consensus.cc:2804] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:30.136037 30382 raft_consensus.cc:493] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.136441 30382 raft_consensus.cc:3060] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.154129 30035 raft_consensus.cc:359] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.155158 30035 raft_consensus.cc:385] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.156064 30035 raft_consensus.cc:740] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.156955 30035 consensus_queue.cc:260] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.130223 30354 raft_consensus.cc:697] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:30.162586 30354 consensus_queue.cc:237] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.164674 30037 tablet_bootstrap.cc:492] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:30.165205 30037 ts_tablet_manager.cc:1403] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.167s	user 0.007s	sys 0.018s
I20260709 13:10:30.165418 30036 tablet_bootstrap.cc:492] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:30.165901 30036 ts_tablet_manager.cc:1403] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.397s	user 0.025s	sys 0.027s
I20260709 13:10:30.193389 30036 raft_consensus.cc:359] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.194155 30036 raft_consensus.cc:385] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.194437 30036 raft_consensus.cc:740] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.184566 30309 consensus_queue.cc:1048] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:10:30.203222 30036 consensus_queue.cc:260] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.206144 30034 tablet_bootstrap.cc:492] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:30.215153 30034 ts_tablet_manager.cc:1403] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.198s	user 0.014s	sys 0.009s
I20260709 13:10:30.190174 30382 raft_consensus.cc:515] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.211709 30035 ts_tablet_manager.cc:1434] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.115s	user 0.006s	sys 0.001s
I20260709 13:10:30.218484 29665 catalog_manager.cc:5697] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.226627 30036 ts_tablet_manager.cc:1434] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
I20260709 13:10:30.232661 30034 raft_consensus.cc:359] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
W20260709 13:10:30.233121 29807 consensus_peers.cc:597] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:30.233457 30034 raft_consensus.cc:385] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.233767 30034 raft_consensus.cc:740] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.241128 30036 tablet_bootstrap.cc:492] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:30.243848 30180 consensus_queue.cc:1048] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:30.247364 30035 tablet_bootstrap.cc:492] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:30.258322 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523db1e9cca74ffb938f22ae9105dbe1" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:30.259044 29988 raft_consensus.cc:3060] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.255858 30037 raft_consensus.cc:359] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.261515 30037 raft_consensus.cc:385] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.261804 30037 raft_consensus.cc:740] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.263302 30037 consensus_queue.cc:260] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.264886 30037 ts_tablet_manager.cc:1434] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.001s
I20260709 13:10:30.267086 30036 tablet_bootstrap.cc:654] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.267725 30075 raft_consensus.cc:493] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.268256 30075 raft_consensus.cc:515] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.270179 30075 leader_election.cc:290] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:30.266021 30037 tablet_bootstrap.cc:492] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:30.272831 29912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96641d56c48649d993e0eec66d091ee5" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:30.273871 29912 raft_consensus.cc:2393] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:30.275084 30035 tablet_bootstrap.cc:654] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.275451 30036 tablet_bootstrap.cc:492] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:30.275967 30036 ts_tablet_manager.cc:1403] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.001s
I20260709 13:10:30.272966 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96641d56c48649d993e0eec66d091ee5" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:30.277038 29661 catalog_manager.cc:5697] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.279551 30382 leader_election.cc:290] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:30.281333 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523db1e9cca74ffb938f22ae9105dbe1" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:30.281891 29732 leader_election.cc:304] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:30.280948 30034 consensus_queue.cc:260] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.282852 30036 raft_consensus.cc:359] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
W20260709 13:10:30.283162 29807 leader_election.cc:343] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.283612 29987 raft_consensus.cc:3060] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.283947 30036 raft_consensus.cc:385] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.284269 30036 raft_consensus.cc:740] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.283175 30403 raft_consensus.cc:3060] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.285210 30036 consensus_queue.cc:260] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.287582 30036 ts_tablet_manager.cc:1434] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:10:30.288774 30036 tablet_bootstrap.cc:492] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:30.290210 30037 tablet_bootstrap.cc:654] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.291265 29988 raft_consensus.cc:2468] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:30.295002 30036 tablet_bootstrap.cc:654] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.307605 29807 leader_election.cc:304] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:30.308650 30180 raft_consensus.cc:2804] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:30.309162 30180 raft_consensus.cc:697] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:30.309918 30180 consensus_queue.cc:237] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.317692 30403 raft_consensus.cc:2749] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:30.296080 30034 ts_tablet_manager.cc:1434] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.081s	user 0.005s	sys 0.001s
I20260709 13:10:30.299754 30035 tablet_bootstrap.cc:492] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:30.326771 30035 ts_tablet_manager.cc:1403] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.080s	user 0.014s	sys 0.000s
I20260709 13:10:30.330806 30035 raft_consensus.cc:359] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.331887 30035 raft_consensus.cc:385] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.332300 30035 raft_consensus.cc:740] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.300609 30037 tablet_bootstrap.cc:492] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:30.333401 30037 ts_tablet_manager.cc:1403] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.002s
I20260709 13:10:30.338375 30035 consensus_queue.cc:260] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.341912 30075 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.342801 30036 tablet_bootstrap.cc:492] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:30.342610 30075 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.343367 30036 ts_tablet_manager.cc:1403] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.012s
I20260709 13:10:30.346364 30036 raft_consensus.cc:359] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.353091 30036 raft_consensus.cc:385] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.353415 30036 raft_consensus.cc:740] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.354130 30036 consensus_queue.cc:260] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.363152 30036 ts_tablet_manager.cc:1434] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:10:30.364262 30036 tablet_bootstrap.cc:492] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:30.370860 30036 tablet_bootstrap.cc:654] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.340725 30037 raft_consensus.cc:359] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.380165 30037 raft_consensus.cc:385] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.381309 30037 raft_consensus.cc:740] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.383924 30037 consensus_queue.cc:260] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.385663 30354 raft_consensus.cc:493] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.390842 30354 raft_consensus.cc:515] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.391844 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:30.392832 30354 leader_election.cc:290] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:30.386657 29665 catalog_manager.cc:5697] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:30.393852 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:30.382783 30075 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:30.388577 30036 tablet_bootstrap.cc:492] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:30.397022 30036 ts_tablet_manager.cc:1403] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.007s
I20260709 13:10:30.399727 30036 raft_consensus.cc:359] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.400383 30036 raft_consensus.cc:385] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.400636 30036 raft_consensus.cc:740] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.401238 30036 consensus_queue.cc:260] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.404029 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
W20260709 13:10:30.404599 29955 leader_election.cc:343] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.404785 30037 ts_tablet_manager.cc:1434] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.000s
I20260709 13:10:30.404851 29913 raft_consensus.cc:2468] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:30.405846 30037 tablet_bootstrap.cc:492] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:30.408259 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:30.410324 29955 leader_election.cc:304] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:30.390215 30035 ts_tablet_manager.cc:1434] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 13:10:30.386606 30034 tablet_bootstrap.cc:492] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:30.413127 30349 raft_consensus.cc:2804] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:30.413538 30349 raft_consensus.cc:493] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.413862 30349 raft_consensus.cc:3060] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.414291 30036 ts_tablet_manager.cc:1434] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:10:30.415603 30036 tablet_bootstrap.cc:492] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:30.418195 30034 tablet_bootstrap.cc:654] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.421664 30349 raft_consensus.cc:515] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.429692 30037 tablet_bootstrap.cc:654] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.430682 29732 leader_election.cc:304] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:30.433038 30121 raft_consensus.cc:2749] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:30.433898 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:30.434576 29914 raft_consensus.cc:3060] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.433065 30349 leader_election.cc:290] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:30.436061 30034 tablet_bootstrap.cc:492] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:30.429692 30036 tablet_bootstrap.cc:654] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.436705 30034 ts_tablet_manager.cc:1403] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.050s	user 0.008s	sys 0.012s
I20260709 13:10:30.439797 30034 raft_consensus.cc:359] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.440601 30034 raft_consensus.cc:385] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.440971 30034 raft_consensus.cc:740] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.441068 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:30.441955 30034 consensus_queue.cc:260] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
W20260709 13:10:30.443594 29955 leader_election.cc:343] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.444152 30034 ts_tablet_manager.cc:1434] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:30.445261 30034 tablet_bootstrap.cc:492] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:30.445379 29914 raft_consensus.cc:2468] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:30.447121 29955 leader_election.cc:304] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:30.448948 30350 raft_consensus.cc:2804] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:30.449440 30350 raft_consensus.cc:697] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:30.449986 30035 tablet_bootstrap.cc:492] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:30.450502 30350 consensus_queue.cc:237] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.451992 30034 tablet_bootstrap.cc:654] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.461048 30382 raft_consensus.cc:493] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.461630 30382 raft_consensus.cc:515] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.468268 30382 leader_election.cc:290] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:30.470716 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:30.471478 29987 raft_consensus.cc:2468] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:30.474377 29807 leader_election.cc:304] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:30.475509 30180 consensus_queue.cc:1048] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:30.477069 30309 raft_consensus.cc:2804] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:30.477643 30309 raft_consensus.cc:493] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.478081 30309 raft_consensus.cc:3060] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.471355 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:30.498124 29914 raft_consensus.cc:2468] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:30.511983 30309 raft_consensus.cc:515] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.519156 30035 tablet_bootstrap.cc:654] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.554432 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:30.558308 30417 raft_consensus.cc:493] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.559109 30417 raft_consensus.cc:515] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.563448 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20470bec53744d58b8955d49e2e7ea2a" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:30.564697 30417 leader_election.cc:290] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
W20260709 13:10:30.564991 29881 leader_election.cc:343] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.555168 29913 raft_consensus.cc:3060] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.568377 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20470bec53744d58b8955d49e2e7ea2a" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:30.570219 29881 leader_election.cc:343] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.570804 29881 leader_election.cc:304] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:30.572105 30417 raft_consensus.cc:2749] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:30.600808 30309 leader_election.cc:290] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:30.603192 30323 raft_consensus.cc:493] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.603763 30323 raft_consensus.cc:515] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.605806 30323 leader_election.cc:290] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:30.607672 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:30.612236 29881 leader_election.cc:343] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.617096 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:30.617794 29987 raft_consensus.cc:3060] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.625797 29987 raft_consensus.cc:2468] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:30.625550 30382 consensus_queue.cc:1048] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:30.638617 29807 leader_election.cc:304] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:30.645040 30382 raft_consensus.cc:2804] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:30.645720 30382 raft_consensus.cc:697] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:30.646775 30382 consensus_queue.cc:237] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.613503 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:30.661227 29881 leader_election.cc:343] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.661942 29881 leader_election.cc:304] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:30.663414 30417 raft_consensus.cc:2749] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:30.660962 29913 raft_consensus.cc:2468] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:30.666030 30417 raft_consensus.cc:493] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.666608 30417 raft_consensus.cc:515] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.670892 30417 leader_election.cc:290] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:30.673686 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f721871c07c4c57b9c2a1185604e336" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:30.683215 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f721871c07c4c57b9c2a1185604e336" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:30.752990 30035 tablet_bootstrap.cc:492] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:30.753623 30035 ts_tablet_manager.cc:1403] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.304s	user 0.013s	sys 0.000s
I20260709 13:10:30.687225 30354 consensus_queue.cc:1048] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:30.755645 29881 leader_election.cc:343] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:30.757005 29713 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:30.688347 30382 raft_consensus.cc:493] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.757859 30035 raft_consensus.cc:359] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.758772 30035 raft_consensus.cc:385] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.759372 30035 raft_consensus.cc:740] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.758738 30382 raft_consensus.cc:515] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.743692 30309 consensus_queue.cc:1048] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:30.760226 29839 raft_consensus.cc:2468] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 0.
I20260709 13:10:30.765761 30398 consensus_queue.cc:1048] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:30.766999 29881 leader_election.cc:304] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:30.768409 30323 raft_consensus.cc:2804] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:30.768797 30323 raft_consensus.cc:493] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.769165 30323 raft_consensus.cc:3060] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.760102 30035 consensus_queue.cc:260] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.771631 30035 ts_tablet_manager.cc:1434] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:10:30.771803 30382 leader_election.cc:290] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:30.772459 29912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f721871c07c4c57b9c2a1185604e336" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:30.775822 30036 tablet_bootstrap.cc:492] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:30.776001 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f721871c07c4c57b9c2a1185604e336" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:30.776510 30036 ts_tablet_manager.cc:1403] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.361s	user 0.061s	sys 0.084s
W20260709 13:10:30.777499 29807 leader_election.cc:343] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.779421 30036 raft_consensus.cc:359] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.780172 30036 raft_consensus.cc:385] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.780239 30147 raft_consensus.cc:493] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.780483 30036 raft_consensus.cc:740] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.780818 30147 raft_consensus.cc:515] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.781247 30036 consensus_queue.cc:260] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.760237 30037 tablet_bootstrap.cc:492] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:30.772773 30035 tablet_bootstrap.cc:492] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:30.783233 30147 leader_election.cc:290] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:30.784994 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:30.784823 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:30.789000 29663 catalog_manager.cc:5697] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.802305 30037 ts_tablet_manager.cc:1403] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.397s	user 0.047s	sys 0.103s
I20260709 13:10:30.803809 30034 tablet_bootstrap.cc:492] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:30.804351 30034 ts_tablet_manager.cc:1403] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.359s	user 0.115s	sys 0.141s
I20260709 13:10:30.805147 30037 raft_consensus.cc:359] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.805866 30037 raft_consensus.cc:385] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.806130 30037 raft_consensus.cc:740] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
W20260709 13:10:30.807848 29881 leader_election.cc:343] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:30.808815 29881 leader_election.cc:343] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.809258 29881 leader_election.cc:304] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:30.812120 30147 raft_consensus.cc:2749] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:30.813202 29765 raft_consensus.cc:3060] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.784682 30323 raft_consensus.cc:515] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.814340 29912 raft_consensus.cc:2393] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 1: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:30.816227 29807 leader_election.cc:304] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:30.817548 30309 raft_consensus.cc:3060] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.820180 30323 leader_election.cc:290] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:30.823177 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f721871c07c4c57b9c2a1185604e336" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
W20260709 13:10:30.824479 29881 leader_election.cc:343] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.826007 30309 raft_consensus.cc:2749] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:30.827512 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f721871c07c4c57b9c2a1185604e336" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
W20260709 13:10:30.829497 29807 consensus_peers.cc:597] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:30.807017 30034 raft_consensus.cc:359] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.839217 30034 raft_consensus.cc:385] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.839537 30034 raft_consensus.cc:740] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.840415 30034 consensus_queue.cc:260] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.842610 30034 ts_tablet_manager.cc:1434] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:10:30.843758 30034 tablet_bootstrap.cc:492] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:30.838300 30035 tablet_bootstrap.cc:654] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.806751 30037 consensus_queue.cc:260] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.851752 30037 ts_tablet_manager.cc:1434] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 13:10:30.854501 30037 tablet_bootstrap.cc:492] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:30.858009 29767 raft_consensus.cc:3060] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:30.869501 29661 catalog_manager.cc:5697] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.876376 30037 tablet_bootstrap.cc:654] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.877754 30398 consensus_queue.cc:1048] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:30.877825 30034 tablet_bootstrap.cc:654] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.897714 30036 ts_tablet_manager.cc:1434] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.121s	user 0.004s	sys 0.023s
I20260709 13:10:30.899513 30036 tablet_bootstrap.cc:492] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:30.913682 30034 tablet_bootstrap.cc:492] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:30.914347 30034 ts_tablet_manager.cc:1403] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.003s
I20260709 13:10:30.914338 29839 raft_consensus.cc:2468] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:30.919232 30034 raft_consensus.cc:359] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.920076 30034 raft_consensus.cc:385] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.920397 30034 raft_consensus.cc:740] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.921092 30034 consensus_queue.cc:260] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:30.922853 30034 ts_tablet_manager.cc:1434] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:10:30.926877 30034 tablet_bootstrap.cc:492] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:30.928607 30349 consensus_queue.cc:1048] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:30.928988 30037 tablet_bootstrap.cc:492] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:30.929687 30037 ts_tablet_manager.cc:1403] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.001s
I20260709 13:10:30.932996 30037 raft_consensus.cc:359] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.933880 30037 raft_consensus.cc:385] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.934175 30037 raft_consensus.cc:740] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.934947 30037 consensus_queue.cc:260] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.935672 30034 tablet_bootstrap.cc:654] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.937572 30037 ts_tablet_manager.cc:1434] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:30.945286 30037 tablet_bootstrap.cc:492] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:30.952430 30037 tablet_bootstrap.cc:654] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.940289 30036 tablet_bootstrap.cc:654] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.955363 30349 raft_consensus.cc:493] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.956036 30349 raft_consensus.cc:515] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.956692 29881 leader_election.cc:304] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:30.956923 30034 tablet_bootstrap.cc:492] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:30.957803 30034 ts_tablet_manager.cc:1403] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.004s
I20260709 13:10:30.958205 30349 leader_election.cc:290] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:30.960757 30034 raft_consensus.cc:359] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.961577 30034 raft_consensus.cc:385] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.961915 30034 raft_consensus.cc:740] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.962611 30034 consensus_queue.cc:260] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.963075 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a6337639944433881ce49be8f337030" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:30.964437 30034 ts_tablet_manager.cc:1434] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:30.964540 30441 raft_consensus.cc:2804] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:30.965097 30441 raft_consensus.cc:697] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:30.965632 30034 tablet_bootstrap.cc:492] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:30.965699 30035 tablet_bootstrap.cc:492] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:30.967186 30443 raft_consensus.cc:493] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.967738 30443 raft_consensus.cc:515] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:30.965965 30441 consensus_queue.cc:237] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:30.969983 30443 leader_election.cc:290] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:30.967227 30035 ts_tablet_manager.cc:1403] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.195s	user 0.028s	sys 0.003s
I20260709 13:10:30.972885 30034 tablet_bootstrap.cc:654] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:30.975207 30035 raft_consensus.cc:359] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.975997 30035 raft_consensus.cc:385] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.975823 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:30.976449 30035 raft_consensus.cc:740] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.977252 30035 consensus_queue.cc:260] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.978235 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:30.985074 29881 leader_election.cc:343] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:30.985997 29881 leader_election.cc:343] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.986490 29881 leader_election.cc:304] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:30.987432 30443 raft_consensus.cc:2749] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:30.990392 30034 tablet_bootstrap.cc:492] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:30.990999 30034 ts_tablet_manager.cc:1403] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.005s
I20260709 13:10:30.993887 30034 raft_consensus.cc:359] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.995647 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a6337639944433881ce49be8f337030" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:30.996376 29839 raft_consensus.cc:2468] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:30.994719 30034 raft_consensus.cc:385] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:30.997179 30034 raft_consensus.cc:740] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:30.997968 30034 consensus_queue.cc:260] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
W20260709 13:10:30.998256 29955 leader_election.cc:343] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:30.999773 29955 leader_election.cc:304] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:30.999908 30349 raft_consensus.cc:493] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.020661 30349 raft_consensus.cc:515] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.019774 30406 raft_consensus.cc:2804] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.023418 30350 raft_consensus.cc:493] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.024115 30350 raft_consensus.cc:515] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.024961 30349 leader_election.cc:290] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:31.023515 30406 raft_consensus.cc:493] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.026556 30406 raft_consensus.cc:3060] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.027062 30037 tablet_bootstrap.cc:492] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:31.027631 30037 ts_tablet_manager.cc:1403] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.083s	user 0.028s	sys 0.013s
I20260709 13:10:31.030340 30037 raft_consensus.cc:359] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.031060 30037 raft_consensus.cc:385] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.031411 30037 raft_consensus.cc:740] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.026113 30350 leader_election.cc:290] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:31.032353 30037 consensus_queue.cc:260] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:30.999922 30034 ts_tablet_manager.cc:1434] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:10:31.034713 30037 ts_tablet_manager.cc:1434] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:10:31.002281 30036 tablet_bootstrap.cc:492] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:31.035845 30034 tablet_bootstrap.cc:492] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:31.035924 30037 tablet_bootstrap.cc:492] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:31.036154 30406 raft_consensus.cc:515] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.035871 30036 ts_tablet_manager.cc:1403] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.137s	user 0.024s	sys 0.011s
I20260709 13:10:31.038295 30380 consensus_queue.cc:1048] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:31.043272 30406 leader_election.cc:290] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:31.045009 30036 raft_consensus.cc:359] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.045943 30036 raft_consensus.cc:385] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.046273 30036 raft_consensus.cc:740] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.047135 30036 consensus_queue.cc:260] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.050185 30035 ts_tablet_manager.cc:1434] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.078s	user 0.001s	sys 0.005s
I20260709 13:10:31.053016 30036 ts_tablet_manager.cc:1434] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:10:31.054210 30036 tablet_bootstrap.cc:492] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:31.053135 30037 tablet_bootstrap.cc:654] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.057868 29661 catalog_manager.cc:5697] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:31.061376 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:31.061539 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:31.062975 30036 tablet_bootstrap.cc:654] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.062894 29913 raft_consensus.cc:2468] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:31.062602 29839 raft_consensus.cc:2468] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:31.066131 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e5a681df5b42e98dac6b49a96773fe" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:31.066248 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e5a681df5b42e98dac6b49a96773fe" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:31.067027 29842 raft_consensus.cc:2468] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:31.069545 30034 tablet_bootstrap.cc:654] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.077541 30035 tablet_bootstrap.cc:492] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:31.091892 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a6337639944433881ce49be8f337030" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:31.097390 30298 consensus_queue.cc:1048] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:31.101064 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a6337639944433881ce49be8f337030" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:31.101943 29842 raft_consensus.cc:3060] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.114655 29842 raft_consensus.cc:2468] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:31.129741 30034 tablet_bootstrap.cc:492] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:31.130520 30034 ts_tablet_manager.cc:1403] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.095s	user 0.022s	sys 0.014s
I20260709 13:10:31.143422 30037 tablet_bootstrap.cc:492] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:31.144115 30037 ts_tablet_manager.cc:1403] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.108s	user 0.024s	sys 0.014s
I20260709 13:10:31.147456 30037 raft_consensus.cc:359] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.148483 30037 raft_consensus.cc:385] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.148789 30037 raft_consensus.cc:740] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.149621 30037 consensus_queue.cc:260] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.150460 30035 tablet_bootstrap.cc:654] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.157461 30037 ts_tablet_manager.cc:1434] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:10:31.158689 30037 tablet_bootstrap.cc:492] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:31.170507 30037 tablet_bootstrap.cc:654] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.174082 30034 raft_consensus.cc:359] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.174851 30034 raft_consensus.cc:385] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.174964 29955 leader_election.cc:304] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:31.176204 30406 raft_consensus.cc:493] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.176774 30455 raft_consensus.cc:2804] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.177386 30455 raft_consensus.cc:493] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.176838 30406 raft_consensus.cc:515] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.177742 30455 raft_consensus.cc:3060] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.180168 30406 leader_election.cc:290] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:31.181200 29955 leader_election.cc:304] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:31.175154 30034 raft_consensus.cc:740] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.182300 30354 raft_consensus.cc:2804] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.182838 30354 raft_consensus.cc:493] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.182683 30034 consensus_queue.cc:260] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.183449 30354 raft_consensus.cc:3060] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.185086 30034 ts_tablet_manager.cc:1434] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
I20260709 13:10:31.186152 30034 tablet_bootstrap.cc:492] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:31.187772 29955 leader_election.cc:304] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:31.192480 30455 raft_consensus.cc:515] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.195859 30035 tablet_bootstrap.cc:492] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:31.206955 30035 ts_tablet_manager.cc:1403] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.156s	user 0.014s	sys 0.002s
I20260709 13:10:31.211030 30035 raft_consensus.cc:359] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.211977 30035 raft_consensus.cc:385] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.212252 30035 raft_consensus.cc:740] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.212836 30035 consensus_queue.cc:260] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.215736 30035 ts_tablet_manager.cc:1434] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:31.216810 30455 leader_election.cc:290] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:31.218053 30034 tablet_bootstrap.cc:654] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.233575 30441 raft_consensus.cc:493] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.235162 30406 raft_consensus.cc:2804] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.250633 30035 tablet_bootstrap.cc:492] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:31.252528 30034 tablet_bootstrap.cc:492] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:31.253274 30034 ts_tablet_manager.cc:1403] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.067s	user 0.012s	sys 0.002s
I20260709 13:10:31.271457 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:31.272558 29839 raft_consensus.cc:3060] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.276376 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:31.277186 29914 raft_consensus.cc:2468] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:31.278391 29764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:31.252342 30441 raft_consensus.cc:515] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.288043 30441 leader_election.cc:290] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:31.289578 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1bb074fa4f46b48be16325eabd6d7d" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:31.290716 30037 tablet_bootstrap.cc:492] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:31.291247 30037 ts_tablet_manager.cc:1403] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.029s
W20260709 13:10:31.295616 29881 leader_election.cc:343] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.300081 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1bb074fa4f46b48be16325eabd6d7d" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:31.301872 29766 raft_consensus.cc:3060] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.295177 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
W20260709 13:10:31.320854 29955 leader_election.cc:343] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:31.326927 29955 leader_election.cc:343] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:31.330096 29955 leader_election.cc:343] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.332470 30036 tablet_bootstrap.cc:492] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:31.333031 30036 ts_tablet_manager.cc:1403] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.279s	user 0.053s	sys 0.117s
I20260709 13:10:31.334362 29955 leader_election.cc:304] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.335525 30349 raft_consensus.cc:2804] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.335914 30349 raft_consensus.cc:493] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.336225 30349 raft_consensus.cc:3060] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.317770 29913 raft_consensus.cc:3060] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.345067 30034 raft_consensus.cc:359] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.345855 30034 raft_consensus.cc:385] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.346131 30034 raft_consensus.cc:740] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.352281 29839 raft_consensus.cc:2468] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
W20260709 13:10:31.356914 29881 leader_election.cc:343] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.357458 29881 leader_election.cc:304] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:31.360273 30036 raft_consensus.cc:359] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.361007 30036 raft_consensus.cc:385] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.361263 30036 raft_consensus.cc:740] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.362181 30455 raft_consensus.cc:493] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.362747 30455 raft_consensus.cc:515] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.364959 30455 leader_election.cc:290] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:31.363202 30036 consensus_queue.cc:260] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.367530 30443 raft_consensus.cc:2749] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:31.368422 30036 ts_tablet_manager.cc:1434] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:10:31.369496 30036 tablet_bootstrap.cc:492] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06: Bootstrap starting.
I20260709 13:10:31.375010 30035 tablet_bootstrap.cc:654] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.402307 29955 leader_election.cc:304] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:31.380564 29764 raft_consensus.cc:3060] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.383714 30037 raft_consensus.cc:359] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.391546 30349 raft_consensus.cc:515] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.408411 30349 leader_election.cc:290] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:31.409003 30349 raft_consensus.cc:2804] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.409493 30349 raft_consensus.cc:697] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:31.410355 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:31.410292 30349 consensus_queue.cc:237] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.346796 30034 consensus_queue.cc:260] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.376590 30354 raft_consensus.cc:515] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.416846 30354 leader_election.cc:290] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:31.417410 30037 raft_consensus.cc:385] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.417712 30037 raft_consensus.cc:740] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.418365 30037 consensus_queue.cc:260] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.386948 29913 raft_consensus.cc:2468] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:31.405498 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:31.419603 29914 raft_consensus.cc:2468] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:31.424268 30034 ts_tablet_manager.cc:1434] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.170s	user 0.006s	sys 0.000s
I20260709 13:10:31.433749 30035 tablet_bootstrap.cc:492] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:31.434295 30034 tablet_bootstrap.cc:492] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:31.436467 30036 tablet_bootstrap.cc:654] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.434300 30035 ts_tablet_manager.cc:1403] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.186s	user 0.013s	sys 0.000s
I20260709 13:10:31.440927 30034 tablet_bootstrap.cc:654] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.441602 30035 raft_consensus.cc:359] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.442380 30035 raft_consensus.cc:385] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.442648 30035 raft_consensus.cc:740] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.425504 30037 ts_tablet_manager.cc:1434] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.134s	user 0.004s	sys 0.002s
W20260709 13:10:31.426411 29955 leader_election.cc:343] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.427335 30406 raft_consensus.cc:697] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:31.427804 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:31.427932 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:31.449404 29913 raft_consensus.cc:3060] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.449997 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e5a681df5b42e98dac6b49a96773fe" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:31.450037 29955 leader_election.cc:304] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.452422 30470 raft_consensus.cc:2804] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.452772 30470 raft_consensus.cc:493] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.431371 30443 consensus_queue.cc:1048] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:31.453266 29955 leader_election.cc:343] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:31.453961 29955 leader_election.cc:343] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.454847 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e5a681df5b42e98dac6b49a96773fe" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:31.455675 29839 raft_consensus.cc:3060] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.461089 30034 tablet_bootstrap.cc:492] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:31.461877 30034 ts_tablet_manager.cc:1403] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.000s
I20260709 13:10:31.453076 30470 raft_consensus.cc:3060] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.464859 30034 raft_consensus.cc:359] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.465880 30034 raft_consensus.cc:385] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.466226 30034 raft_consensus.cc:740] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.466460 29839 raft_consensus.cc:2468] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:31.467209 30034 consensus_queue.cc:260] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.467963 29955 leader_election.cc:304] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.468752 30354 raft_consensus.cc:2804] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.469205 30354 raft_consensus.cc:697] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:31.469997 30354 consensus_queue.cc:237] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.472350 30406 consensus_queue.cc:237] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.474906 30034 ts_tablet_manager.cc:1434] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:10:31.492982 30035 consensus_queue.cc:260] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.500152 30034 tablet_bootstrap.cc:492] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3: Bootstrap starting.
I20260709 13:10:31.503607 30380 raft_consensus.cc:493] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.504410 30380 raft_consensus.cc:515] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.516654 30034 tablet_bootstrap.cc:654] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.507223 30380 leader_election.cc:290] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:31.472342 30037 tablet_bootstrap.cc:492] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:31.536116 29913 raft_consensus.cc:2468] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:31.541011 29955 leader_election.cc:304] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.541916 30467 raft_consensus.cc:2804] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.542526 30467 raft_consensus.cc:697] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:31.544063 30467 consensus_queue.cc:237] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.548784 30035 ts_tablet_manager.cc:1434] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.110s	user 0.003s	sys 0.004s
I20260709 13:10:31.551060 29766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f49c45a4a5479d8609fe95f8486b97" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:31.551740 30441 consensus_queue.cc:1048] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:31.552841 29807 leader_election.cc:343] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.554045 30036 tablet_bootstrap.cc:492] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06: No bootstrap required, opened a new log
I20260709 13:10:31.554553 30036 ts_tablet_manager.cc:1403] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06: Time spent bootstrapping tablet: real 0.185s	user 0.037s	sys 0.060s
I20260709 13:10:31.559078 30034 tablet_bootstrap.cc:492] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3: No bootstrap required, opened a new log
I20260709 13:10:31.560119 30034 ts_tablet_manager.cc:1403] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3: Time spent bootstrapping tablet: real 0.083s	user 0.026s	sys 0.013s
I20260709 13:10:31.558609 30036 raft_consensus.cc:359] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.561754 30036 raft_consensus.cc:385] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.562060 30036 raft_consensus.cc:740] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.562821 30036 consensus_queue.cc:260] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.564961 30036 ts_tablet_manager.cc:1434] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:10:31.564330 30034 raft_consensus.cc:359] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.573968 30034 raft_consensus.cc:385] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.574291 30457 raft_consensus.cc:493] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.567592 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f49c45a4a5479d8609fe95f8486b97" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:31.574981 30457 raft_consensus.cc:515] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.575939 29916 raft_consensus.cc:2468] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:31.576987 29807 leader_election.cc:304] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.577553 30457 leader_election.cc:290] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:31.577833 30380 raft_consensus.cc:2804] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.578344 30380 raft_consensus.cc:493] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.578661 30380 raft_consensus.cc:3060] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.579815 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a6337639944433881ce49be8f337030" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:31.580628 29841 raft_consensus.cc:2393] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:31.572108 30035 tablet_bootstrap.cc:492] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:31.583654 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a6337639944433881ce49be8f337030" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:31.588455 29732 leader_election.cc:304] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:31.588753 30037 tablet_bootstrap.cc:654] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.590018 30456 raft_consensus.cc:3060] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.590662 30470 raft_consensus.cc:515] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.592834 30470 leader_election.cc:290] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:31.594722 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
W20260709 13:10:31.596177 29955 leader_election.cc:343] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.574311 30034 raft_consensus.cc:740] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.605546 30035 tablet_bootstrap.cc:654] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.605950 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:31.606611 29916 raft_consensus.cc:3060] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.606871 29664 catalog_manager.cc:5697] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:31.613783 30034 consensus_queue.cc:260] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.615653 30380 raft_consensus.cc:515] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.617491 29916 raft_consensus.cc:2468] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:31.618330 30380 leader_election.cc:290] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:31.613236 30456 raft_consensus.cc:2749] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:31.619504 29955 leader_election.cc:304] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.619689 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f49c45a4a5479d8609fe95f8486b97" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:31.620627 30470 raft_consensus.cc:2804] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:10:31.621351 29807 leader_election.cc:343] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.621124 30470 raft_consensus.cc:697] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:31.622321 30470 consensus_queue.cc:237] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.621495 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f49c45a4a5479d8609fe95f8486b97" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:31.629455 29916 raft_consensus.cc:3060] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.630415 30034 ts_tablet_manager.cc:1434] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.002s
I20260709 13:10:31.632783 30035 tablet_bootstrap.cc:492] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:31.638366 30035 ts_tablet_manager.cc:1403] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.002s
I20260709 13:10:31.641146 30035 raft_consensus.cc:359] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.641835 30035 raft_consensus.cc:385] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.642107 30035 raft_consensus.cc:740] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.633523 30037 tablet_bootstrap.cc:492] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:31.643031 30037 ts_tablet_manager.cc:1403] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.171s	user 0.014s	sys 0.001s
I20260709 13:10:31.645349 29916 raft_consensus.cc:2468] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:31.646283 30037 raft_consensus.cc:359] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.647037 30037 raft_consensus.cc:385] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.647092 29807 leader_election.cc:304] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.648075 30382 raft_consensus.cc:2804] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.647919 30035 consensus_queue.cc:260] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.647435 30037 raft_consensus.cc:740] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.649542 30037 consensus_queue.cc:260] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.650463 30035 ts_tablet_manager.cc:1434] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:10:31.650941 30382 raft_consensus.cc:697] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:31.651520 30035 tablet_bootstrap.cc:492] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:31.651966 30037 ts_tablet_manager.cc:1434] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:10:31.651906 30382 consensus_queue.cc:237] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.653242 30037 tablet_bootstrap.cc:492] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb: Bootstrap starting.
I20260709 13:10:31.658412 30035 tablet_bootstrap.cc:654] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.661190 30037 tablet_bootstrap.cc:654] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.670845 30035 tablet_bootstrap.cc:492] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:31.671442 30035 ts_tablet_manager.cc:1403] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.014s
I20260709 13:10:31.674186 30035 raft_consensus.cc:359] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.674863 30035 raft_consensus.cc:385] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.675662 30035 raft_consensus.cc:740] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.676437 30035 consensus_queue.cc:260] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.678793 30035 ts_tablet_manager.cc:1434] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:10:31.679893 30035 tablet_bootstrap.cc:492] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:31.690608 30037 tablet_bootstrap.cc:492] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb: No bootstrap required, opened a new log
I20260709 13:10:31.691425 30037 ts_tablet_manager.cc:1403] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.001s
I20260709 13:10:31.694247 30035 tablet_bootstrap.cc:654] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.694459 30037 raft_consensus.cc:359] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.695647 30037 raft_consensus.cc:385] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.701011 30037 raft_consensus.cc:740] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.701918 30037 consensus_queue.cc:260] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.704030 30037 ts_tablet_manager.cc:1434] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:10:31.709333 30035 tablet_bootstrap.cc:492] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:31.709931 30035 ts_tablet_manager.cc:1403] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.004s
I20260709 13:10:31.712901 30035 raft_consensus.cc:359] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.715493 30035 raft_consensus.cc:385] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.715798 30035 raft_consensus.cc:740] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.716414 30035 consensus_queue.cc:260] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.718622 30035 ts_tablet_manager.cc:1434] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.000s
I20260709 13:10:31.718472 29664 catalog_manager.cc:5697] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:31.719683 30035 tablet_bootstrap.cc:492] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:31.725242 30457 raft_consensus.cc:493] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.725811 30457 raft_consensus.cc:515] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.731642 30035 tablet_bootstrap.cc:654] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.732951 30457 leader_election.cc:290] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:31.734196 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:31.735045 29916 raft_consensus.cc:2393] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:31.736112 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:31.737783 29732 leader_election.cc:304] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:31.738622 30457 raft_consensus.cc:3060] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.740221 30035 tablet_bootstrap.cc:492] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:31.740799 30035 ts_tablet_manager.cc:1403] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.000s
I20260709 13:10:31.746443 30035 raft_consensus.cc:359] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.747697 30035 raft_consensus.cc:385] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.748034 30035 raft_consensus.cc:740] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.748706 30035 consensus_queue.cc:260] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.750407 30457 raft_consensus.cc:2749] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:31.750571 30035 ts_tablet_manager.cc:1434] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:10:31.752058 30349 raft_consensus.cc:493] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.741727 29665 catalog_manager.cc:5697] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:31.752693 30035 tablet_bootstrap.cc:492] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:31.752575 30349 raft_consensus.cc:515] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.754750 30349 leader_election.cc:290] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:31.753672 29665 catalog_manager.cc:5697] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:31.755903 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1bb074fa4f46b48be16325eabd6d7d" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:31.756780 29839 raft_consensus.cc:2468] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:31.757103 29665 catalog_manager.cc:5697] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:31.758508 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1bb074fa4f46b48be16325eabd6d7d" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:31.759399 29916 raft_consensus.cc:2468] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:31.760637 29955 leader_election.cc:304] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:31.759979 29665 catalog_manager.cc:5697] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:31.761988 30035 tablet_bootstrap.cc:654] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.764688 30349 raft_consensus.cc:2804] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.765141 30349 raft_consensus.cc:493] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.765461 30349 raft_consensus.cc:3060] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.774588 30456 raft_consensus.cc:493] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.775470 30456 raft_consensus.cc:515] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.778288 30456 leader_election.cc:290] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:31.781073 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:31.781869 29839 raft_consensus.cc:2468] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:31.781713 30349 raft_consensus.cc:515] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.781105 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:31.783043 29732 leader_election.cc:304] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:31.783267 29916 raft_consensus.cc:2468] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:31.787360 30075 raft_consensus.cc:2804] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.787742 30075 raft_consensus.cc:493] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.788012 30075 raft_consensus.cc:3060] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.792152 30470 raft_consensus.cc:493] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.794317 30470 raft_consensus.cc:515] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.796940 30470 leader_election.cc:290] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:31.798195 30075 raft_consensus.cc:515] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.799182 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20470bec53744d58b8955d49e2e7ea2a" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:31.799997 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20470bec53744d58b8955d49e2e7ea2a" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:31.800375 29914 raft_consensus.cc:2468] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 0.
I20260709 13:10:31.800985 30075 leader_election.cc:290] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:31.801397 29955 leader_election.cc:343] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.802026 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:31.802204 29955 leader_election.cc:304] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.804020 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:31.804629 29914 raft_consensus.cc:3060] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.805154 30470 raft_consensus.cc:2804] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.806512 30349 leader_election.cc:290] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:31.807744 30470 raft_consensus.cc:493] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.808274 30470 raft_consensus.cc:3060] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.811087 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1bb074fa4f46b48be16325eabd6d7d" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:31.811818 29842 raft_consensus.cc:3060] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.804039 29839 raft_consensus.cc:3060] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.816023 30470 raft_consensus.cc:515] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.818332 30470 leader_election.cc:290] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:31.820411 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1bb074fa4f46b48be16325eabd6d7d" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:31.821175 29913 raft_consensus.cc:3060] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.826575 29914 raft_consensus.cc:2468] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:31.833012 29913 raft_consensus.cc:2468] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:31.834923 29912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20470bec53744d58b8955d49e2e7ea2a" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:31.835675 29912 raft_consensus.cc:3060] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.835669 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20470bec53744d58b8955d49e2e7ea2a" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:31.836570 29842 raft_consensus.cc:2468] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:31.844074 29912 raft_consensus.cc:2468] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
W20260709 13:10:31.844468 29955 leader_election.cc:343] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.848486 29839 raft_consensus.cc:2468] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 1.
I20260709 13:10:31.848196 29955 leader_election.cc:304] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
W20260709 13:10:31.835121 29713 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:31.850575 30406 raft_consensus.cc:2804] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.850927 29955 leader_election.cc:304] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.851146 30406 raft_consensus.cc:697] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:31.851951 30349 raft_consensus.cc:2804] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.852476 30349 raft_consensus.cc:697] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:31.852043 30406 consensus_queue.cc:237] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:31.853309 30349 consensus_queue.cc:237] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.854101 29732 leader_election.cc:304] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:31.855609 30457 raft_consensus.cc:2804] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.856107 30457 raft_consensus.cc:697] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:31.856953 30457 consensus_queue.cc:237] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.880216 29664 catalog_manager.cc:5697] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:31.883030 30035 tablet_bootstrap.cc:492] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:31.884548 30035 ts_tablet_manager.cc:1403] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.132s	user 0.028s	sys 0.008s
I20260709 13:10:31.887835 30035 raft_consensus.cc:359] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.888613 30035 raft_consensus.cc:385] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.888909 30035 raft_consensus.cc:740] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.889967 30035 consensus_queue.cc:260] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.891782 30035 ts_tablet_manager.cc:1434] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:31.908118 30035 tablet_bootstrap.cc:492] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:31.910233 29664 catalog_manager.cc:5697] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:31.911072 30406 consensus_queue.cc:1048] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:31.912925 29664 catalog_manager.cc:5697] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 1 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:31.916432 30035 tablet_bootstrap.cc:654] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.922732 30075 raft_consensus.cc:493] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.923313 30075 raft_consensus.cc:515] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:31.925154 30075 leader_election.cc:290] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:31.927657 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20470bec53744d58b8955d49e2e7ea2a" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:31.928798 29914 raft_consensus.cc:2393] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:31.931576 30382 raft_consensus.cc:493] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.932222 30382 raft_consensus.cc:515] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.933590 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20470bec53744d58b8955d49e2e7ea2a" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:31.934659 30382 leader_election.cc:290] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:31.935606 29732 leader_election.cc:304] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:31.936587 30075 raft_consensus.cc:3060] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.937160 29767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:31.933645 30035 tablet_bootstrap.cc:492] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
W20260709 13:10:31.938812 29807 leader_election.cc:343] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:31.938869 30035 ts_tablet_manager.cc:1403] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.006s
I20260709 13:10:31.943634 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:31.944397 29987 raft_consensus.cc:2468] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 0.
I20260709 13:10:31.946337 30035 raft_consensus.cc:359] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.947499 30035 raft_consensus.cc:385] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.948200 30075 raft_consensus.cc:2749] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:31.949563 29807 leader_election.cc:304] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:31.950736 30382 raft_consensus.cc:2804] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.951124 30382 raft_consensus.cc:493] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.951517 30382 raft_consensus.cc:3060] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.954233 30468 consensus_queue.cc:1048] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:31.947818 30035 raft_consensus.cc:740] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.960168 30035 consensus_queue.cc:260] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.963136 30035 ts_tablet_manager.cc:1434] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 13:10:31.965042 30035 tablet_bootstrap.cc:492] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:31.966187 30457 raft_consensus.cc:493] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.966722 30457 raft_consensus.cc:515] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.970265 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:31.970335 30349 consensus_queue.cc:1048] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:31.971038 29987 raft_consensus.cc:2468] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 0.
I20260709 13:10:31.972797 29732 leader_election.cc:304] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:31.974359 30457 leader_election.cc:290] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:31.975515 30456 raft_consensus.cc:2804] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:31.975963 30456 raft_consensus.cc:493] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.976312 30456 raft_consensus.cc:3060] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.976575 30382 raft_consensus.cc:515] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.979632 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:31.980093 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:31.980305 29987 raft_consensus.cc:3060] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.981103 29839 raft_consensus.cc:2393] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:31.974660 30035 tablet_bootstrap.cc:654] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.989111 30456 raft_consensus.cc:515] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.990053 30075 raft_consensus.cc:493] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.989606 29987 raft_consensus.cc:2468] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:31.990535 30075 raft_consensus.cc:515] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:31.992429 30075 leader_election.cc:290] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:31.992372 29807 leader_election.cc:304] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:31.994009 30298 raft_consensus.cc:2804] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.994328 30456 leader_election.cc:290] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:31.994867 30349 consensus_queue.cc:1048] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:31.995364 30298 raft_consensus.cc:697] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:31.996573 30298 consensus_queue.cc:237] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:31.999248 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:32.000066 29987 raft_consensus.cc:2393] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:32.000460 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f49c45a4a5479d8609fe95f8486b97" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:32.001890 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:32.003108 30382 leader_election.cc:290] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 1 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:32.006692 29765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:32.007607 29765 raft_consensus.cc:2393] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 1: Already voted for candidate 2ba28923c4534c78903f759f0a4e1265 in this term.
I20260709 13:10:32.011030 29732 leader_election.cc:304] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:32.011461 30349 consensus_queue.cc:1048] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.011883 30075 raft_consensus.cc:2749] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:32.007016 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f49c45a4a5479d8609fe95f8486b97" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:32.019586 29914 raft_consensus.cc:2393] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:32.028820 29732 leader_election.cc:304] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:32.038861 30075 raft_consensus.cc:3060] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:32.039737 30468 consensus_queue.cc:1048] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.040558 29664 catalog_manager.cc:5697] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 0 to 1, leader changed from <none> to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 1 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:32.053541 30075 raft_consensus.cc:2749] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:32.054981 30456 raft_consensus.cc:493] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.056286 30456 raft_consensus.cc:515] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:32.058508 30456 leader_election.cc:290] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:32.060565 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e5a681df5b42e98dac6b49a96773fe" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:32.061225 30035 tablet_bootstrap.cc:492] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: No bootstrap required, opened a new log
I20260709 13:10:32.061466 29841 raft_consensus.cc:2393] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 1: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:32.061717 30035 ts_tablet_manager.cc:1403] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.097s	user 0.022s	sys 0.006s
I20260709 13:10:32.063511 30468 consensus_queue.cc:1048] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.071753 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e5a681df5b42e98dac6b49a96773fe" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:32.081578 30035 raft_consensus.cc:359] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:32.082422 30035 raft_consensus.cc:385] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:32.083091 30035 raft_consensus.cc:740] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:32.083884 30035 consensus_queue.cc:260] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:32.084780 29732 leader_election.cc:304] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:32.091588 30035 ts_tablet_manager.cc:1434] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:10:32.092435 30457 raft_consensus.cc:3060] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:32.108392 30468 consensus_queue.cc:1048] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:32.128674 30380 consensus_queue.cc:1048] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:10:32.133427 30505 log.cc:927] Time spent T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:10:32.148460 30457 raft_consensus.cc:2749] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:32.208091 30382 consensus_queue.cc:1048] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:10:32.249608 29767 raft_consensus.cc:3060] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:32.268719 30511 consensus_queue.cc:1048] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.272575 30511 consensus_queue.cc:1048] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.290416 30470 consensus_queue.cc:1048] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.310497 30457 consensus_queue.cc:1048] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:32.347554 30075 consensus_queue.cc:1048] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.347872 30511 consensus_queue.cc:1048] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:32.386059 30511 consensus_queue.cc:1048] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.399480 30349 consensus_queue.cc:1048] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.426538 30380 consensus_queue.cc:1048] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.456286 30380 consensus_queue.cc:1048] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:32.856942 19967 tablet_server.cc:179] TabletServer@127.19.127.193:0 shutting down...
W20260709 13:10:32.863517 29807 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20260709 13:10:32.872306 29807 consensus_peers.cc:597] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.898274 29807 consensus_peers.cc:597] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.898904 29807 consensus_peers.cc:597] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.911649 29955 consensus_peers.cc:597] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.928498 29955 consensus_peers.cc:597] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.933131 29955 consensus_peers.cc:597] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.938894 29807 consensus_peers.cc:597] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.951452 29807 consensus_peers.cc:597] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.960861 29881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:35599, user_credentials={real_user=slave}} blocked reactor thread for 46398.7us
W20260709 13:10:32.962306 29955 consensus_peers.cc:597] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.964622 29881 consensus_peers.cc:597] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.967370 29955 consensus_peers.cc:597] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.968036 29955 consensus_peers.cc:597] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.979939 29955 consensus_peers.cc:597] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:32.980450 29881 consensus_peers.cc:597] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:32.988315 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:32.989478 19967 tablet_replica.cc:333] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:32.990197 19967 raft_consensus.cc:2243] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:32.990654 19967 raft_consensus.cc:2272] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:32.993463 19967 tablet_replica.cc:333] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:32.994148 19967 raft_consensus.cc:2243] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:32.995118 19967 raft_consensus.cc:2272] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:32.997982 19967 tablet_replica.cc:333] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:32.998795 19967 raft_consensus.cc:2243] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:32.999359 19967 raft_consensus.cc:2272] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:33.001539 29955 consensus_peers.cc:597] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.002038 19967 tablet_replica.cc:333] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.002820 19967 raft_consensus.cc:2243] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.003258 19967 raft_consensus.cc:2272] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.005523 19967 tablet_replica.cc:333] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.006253 19967 raft_consensus.cc:2243] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.006736 19967 raft_consensus.cc:2272] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:33.007331 29807 consensus_peers.cc:597] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.008749 19967 tablet_replica.cc:333] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.009577 19967 raft_consensus.cc:2243] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.010058 19967 raft_consensus.cc:2272] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.011982 19967 tablet_replica.cc:333] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.012601 19967 raft_consensus.cc:2243] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.013018 19967 raft_consensus.cc:2272] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.014904 19967 tablet_replica.cc:333] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.015733 19967 raft_consensus.cc:2243] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.016238 19967 raft_consensus.cc:2272] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.018901 19967 tablet_replica.cc:333] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.019814 19967 raft_consensus.cc:2243] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.020339 19967 raft_consensus.cc:2272] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.022145 19967 tablet_replica.cc:333] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.022765 19967 raft_consensus.cc:2243] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.023241 19967 raft_consensus.cc:2272] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.025068 19967 tablet_replica.cc:333] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.025672 19967 raft_consensus.cc:2243] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.026093 19967 raft_consensus.cc:2272] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.028352 19967 tablet_replica.cc:333] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.029067 19967 raft_consensus.cc:2243] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.029579 19967 raft_consensus.cc:2272] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.031368 19967 tablet_replica.cc:333] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.031934 19967 raft_consensus.cc:2243] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.032306 19967 raft_consensus.cc:2272] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.034044 19967 tablet_replica.cc:333] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.034574 19967 raft_consensus.cc:2243] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.034929 19967 raft_consensus.cc:2272] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.037235 19967 tablet_replica.cc:333] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.039215 19967 raft_consensus.cc:2243] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.039775 19967 raft_consensus.cc:2272] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.043623 19967 tablet_replica.cc:333] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.044306 19967 raft_consensus.cc:2243] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.044814 19967 raft_consensus.cc:2272] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.046759 19967 tablet_replica.cc:333] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.047518 19967 raft_consensus.cc:2243] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.048435 19967 raft_consensus.cc:2272] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.050480 19967 tablet_replica.cc:333] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.051054 19967 raft_consensus.cc:2243] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.051919 19967 raft_consensus.cc:2272] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.054131 19967 tablet_replica.cc:333] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.054699 19967 raft_consensus.cc:2243] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.055130 19967 raft_consensus.cc:2272] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.057801 19967 tablet_replica.cc:333] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.058656 19967 raft_consensus.cc:2243] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.059655 19967 raft_consensus.cc:2272] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.062086 19967 tablet_replica.cc:333] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.062677 19967 raft_consensus.cc:2243] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.063501 19967 raft_consensus.cc:2272] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.065531 19967 tablet_replica.cc:333] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.066108 19967 raft_consensus.cc:2243] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.066958 19967 raft_consensus.cc:2272] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.069078 19967 tablet_replica.cc:333] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.069640 19967 raft_consensus.cc:2243] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.070107 19967 raft_consensus.cc:2272] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.072551 19967 tablet_replica.cc:333] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.073145 19967 raft_consensus.cc:2243] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.073583 19967 raft_consensus.cc:2272] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:33.074169 29807 consensus_peers.cc:597] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.075700 19967 tablet_replica.cc:333] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.076368 19967 raft_consensus.cc:2243] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.077153 19967 raft_consensus.cc:2272] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.080070 19967 tablet_replica.cc:333] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.080883 19967 raft_consensus.cc:2243] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.081418 19967 raft_consensus.cc:2272] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:33.081918 29955 consensus_peers.cc:597] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.083671 19967 tablet_replica.cc:333] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.084326 19967 raft_consensus.cc:2243] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.084764 19967 raft_consensus.cc:2272] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.086787 19967 tablet_replica.cc:333] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.087463 19967 raft_consensus.cc:2243] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.088001 19967 raft_consensus.cc:2272] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.089947 19967 tablet_replica.cc:333] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.090492 19967 raft_consensus.cc:2243] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.090962 19967 raft_consensus.cc:2272] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.093806 19967 tablet_replica.cc:333] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.094539 19967 raft_consensus.cc:2243] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.095018 19967 raft_consensus.cc:2272] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.097249 19967 tablet_replica.cc:333] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.097836 19967 raft_consensus.cc:2243] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.098268 19967 raft_consensus.cc:2272] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.100818 19967 tablet_replica.cc:333] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.101617 19967 raft_consensus.cc:2243] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.102138 19967 raft_consensus.cc:2272] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.104913 19967 tablet_replica.cc:333] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.105688 19967 raft_consensus.cc:2243] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.106707 19967 raft_consensus.cc:2272] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.109110 19967 tablet_replica.cc:333] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.109853 19967 raft_consensus.cc:2243] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.110383 19967 raft_consensus.cc:2272] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.112953 19967 tablet_replica.cc:333] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
W20260709 13:10:33.113341 29881 consensus_peers.cc:597] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.113667 19967 raft_consensus.cc:2243] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.114269 19967 raft_consensus.cc:2272] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.117156 19967 tablet_replica.cc:333] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.117954 19967 raft_consensus.cc:2243] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.118350 19967 raft_consensus.cc:2272] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.120944 19967 tablet_replica.cc:333] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.121587 19967 raft_consensus.cc:2243] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.122040 19967 raft_consensus.cc:2272] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.124730 19967 tablet_replica.cc:333] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.125371 19967 raft_consensus.cc:2243] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.125825 19967 raft_consensus.cc:2272] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.128283 19967 tablet_replica.cc:333] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.128855 19967 raft_consensus.cc:2243] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.129611 19967 raft_consensus.cc:2272] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.131712 19967 tablet_replica.cc:333] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.132249 19967 raft_consensus.cc:2243] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.132673 19967 raft_consensus.cc:2272] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.135064 19967 tablet_replica.cc:333] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.135771 19967 raft_consensus.cc:2243] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.136274 19967 raft_consensus.cc:2272] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:33.138469 29807 consensus_peers.cc:597] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.138983 19967 tablet_replica.cc:333] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.139709 19967 raft_consensus.cc:2243] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.140132 19967 raft_consensus.cc:2272] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.142109 19967 tablet_replica.cc:333] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.142619 19967 raft_consensus.cc:2243] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.143539 19967 raft_consensus.cc:2272] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.145481 19967 tablet_replica.cc:333] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.145988 19967 raft_consensus.cc:2243] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.146394 19967 raft_consensus.cc:2272] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.148699 19967 tablet_replica.cc:333] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: stopping tablet replica
I20260709 13:10:33.149199 19967 raft_consensus.cc:2243] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.150280 19967 raft_consensus.cc:2272] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:33.163830 29881 consensus_peers.cc:597] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.166771 29955 consensus_peers.cc:597] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.168334 29955 consensus_peers.cc:597] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.174751 29807 consensus_peers.cc:597] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.175594 29881 consensus_peers.cc:597] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.177292 29807 consensus_peers.cc:597] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.187508 29955 consensus_peers.cc:597] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.192987 29807 consensus_peers.cc:597] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.229615 19967 tablet_server.cc:196] TabletServer@127.19.127.193:0 shutdown complete.
W20260709 13:10:33.236479 29955 consensus_peers.cc:597] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.249858 29807 consensus_peers.cc:597] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.289105 29955 consensus_peers.cc:597] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.309595 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:33.320596 30533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:33.324853 30534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:33.336261 29955 consensus_peers.cc:597] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:33.337170 30536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:33.338709 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:33.341764 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:33.342079 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:33.342306 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602633342290 us; error 0 us; skew 500 ppm
I20260709 13:10:33.342849 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:33.345691 19967 webserver.cc:533] Webserver started at http://127.19.127.193:37711/ using document root <none> and password file <none>
I20260709 13:10:33.346204 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:33.346412 19967 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:10:33.350648 19967 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.001s
W20260709 13:10:33.357909 29807 consensus_peers.cc:597] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.360725 30541 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:33.361833 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 13:10:33.362260 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "2ba28923c4534c78903f759f0a4e1265"
format_stamp: "Formatted at 2026-07-09 13:10:21 on dist-test-slave-s3n6"
I20260709 13:10:33.363257 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:33.385174 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:33.386615 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:33.388566 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:33.394641 30548 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:10:33.444790 29881 consensus_peers.cc:597] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.692207 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:10:33.692660 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.300s	user 0.006s	sys 0.004s
I20260709 13:10:33.693053 19967 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:10:33.697727 30548 tablet_bootstrap.cc:492] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.712975 30548 tablet_bootstrap.cc:492] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.713960 30548 tablet_bootstrap.cc:492] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:33.714555 30548 ts_tablet_manager.cc:1403] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:10:33.717113 30548 raft_consensus.cc:359] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:33.717764 30548 raft_consensus.cc:740] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.718500 30548 consensus_queue.cc:260] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:33.721242 30548 ts_tablet_manager.cc:1434] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.002s
I20260709 13:10:33.722054 30548 tablet_bootstrap.cc:492] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.749716 30548 tablet_bootstrap.cc:492] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.750489 30548 tablet_bootstrap.cc:492] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:33.751206 30548 ts_tablet_manager.cc:1403] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.008s
I20260709 13:10:33.753528 30548 raft_consensus.cc:359] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:33.754200 30548 raft_consensus.cc:740] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.754809 30548 consensus_queue.cc:260] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:33.756659 30548 ts_tablet_manager.cc:1434] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:33.758961 30548 tablet_bootstrap.cc:492] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.814483 30548 tablet_bootstrap.cc:492] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.815670 30548 tablet_bootstrap.cc:492] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:33.816269 30548 ts_tablet_manager.cc:1403] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.058s	user 0.022s	sys 0.031s
I20260709 13:10:33.819463 30548 raft_consensus.cc:359] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:33.820133 30548 raft_consensus.cc:740] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.820904 30548 consensus_queue.cc:260] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:33.822690 30548 ts_tablet_manager.cc:1434] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:33.823559 30548 tablet_bootstrap.cc:492] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.842926 30548 tablet_bootstrap.cc:492] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.843909 30548 tablet_bootstrap.cc:492] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:33.844550 30548 ts_tablet_manager.cc:1403] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.001s
I20260709 13:10:33.846817 30548 raft_consensus.cc:359] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:33.847507 30548 raft_consensus.cc:740] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.848177 30548 consensus_queue.cc:260] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:33.850273 30548 ts_tablet_manager.cc:1434] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:10:33.851352 30548 tablet_bootstrap.cc:492] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.873917 30548 tablet_bootstrap.cc:492] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.874876 30548 tablet_bootstrap.cc:492] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:33.875540 30548 ts_tablet_manager.cc:1403] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.015s
I20260709 13:10:33.877614 30548 raft_consensus.cc:359] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:33.878121 30548 raft_consensus.cc:740] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.878778 30548 consensus_queue.cc:260] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:33.880290 30548 ts_tablet_manager.cc:1434] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:10:33.881064 30548 tablet_bootstrap.cc:492] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.898285 30548 tablet_bootstrap.cc:492] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.899035 30548 tablet_bootstrap.cc:492] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:33.899614 30548 ts_tablet_manager.cc:1403] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:10:33.901427 30548 raft_consensus.cc:359] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:33.901927 30548 raft_consensus.cc:740] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.902478 30548 consensus_queue.cc:260] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:33.904062 30548 ts_tablet_manager.cc:1434] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:10:33.904837 30548 tablet_bootstrap.cc:492] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.924276 30548 tablet_bootstrap.cc:492] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.925740 30548 tablet_bootstrap.cc:492] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:33.926384 30548 ts_tablet_manager.cc:1403] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 13:10:33.928879 30548 raft_consensus.cc:359] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:33.929554 30548 raft_consensus.cc:740] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.930153 30548 consensus_queue.cc:260] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:33.932282 30548 ts_tablet_manager.cc:1434] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:33.933102 30548 tablet_bootstrap.cc:492] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.958246 30548 tablet_bootstrap.cc:492] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.959271 30548 tablet_bootstrap.cc:492] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:33.960005 30548 ts_tablet_manager.cc:1403] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.008s
I20260709 13:10:33.962643 30548 raft_consensus.cc:359] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:33.963428 30548 raft_consensus.cc:740] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.964265 30548 consensus_queue.cc:260] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:33.966446 30548 ts_tablet_manager.cc:1434] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:33.967180 30548 tablet_bootstrap.cc:492] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.979218 30548 tablet_bootstrap.cc:492] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.980053 30548 tablet_bootstrap.cc:492] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:33.980667 30548 ts_tablet_manager.cc:1403] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.007s
I20260709 13:10:33.983036 30548 raft_consensus.cc:359] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:33.983728 30548 raft_consensus.cc:740] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.984387 30548 consensus_queue.cc:260] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:33.985035 30530 raft_consensus.cc:493] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:33.985565 30530 raft_consensus.cc:515] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:33.986358 30548 ts_tablet_manager.cc:1434] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:33.987179 30548 tablet_bootstrap.cc:492] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:33.987577 30530 leader_election.cc:290] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:33.991263 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:33.991782 19967 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:10:33.992094 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.299s	user 0.252s	sys 0.002s
W20260709 13:10:33.998028 29807 leader_election.cc:336] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
I20260709 13:10:33.998926 29807 leader_election.cc:304] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:34.000805 30556 raft_consensus.cc:2749] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.006799 30548 tablet_bootstrap.cc:492] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.007928 30548 tablet_bootstrap.cc:492] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.008605 30548 ts_tablet_manager.cc:1403] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 13:10:34.011072 30548 raft_consensus.cc:359] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.011826 30548 raft_consensus.cc:740] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.012640 30548 consensus_queue.cc:260] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.015340 30548 ts_tablet_manager.cc:1434] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:10:34.016151 30548 tablet_bootstrap.cc:492] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.043365 30548 tablet_bootstrap.cc:492] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.044356 30548 tablet_bootstrap.cc:492] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.045019 30548 ts_tablet_manager.cc:1403] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.004s
I20260709 13:10:34.047883 30548 raft_consensus.cc:359] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.048588 30548 raft_consensus.cc:740] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.049331 30548 consensus_queue.cc:260] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.052168 30548 ts_tablet_manager.cc:1434] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:34.052968 30548 tablet_bootstrap.cc:492] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.073535 30548 tablet_bootstrap.cc:492] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.074481 30548 tablet_bootstrap.cc:492] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.075098 30548 ts_tablet_manager.cc:1403] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.000s
I20260709 13:10:34.077898 30548 raft_consensus.cc:359] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.078562 30548 raft_consensus.cc:740] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.079339 30548 consensus_queue.cc:260] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.081362 30512 raft_consensus.cc:493] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.081941 30512 raft_consensus.cc:515] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.084118 30512 leader_election.cc:290] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.085202 30548 ts_tablet_manager.cc:1434] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:10:34.086079 30548 tablet_bootstrap.cc:492] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.086292 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:34.090529 29807 leader_election.cc:336] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
I20260709 13:10:34.091355 29807 leader_election.cc:304] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:34.092175 30512 raft_consensus.cc:2749] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.100575 30548 tablet_bootstrap.cc:492] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.101562 30548 tablet_bootstrap.cc:492] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.102226 30548 ts_tablet_manager.cc:1403] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:10:34.105074 30548 raft_consensus.cc:359] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.105927 30548 raft_consensus.cc:740] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.106664 30548 consensus_queue.cc:260] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.108951 30512 raft_consensus.cc:493] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.109587 30548 ts_tablet_manager.cc:1434] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:34.109577 30512 raft_consensus.cc:515] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.110841 30548 tablet_bootstrap.cc:492] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.111721 30512 leader_election.cc:290] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.112663 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
W20260709 13:10:34.118355 29807 leader_election.cc:336] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
I20260709 13:10:34.118854 29807 leader_election.cc:304] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:34.120059 30512 raft_consensus.cc:2749] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.126899 30549 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.127516 30549 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.129776 30549 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.132462 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:34.134809 29881 leader_election.cc:336] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
I20260709 13:10:34.135707 29881 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:34.136427 30549 raft_consensus.cc:2749] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.138278 30354 raft_consensus.cc:493] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.138844 30354 raft_consensus.cc:515] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.140659 30354 leader_election.cc:290] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.142148 30548 tablet_bootstrap.cc:492] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.143153 30548 tablet_bootstrap.cc:492] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.144145 30548 ts_tablet_manager.cc:1403] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.000s
I20260709 13:10:34.145525 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:34.150991 30548 raft_consensus.cc:359] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.151741 30548 raft_consensus.cc:740] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
W20260709 13:10:34.152418 29955 leader_election.cc:336] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
I20260709 13:10:34.152395 30548 consensus_queue.cc:260] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.153156 29955 leader_election.cc:304] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:34.153985 30354 raft_consensus.cc:2749] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.155200 30548 ts_tablet_manager.cc:1434] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:10:34.156529 30548 tablet_bootstrap.cc:492] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.168452 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:35599
I20260709 13:10:34.168635 30625 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:35599 every 8 connection(s)
I20260709 13:10:34.170779 30548 tablet_bootstrap.cc:492] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.171815 30548 tablet_bootstrap.cc:492] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.172356 30548 ts_tablet_manager.cc:1403] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.002s
I20260709 13:10:34.175076 30548 raft_consensus.cc:359] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.175802 30548 raft_consensus.cc:740] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.176582 30548 consensus_queue.cc:260] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.184463 30548 ts_tablet_manager.cc:1434] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:10:34.185384 30548 tablet_bootstrap.cc:492] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.208218 30548 tablet_bootstrap.cc:492] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.209172 30548 tablet_bootstrap.cc:492] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.209805 30548 ts_tablet_manager.cc:1403] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.000s
I20260709 13:10:34.212289 30548 raft_consensus.cc:359] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.212967 30548 raft_consensus.cc:740] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.213619 30548 consensus_queue.cc:260] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.215750 30548 ts_tablet_manager.cc:1434] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:10:34.216362 30626 heartbeater.cc:344] Connected to a master server at 127.19.127.254:37437
I20260709 13:10:34.216562 30548 tablet_bootstrap.cc:492] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.216832 30626 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:34.217772 30626 heartbeater.cc:507] Master 127.19.127.254:37437 requested a full tablet report, sending...
I20260709 13:10:34.232314 30548 tablet_bootstrap.cc:492] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.233167 30548 tablet_bootstrap.cc:492] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.233742 30548 ts_tablet_manager.cc:1403] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.003s
I20260709 13:10:34.235636 29661 ts_manager.cc:194] Re-registered known tserver with Master: 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.235664 30548 raft_consensus.cc:359] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.236478 30548 raft_consensus.cc:740] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.237038 30548 consensus_queue.cc:260] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.238876 30548 ts_tablet_manager.cc:1434] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:34.239604 30548 tablet_bootstrap.cc:492] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.247547 30549 raft_consensus.cc:493] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.248145 30549 raft_consensus.cc:515] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.252774 30549 leader_election.cc:290] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.253672 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
W20260709 13:10:34.256013 29881 leader_election.cc:343] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.256778 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:34.260749 29881 leader_election.cc:304] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:34.261631 30549 raft_consensus.cc:2749] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.265915 30548 tablet_bootstrap.cc:492] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.266659 30512 raft_consensus.cc:493] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.267047 30548 tablet_bootstrap.cc:492] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.267335 30512 raft_consensus.cc:515] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.267872 30548 ts_tablet_manager.cc:1403] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.008s
I20260709 13:10:34.269148 30354 raft_consensus.cc:493] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.269743 30354 raft_consensus.cc:515] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.270666 30548 raft_consensus.cc:359] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.271416 30548 raft_consensus.cc:740] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.272207 30548 consensus_queue.cc:260] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.274115 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:34.274413 30548 ts_tablet_manager.cc:1434] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:34.273176 30512 leader_election.cc:290] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.274784 29988 raft_consensus.cc:2468] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:34.275359 30548 tablet_bootstrap.cc:492] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.276420 29807 leader_election.cc:304] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:34.278676 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.281890 30599 raft_consensus.cc:2468] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:34.282958 30598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.283715 30598 raft_consensus.cc:2468] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:34.283747 29955 leader_election.cc:304] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:34.279215 30512 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.285257 30512 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.279417 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:34.287681 30349 raft_consensus.cc:2804] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.288120 30349 raft_consensus.cc:493] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.288447 30349 raft_consensus.cc:3060] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.277573 30354 leader_election.cc:290] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.280997 30556 raft_consensus.cc:2804] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.291486 30556 raft_consensus.cc:493] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.291913 30556 raft_consensus.cc:3060] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.292858 30598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.293586 30598 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:34.295466 30349 raft_consensus.cc:515] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.297501 30349 leader_election.cc:290] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.298549 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:34.299873 30556 raft_consensus.cc:515] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.300776 30598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.301070 29839 raft_consensus.cc:2393] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75162022240a4fe1a113333e77964ca3 in current term 2: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:34.301882 30598 raft_consensus.cc:3060] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.311518 29807 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:34.311132 30556 leader_election.cc:290] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.313211 30549 raft_consensus.cc:493] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.313786 30549 raft_consensus.cc:515] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.314160 29661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56912
I20260709 13:10:34.316058 30549 leader_election.cc:290] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.316910 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.317603 30599 raft_consensus.cc:2468] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:34.318074 30530 raft_consensus.cc:2804] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.318506 30530 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.318809 29881 leader_election.cc:304] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:34.318852 30530 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.320169 30549 raft_consensus.cc:2804] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.320521 30626 heartbeater.cc:499] Master 127.19.127.254:37437 was elected leader, sending a full tablet report...
I20260709 13:10:34.320611 30549 raft_consensus.cc:493] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.322409 30549 raft_consensus.cc:3060] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.323434 30147 raft_consensus.cc:493] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.323994 30147 raft_consensus.cc:515] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.326363 30147 leader_election.cc:290] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.328616 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.329573 30599 raft_consensus.cc:2468] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:34.332681 29881 leader_election.cc:304] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:34.333307 30556 raft_consensus.cc:493] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.333496 30441 raft_consensus.cc:2804] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.333899 30441 raft_consensus.cc:493] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.333866 30556 raft_consensus.cc:515] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.334424 30441 raft_consensus.cc:3060] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.341318 30556 leader_election.cc:290] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:34.344203 30530 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.351469 30530 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:34.345227 30548 tablet_bootstrap.cc:492] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.346990 30597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.347604 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:34.363057 29987 raft_consensus.cc:2393] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 2: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:34.348194 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:34.364349 29916 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:34.344203 30549 raft_consensus.cc:515] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.312322 30512 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:34.353659 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.356139 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:34.366381 29914 raft_consensus.cc:2468] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:34.356523 30441 raft_consensus.cc:515] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.368422 30441 leader_election.cc:290] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.369239 30549 leader_election.cc:290] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.357632 30636 raft_consensus.cc:493] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.357920 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.359046 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:34.360915 30598 raft_consensus.cc:2468] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:34.362365 30548 tablet_bootstrap.cc:492] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.370191 30599 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.370746 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.371384 30548 ts_tablet_manager.cc:1403] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.096s	user 0.031s	sys 0.013s
I20260709 13:10:34.371647 30596 raft_consensus.cc:3060] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.374207 30548 raft_consensus.cc:359] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.370218 29911 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.374941 30548 raft_consensus.cc:740] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.378969 30599 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:34.379944 30548 consensus_queue.cc:260] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.380916 30638 raft_consensus.cc:493] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.381820 30548 ts_tablet_manager.cc:1434] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:10:34.381678 30638 raft_consensus.cc:515] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.382648 30548 tablet_bootstrap.cc:492] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.384770 30638 leader_election.cc:290] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.387647 29911 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:34.389483 29955 leader_election.cc:304] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:34.389870 30596 raft_consensus.cc:2468] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:34.390583 30511 raft_consensus.cc:2804] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.391098 30511 raft_consensus.cc:697] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:34.391171 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.391849 30599 raft_consensus.cc:3060] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.391983 30511 consensus_queue.cc:237] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.398180 30636 raft_consensus.cc:515] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.400938 30598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.401432 30637 raft_consensus.cc:493] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.406190 29881 leader_election.cc:304] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:34.399102 30599 raft_consensus.cc:2468] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:34.408368 30549 raft_consensus.cc:2804] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.408893 30549 raft_consensus.cc:697] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:34.409219 29661 catalog_manager.cc:5697] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 2 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.409691 30549 consensus_queue.cc:237] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.410522 30637 raft_consensus.cc:515] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.412781 30637 leader_election.cc:290] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:34.413700 30636 leader_election.cc:290] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:34.414163 29881 leader_election.cc:343] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.414340 30548 tablet_bootstrap.cc:492] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.415622 30548 tablet_bootstrap.cc:492] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.415720 29881 leader_election.cc:304] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:34.416430 30548 ts_tablet_manager.cc:1403] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.006s
I20260709 13:10:34.416771 30147 raft_consensus.cc:2804] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.417295 30147 raft_consensus.cc:697] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:34.418025 30147 consensus_queue.cc:237] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.419589 30548 raft_consensus.cc:359] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.420392 30548 raft_consensus.cc:740] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.422379 30349 raft_consensus.cc:493] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.422895 30349 raft_consensus.cc:515] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.423205 30354 raft_consensus.cc:493] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.423722 30354 raft_consensus.cc:515] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.424829 30349 leader_election.cc:290] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.425433 30354 leader_election.cc:290] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.427423 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:34.427716 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:34.428354 29841 raft_consensus.cc:2468] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:34.428253 30548 consensus_queue.cc:260] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.427430 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.429822 30599 raft_consensus.cc:2468] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:34.430509 30548 ts_tablet_manager.cc:1434] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 13:10:34.431129 29955 leader_election.cc:304] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:34.429442 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.432574 29955 leader_election.cc:304] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:34.433257 30349 raft_consensus.cc:2804] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.433596 30349 raft_consensus.cc:493] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.433800 30354 raft_consensus.cc:2804] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.433952 30349 raft_consensus.cc:3060] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.434185 30354 raft_consensus.cc:493] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.434546 30354 raft_consensus.cc:3060] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.440994 30349 raft_consensus.cc:515] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.444144 30349 leader_election.cc:290] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
W20260709 13:10:34.444679 29807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:46707, user_credentials={real_user=slave}} blocked reactor thread for 85583.1us
I20260709 13:10:34.431386 30548 tablet_bootstrap.cc:492] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.445933 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:34.446488 29955 leader_election.cc:343] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.447122 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:34.447805 29842 raft_consensus.cc:2468] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:34.449649 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:34.451023 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.452299 30599 raft_consensus.cc:3060] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.457373 30354 raft_consensus.cc:515] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.459815 29837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:34.460338 30354 leader_election.cc:290] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.460508 29837 raft_consensus.cc:3060] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.461776 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:34.462039 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a9fd11d5a348389207646896abb152" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.462185 30599 raft_consensus.cc:2468] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:34.462379 29841 raft_consensus.cc:3060] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:34.463315 29955 leader_election.cc:343] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.464097 29955 leader_election.cc:304] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:34.465004 30511 raft_consensus.cc:2804] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.465510 30511 raft_consensus.cc:697] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:34.466312 30511 consensus_queue.cc:237] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.468676 30639 raft_consensus.cc:493] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.469251 30639 raft_consensus.cc:515] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.470495 29837 raft_consensus.cc:2468] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:34.472268 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:34.472878 29842 raft_consensus.cc:2468] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:34.473238 29841 raft_consensus.cc:2468] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:34.474149 29837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:34.474777 29837 raft_consensus.cc:3060] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.475114 29955 leader_election.cc:304] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:34.475932 30354 raft_consensus.cc:2804] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.476660 30354 raft_consensus.cc:697] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:34.477466 30354 consensus_queue.cc:237] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.479539 30639 leader_election.cc:290] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:34.480394 29807 leader_election.cc:343] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.481343 29807 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:34.483896 30639 raft_consensus.cc:2804] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.484388 30639 raft_consensus.cc:697] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:34.485142 30639 consensus_queue.cc:237] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.486539 29664 catalog_manager.cc:5697] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:34.497025 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.497961 30599 raft_consensus.cc:2393] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 2: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:34.499945 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.500546 30599 raft_consensus.cc:2468] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:34.501561 30639 raft_consensus.cc:493] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.502125 30639 raft_consensus.cc:515] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.504760 30639 leader_election.cc:290] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.506338 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:34.506561 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:34.515190 29881 leader_election.cc:304] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:34.515401 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.516415 30147 raft_consensus.cc:2804] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.516680 30147 raft_consensus.cc:493] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.516989 30147 raft_consensus.cc:3060] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.519804 30596 raft_consensus.cc:2393] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 2: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:34.520931 30640 raft_consensus.cc:493] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.521571 30640 raft_consensus.cc:515] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.520814 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:34.526006 30640 leader_election.cc:290] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.527421 29807 leader_election.cc:304] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:34.528064 30640 raft_consensus.cc:2804] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.528473 30640 raft_consensus.cc:493] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.528877 30640 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.529390 30147 raft_consensus.cc:515] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.531054 30511 raft_consensus.cc:493] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.531826 30511 raft_consensus.cc:515] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.533025 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.533591 30511 leader_election.cc:290] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
W20260709 13:10:34.534397 29881 leader_election.cc:343] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.535426 29807 leader_election.cc:304] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:34.536478 30636 raft_consensus.cc:2749] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.538528 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:34.539170 29841 raft_consensus.cc:2468] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:34.539768 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.540124 30549 raft_consensus.cc:493] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.540647 30549 raft_consensus.cc:515] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.541487 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.541872 29663 catalog_manager.cc:5697] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 2 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.542682 30549 leader_election.cc:290] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.543610 29955 leader_election.cc:304] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
W20260709 13:10:34.544493 29955 leader_election.cc:343] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.544838 30147 leader_election.cc:290] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.544988 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:34.545637 29987 raft_consensus.cc:2468] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 1.
I20260709 13:10:34.545775 30349 raft_consensus.cc:2804] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.546216 30349 raft_consensus.cc:493] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.546550 30349 raft_consensus.cc:3060] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.547510 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.549336 30596 raft_consensus.cc:2468] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:34.551055 29881 leader_election.cc:304] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:34.548993 29837 raft_consensus.cc:2468] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:34.552001 30147 raft_consensus.cc:2804] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.552459 30147 raft_consensus.cc:493] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.552858 30147 raft_consensus.cc:3060] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.553208 30548 tablet_bootstrap.cc:492] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.553784 30349 raft_consensus.cc:515] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.556604 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:34.560106 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:34.560748 30598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.561005 30599 raft_consensus.cc:2393] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 2: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
W20260709 13:10:34.562084 29955 leader_election.cc:343] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.562798 30548 tablet_bootstrap.cc:492] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.565459 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:34.566118 29842 raft_consensus.cc:3060] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.566423 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:34.564766 29807 leader_election.cc:304] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:34.567021 29839 raft_consensus.cc:3060] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.567968 29807 leader_election.cc:304] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06
I20260709 13:10:34.568773 30556 raft_consensus.cc:2749] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.569471 30556 raft_consensus.cc:2804] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.569805 30556 raft_consensus.cc:493] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.570160 30556 raft_consensus.cc:3060] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.574707 29664 catalog_manager.cc:5697] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:34.576126 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:34.577597 30556 raft_consensus.cc:515] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.578737 29841 raft_consensus.cc:2393] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74191176370841578d396cc01f230f06 in current term 2: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
I20260709 13:10:34.580412 29842 raft_consensus.cc:2468] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:34.566141 30548 ts_tablet_manager.cc:1403] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.135s	user 0.017s	sys 0.017s
I20260709 13:10:34.565474 30349 leader_election.cc:290] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:34.583052 29839 raft_consensus.cc:2468] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:34.584348 29955 leader_election.cc:304] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:34.585003 30548 raft_consensus.cc:359] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.585281 30511 raft_consensus.cc:2804] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.567024 30640 raft_consensus.cc:515] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.585907 30511 raft_consensus.cc:697] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:34.586304 30548 raft_consensus.cc:740] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.586971 30511 consensus_queue.cc:237] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.586879 29661 catalog_manager.cc:5697] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 2 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.588231 30556 leader_election.cc:290] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:34.588856 30548 consensus_queue.cc:260] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.588665 29661 catalog_manager.cc:5697] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 2 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.589581 29807 leader_election.cc:304] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:34.592543 30556 raft_consensus.cc:2749] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:34.592984 29807 leader_election.cc:343] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.593946 30548 ts_tablet_manager.cc:1434] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:10:34.595626 30640 leader_election.cc:290] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:34.599320 30548 tablet_bootstrap.cc:492] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.599165 30147 raft_consensus.cc:515] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.606006 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:34.608559 29916 raft_consensus.cc:2393] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 2: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:34.609077 29881 leader_election.cc:304] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:34.610210 30147 leader_election.cc:290] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.611152 30549 raft_consensus.cc:2804] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.611653 30549 raft_consensus.cc:697] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:34.612433 30549 consensus_queue.cc:237] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.607522 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:34.615626 29914 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.620536 29807 leader_election.cc:304] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:34.620869 30598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.621393 30640 raft_consensus.cc:2804] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:34.621754 30640 raft_consensus.cc:493] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:34.622093 30640 raft_consensus.cc:3060] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.620548 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.625393 30596 raft_consensus.cc:3060] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.630890 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.634547 30640 raft_consensus.cc:515] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.636804 30640 leader_election.cc:290] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:34.637948 29807 leader_election.cc:304] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:34.639747 30639 raft_consensus.cc:3060] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.647632 30639 raft_consensus.cc:2749] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.646020 29914 raft_consensus.cc:2468] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:34.652735 30596 raft_consensus.cc:2468] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:34.660779 29664 catalog_manager.cc:5697] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:34.662386 30548 tablet_bootstrap.cc:492] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.663148 29807 leader_election.cc:304] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:34.663492 30548 tablet_bootstrap.cc:492] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.664500 30640 raft_consensus.cc:2804] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.664405 29661 catalog_manager.cc:5697] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 2 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.665443 30548 ts_tablet_manager.cc:1403] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.069s	user 0.032s	sys 0.015s
I20260709 13:10:34.666962 30640 raft_consensus.cc:697] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:34.669603 30548 raft_consensus.cc:359] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.670442 30548 raft_consensus.cc:740] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.671586 30548 consensus_queue.cc:260] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.670285 30640 consensus_queue.cc:237] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.673908 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:34.673969 30548 ts_tablet_manager.cc:1434] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:34.674546 29987 raft_consensus.cc:3060] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.676646 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:34.676879 30548 tablet_bootstrap.cc:492] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.679976 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:34.682376 29839 raft_consensus.cc:2393] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 74191176370841578d396cc01f230f06 in current term 2: Already voted for candidate d08b1a18bb214fa1b08560d4d8149deb in this term.
W20260709 13:10:34.680235 29807 leader_election.cc:343] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.686712 29807 leader_election.cc:304] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06
I20260709 13:10:34.692040 30639 raft_consensus.cc:2804] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.692591 30639 raft_consensus.cc:697] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:34.693444 30639 consensus_queue.cc:237] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
W20260709 13:10:34.699012 29807 leader_election.cc:343] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:34.704450 29881 leader_election.cc:304] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:34.705667 30549 raft_consensus.cc:2749] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:34.714896 29987 raft_consensus.cc:2468] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:34.729533 29807 leader_election.cc:304] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:34.731591 30530 raft_consensus.cc:2804] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.732146 30530 raft_consensus.cc:697] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:34.733007 30530 consensus_queue.cc:237] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.755112 29665 catalog_manager.cc:5697] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 2 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.763775 30548 tablet_bootstrap.cc:492] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.764838 30548 tablet_bootstrap.cc:492] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.765864 30548 ts_tablet_manager.cc:1403] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.090s	user 0.014s	sys 0.007s
I20260709 13:10:34.769767 30548 raft_consensus.cc:359] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.770555 30548 raft_consensus.cc:740] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.771765 30548 consensus_queue.cc:260] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:34.774657 30548 ts_tablet_manager.cc:1434] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:10:34.776850 30548 tablet_bootstrap.cc:492] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.780134 29664 catalog_manager.cc:5697] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 2 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.781517 29664 catalog_manager.cc:5697] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 1 to 2, leader changed from 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193) to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 2 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:34.807158 30548 tablet_bootstrap.cc:492] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.808048 30548 tablet_bootstrap.cc:492] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.808740 30548 ts_tablet_manager.cc:1403] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.011s
I20260709 13:10:34.810935 30548 raft_consensus.cc:359] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.811789 30548 raft_consensus.cc:740] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.812354 30548 consensus_queue.cc:260] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.814102 30548 ts_tablet_manager.cc:1434] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 13:10:34.814911 30548 tablet_bootstrap.cc:492] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.828680 30548 tablet_bootstrap.cc:492] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.829380 30548 tablet_bootstrap.cc:492] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.829887 30548 ts_tablet_manager.cc:1403] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:10:34.832417 30548 raft_consensus.cc:359] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.833081 30548 raft_consensus.cc:740] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.833688 30548 consensus_queue.cc:260] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.835613 30548 ts_tablet_manager.cc:1434] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:10:34.836794 30548 tablet_bootstrap.cc:492] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.851372 30548 tablet_bootstrap.cc:492] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.852212 30548 tablet_bootstrap.cc:492] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.852959 30548 ts_tablet_manager.cc:1403] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 13:10:34.855297 30548 raft_consensus.cc:359] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.855966 30548 raft_consensus.cc:740] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.856594 30548 consensus_queue.cc:260] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:34.858565 30548 ts_tablet_manager.cc:1434] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:34.859525 30548 tablet_bootstrap.cc:492] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.861207 29839 raft_consensus.cc:1275] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:34.862751 30147 consensus_queue.cc:1048] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:34.876977 30596 raft_consensus.cc:1275] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:34.878684 30147 consensus_queue.cc:1048] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:34.892261 30548 tablet_bootstrap.cc:492] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:34.893409 30548 tablet_bootstrap.cc:492] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:34.894138 30548 ts_tablet_manager.cc:1403] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.000s
I20260709 13:10:34.896780 30548 raft_consensus.cc:359] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.897533 30548 raft_consensus.cc:740] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.898305 30548 consensus_queue.cc:260] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:34.900899 30548 ts_tablet_manager.cc:1434] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.004s
I20260709 13:10:34.902777 30597 raft_consensus.cc:1275] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:34.908733 30548 tablet_bootstrap.cc:492] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:34.910451 30549 consensus_queue.cc:1048] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:34.915479 29842 raft_consensus.cc:1275] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:34.919207 30354 consensus_queue.cc:1048] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:34.923154 29841 raft_consensus.cc:1275] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:34.926651 30349 consensus_queue.cc:1048] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:34.929332 29955 consensus_peers.cc:597] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:34.940227 29837 raft_consensus.cc:1275] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:34.946241 30147 consensus_queue.cc:1048] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:34.967633 30599 raft_consensus.cc:1275] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:34.969795 30530 consensus_queue.cc:1048] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:35.008625 30548 tablet_bootstrap.cc:492] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:35.012156 30548 tablet_bootstrap.cc:492] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:35.013098 30548 ts_tablet_manager.cc:1403] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.112s	user 0.024s	sys 0.021s
I20260709 13:10:35.016726 30548 raft_consensus.cc:359] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:35.025125 30548 raft_consensus.cc:740] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.029778 30548 consensus_queue.cc:260] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:35.036038 30548 ts_tablet_manager.cc:1434] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:10:35.042611 30548 tablet_bootstrap.cc:492] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:35.070581 29914 raft_consensus.cc:1275] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.072438 30640 consensus_queue.cc:1048] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:10:34.997313 29955 consensus_peers.cc:597] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:35.088371 29955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:35599, user_credentials={real_user=slave}} blocked reactor thread for 91263.9us
I20260709 13:10:35.091169 30597 raft_consensus.cc:1275] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.100371 30597 raft_consensus.cc:1275] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.103112 30640 consensus_queue.cc:1048] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.116514 30594 raft_consensus.cc:1275] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.127213 30548 tablet_bootstrap.cc:492] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:35.128235 30548 tablet_bootstrap.cc:492] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:35.128890 30548 ts_tablet_manager.cc:1403] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.087s	user 0.017s	sys 0.016s
W20260709 13:10:35.153090 29807 consensus_peers.cc:597] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:35.157800 30595 raft_consensus.cc:3060] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:35.175606 30511 consensus_queue.cc:1048] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.177801 30511 consensus_queue.cc:1048] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.171849 30599 raft_consensus.cc:3060] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:35.189257 29914 raft_consensus.cc:1275] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.175069 29837 raft_consensus.cc:1275] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:10:35.179312 29955 consensus_peers.cc:597] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:35.163107 30548 raft_consensus.cc:359] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:35.200102 30548 raft_consensus.cc:740] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.207854 29987 raft_consensus.cc:1275] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.210372 30595 raft_consensus.cc:1275] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.210812 29914 raft_consensus.cc:1275] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.214282 30530 consensus_queue.cc:1048] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:35.204111 30548 consensus_queue.cc:260] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:35.220266 30512 consensus_queue.cc:1048] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.226969 30548 ts_tablet_manager.cc:1434] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.098s	user 0.007s	sys 0.000s
I20260709 13:10:35.227486 30671 consensus_queue.cc:1048] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.227942 30548 tablet_bootstrap.cc:492] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:35.204778 30639 consensus_queue.cc:1048] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
W20260709 13:10:35.232697 29807 consensus_peers.cc:597] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:35.233645 29807 consensus_peers.cc:597] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:35.252635 29955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39603, user_credentials={real_user=slave}} blocked reactor thread for 50643.3us
I20260709 13:10:35.260847 30595 raft_consensus.cc:3060] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:35.262238 29839 raft_consensus.cc:1275] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.266093 29839 raft_consensus.cc:1275] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.267935 30354 consensus_queue.cc:1048] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:35.298269 30349 consensus_queue.cc:1048] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:35.293195 30667 consensus_queue.cc:1048] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:10:35.329632 30599 raft_consensus.cc:1275] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:10:35.340037 29807 consensus_peers.cc:597] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:35.366233 29955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:39603, user_credentials={real_user=slave}} blocked reactor thread for 71106.8us
I20260709 13:10:35.377550 30595 raft_consensus.cc:1275] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.410245 30349 consensus_queue.cc:1048] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:35.447124 29807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:35599, user_credentials={real_user=slave}} blocked reactor thread for 66678.1us
I20260709 13:10:35.420245 30639 consensus_queue.cc:1048] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
W20260709 13:10:35.511883 29955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 95462.2us
W20260709 13:10:35.547359 30695 log.cc:927] Time spent T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:10:35.548089 30548 tablet_bootstrap.cc:492] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:35.549366 30548 tablet_bootstrap.cc:492] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:35.550081 30548 ts_tablet_manager.cc:1403] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.322s	user 0.049s	sys 0.038s
W20260709 13:10:35.561758 29881 consensus_peers.cc:597] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:35.567888 30696 log.cc:927] Time spent T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:10:35.581568 29807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 45053.6us
I20260709 13:10:35.589883 30548 raft_consensus.cc:359] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:35.590792 30548 raft_consensus.cc:740] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.591602 30548 consensus_queue.cc:260] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:35.593842 30548 ts_tablet_manager.cc:1434] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.000s
I20260709 13:10:35.609629 30594 raft_consensus.cc:3060] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:35.643707 30548 tablet_bootstrap.cc:492] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
W20260709 13:10:35.645143 29881 consensus_peers.cc:597] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:35.699632 29955 consensus_peers.cc:597] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:35.726785 30548 tablet_bootstrap.cc:492] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:35.727905 30548 tablet_bootstrap.cc:492] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:35.728569 30548 ts_tablet_manager.cc:1403] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.125s	user 0.019s	sys 0.013s
I20260709 13:10:35.731012 30548 raft_consensus.cc:359] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:35.731734 30548 raft_consensus.cc:740] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.732561 30548 consensus_queue.cc:260] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:35.734083 30548 ts_tablet_manager.cc:1434] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:10:35.734879 30548 tablet_bootstrap.cc:492] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:35.763322 30548 tablet_bootstrap.cc:492] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:35.765338 30548 tablet_bootstrap.cc:492] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:35.766151 30548 ts_tablet_manager.cc:1403] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.007s
I20260709 13:10:35.768925 30548 raft_consensus.cc:359] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:35.769714 30548 raft_consensus.cc:740] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.770610 30548 consensus_queue.cc:260] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:35.773452 30548 ts_tablet_manager.cc:1434] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:35.774893 30548 tablet_bootstrap.cc:492] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:36.616937 30721 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.617470 30721 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.621047 30721 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:36.621532 30721 raft_consensus.cc:493] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.622004 30721 raft_consensus.cc:515] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.624292 30721 leader_election.cc:290] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:36.628412 30719 raft_consensus.cc:493] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.628615 30722 raft_consensus.cc:493] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.629105 30719 raft_consensus.cc:515] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:36.629160 30722 raft_consensus.cc:515] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:36.631568 30719 leader_election.cc:290] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.632266 30722 leader_election.cc:290] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
W20260709 13:10:36.649458 29807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:35599, user_credentials={real_user=slave}} blocked reactor thread for 1.00106e+06us
W20260709 13:10:36.650650 29807 consensus_peers.cc:597] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:36.654321 30721 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.655117 30721 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.656682 30530 raft_consensus.cc:493] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74191176370841578d396cc01f230f06)
I20260709 13:10:36.657435 30721 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:36.657279 30530 raft_consensus.cc:515] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.674021 30721 raft_consensus.cc:493] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:36.674697 30721 raft_consensus.cc:515] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:36.664795 30530 leader_election.cc:290] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:36.666730 30719 raft_consensus.cc:493] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.698782 30721 leader_election.cc:290] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:36.688253 30719 raft_consensus.cc:515] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:36.701788 30719 leader_election.cc:290] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
W20260709 13:10:36.712137 29955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:35599, user_credentials={real_user=slave}} blocked reactor thread for 43923.2us
I20260709 13:10:36.714505 30147 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.715137 30147 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.718160 30147 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.720996 30147 raft_consensus.cc:493] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:36.721860 30147 raft_consensus.cc:515] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:36.725416 30147 leader_election.cc:290] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:36.726074 30554 raft_consensus.cc:493] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.726672 30554 raft_consensus.cc:515] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.726707 30147 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.728072 30147 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.729960 29664 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59048 (request call id 111) took 1090 ms. Trace:
I20260709 13:10:36.728551 30721 raft_consensus.cc:493] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:36.740816 30721 raft_consensus.cc:515] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:36.741441 30717 raft_consensus.cc:493] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.742048 30717 raft_consensus.cc:515] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:36.743669 30721 leader_election.cc:290] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.746414 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:36.748040 30549 raft_consensus.cc:493] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.747090 29665 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56912 (request call id 17) took 1100 ms. Trace:
I20260709 13:10:36.765396 30349 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.765986 30349 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:36.777103 30147 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.747085 30595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523db1e9cca74ffb938f22ae9105dbe1" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:36.735946 29664 rpcz_store.cc:276] 0709 13:10:35.643414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.500567 (+857153us) service_pool.cc:224] Handling call
0709 13:10:36.729901 (+229334us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2}
I20260709 13:10:36.782437 30717 leader_election.cc:290] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.761065 29665 rpcz_store.cc:276] 0709 13:10:35.646537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:35.701871 (+ 55334us) service_pool.cc:224] Handling call
0709 13:10:36.747004 (+1045133us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4}
W20260709 13:10:36.786530 30548 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.995s	user 0.020s	sys 0.076s
I20260709 13:10:36.786674 30349 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:36.789269 30548 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.020s	sys 0.076s
I20260709 13:10:36.766640 30718 raft_consensus.cc:493] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.801736 30554 leader_election.cc:290] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:36.806315 30548 tablet_bootstrap.cc:492] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:36.811347 30595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:36.810149 30718 raft_consensus.cc:515] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.816022 30548 tablet_bootstrap.cc:492] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:36.817617 30548 ts_tablet_manager.cc:1403] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 1.043s	user 0.037s	sys 0.080s
I20260709 13:10:36.765828 30549 raft_consensus.cc:515] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.819792 30549 leader_election.cc:290] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.823524 30548 raft_consensus.cc:359] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:36.824290 30548 raft_consensus.cc:740] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.825070 30548 consensus_queue.cc:260] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:36.826650 30548 ts_tablet_manager.cc:1434] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:10:36.828047 30548 tablet_bootstrap.cc:492] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
W20260709 13:10:36.830644 29955 leader_election.cc:343] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.817754 30718 leader_election.cc:290] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.821281 30554 raft_consensus.cc:493] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 74191176370841578d396cc01f230f06)
W20260709 13:10:36.833148 29955 leader_election.cc:343] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.827807 30725 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.842077 30725 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:36.843344 30719 raft_consensus.cc:493] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.843971 30719 raft_consensus.cc:515] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:36.844583 30705 raft_consensus.cc:493] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:36.844851 30725 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:36.845145 30705 raft_consensus.cc:515] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:36.846282 30719 leader_election.cc:290] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
W20260709 13:10:36.851446 29807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:35599, user_credentials={real_user=slave}} blocked reactor thread for 200134us
I20260709 13:10:36.827979 30722 raft_consensus.cc:493] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.831499 29986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:36.840674 30554 raft_consensus.cc:515] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:36.827596 30592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:36.852953 30640 raft_consensus.cc:493] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:36.829772 30593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:36.862116 30722 raft_consensus.cc:515] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:36.847168 30705 leader_election.cc:290] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:36.893925 30354 raft_consensus.cc:493] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.895856 30354 raft_consensus.cc:515] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:36.913695 30720 raft_consensus.cc:493] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.914357 30720 raft_consensus.cc:515] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.916568 30549 raft_consensus.cc:493] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.917289 30549 raft_consensus.cc:515] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.888900 30721 raft_consensus.cc:493] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.919644 30721 raft_consensus.cc:515] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.921778 30721 leader_election.cc:290] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:36.922405 30721 raft_consensus.cc:493] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:36.873862 30640 raft_consensus.cc:515] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:36.923239 30725 raft_consensus.cc:493] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.923832 30725 raft_consensus.cc:515] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:36.925020 30640 leader_election.cc:290] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:36.925786 30725 leader_election.cc:290] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.926594 30722 leader_election.cc:290] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:36.874104 30592 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:36.880249 30593 raft_consensus.cc:2468] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:36.886682 30733 raft_consensus.cc:493] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:36.921834 30554 leader_election.cc:290] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:36.940013 30354 leader_election.cc:290] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.922861 30721 raft_consensus.cc:515] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
W20260709 13:10:36.958643 29881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.196:46707, user_credentials={real_user=slave}} blocked reactor thread for 71426.4us
I20260709 13:10:36.928429 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:36.961470 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:36.969655 29987 raft_consensus.cc:2468] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:36.963155 30701 raft_consensus.cc:493] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 74191176370841578d396cc01f230f06)
I20260709 13:10:36.964923 30707 raft_consensus.cc:493] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:36.965492 30704 raft_consensus.cc:493] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:36.973332 30701 raft_consensus.cc:515] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
W20260709 13:10:36.974625 29881 consensus_peers.cc:597] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:36.974915 29913 raft_consensus.cc:2468] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:36.975565 29838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523db1e9cca74ffb938f22ae9105dbe1" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:36.983812 29838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b780417ded422690ec3d46dda49a07" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:36.984203 29835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:36.984889 30701 leader_election.cc:290] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:36.985487 29835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:36.986717 29835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:36.946751 30549 leader_election.cc:290] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.987685 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:36.985077 29838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:36.988976 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:36.989619 29881 consensus_peers.cc:597] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:36.987782 30720 leader_election.cc:290] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
W20260709 13:10:36.992929 29955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 102857us
I20260709 13:10:36.995646 29881 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:37.000581 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db6009685e4a3d88d098ea07105ad7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.002411 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:36.980680 30554 raft_consensus.cc:493] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.010576 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:36.981988 30707 raft_consensus.cc:515] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:36.946460 30733 raft_consensus.cc:515] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.016069 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.017179 30733 leader_election.cc:290] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:36.998585 30701 raft_consensus.cc:493] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:36.991606 30704 raft_consensus.cc:515] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:37.018573 30701 raft_consensus.cc:515] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.019436 30721 leader_election.cc:290] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.020514 30701 leader_election.cc:290] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.021508 29838 raft_consensus.cc:2468] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:37.003334 30720 raft_consensus.cc:2804] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:37.011585 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.036965 29955 leader_election.cc:304] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.038436 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.046113 30720 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.046622 30720 raft_consensus.cc:3060] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.047068 29955 leader_election.cc:304] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.047875 30719 raft_consensus.cc:2749] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.048590 30719 raft_consensus.cc:2749] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.048719 29955 leader_election.cc:304] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.050119 30511 raft_consensus.cc:2749] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.052042 30743 raft_consensus.cc:493] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.052695 30743 raft_consensus.cc:515] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:37.041308 30598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ebcaf79fc9947e5a4a8e8ed8eb3dab7" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.043637 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.044092 30597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.023613 30554 raft_consensus.cc:515] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.044204 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.055030 30593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.055584 30596 raft_consensus.cc:2468] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:37.056664 30554 leader_election.cc:290] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:37.047153 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.055109 30597 raft_consensus.cc:2468] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 1.
I20260709 13:10:37.058159 30598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86eb4af5624466bb2d9395a36632d71" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.059340 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a7caaa2edf4bafacd9da8900ebeeb2" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.020535 30704 leader_election.cc:290] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:37.054420 29913 raft_consensus.cc:2468] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:37.043253 30595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "244d599ec58f4e51ac20679e26a3c2b1" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.061699 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a7caaa2edf4bafacd9da8900ebeeb2" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.062754 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.063762 30702 raft_consensus.cc:493] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74191176370841578d396cc01f230f06)
I20260709 13:10:37.061108 30743 leader_election.cc:290] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:37.064273 30702 raft_consensus.cc:515] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.066470 30702 leader_election.cc:290] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:37.069994 29955 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.073882 30349 raft_consensus.cc:2804] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:37.074251 30349 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.074581 30349 raft_consensus.cc:3060] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.084444 30592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db6009685e4a3d88d098ea07105ad7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.087173 30720 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.076557 29955 leader_election.cc:304] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.092272 29955 leader_election.cc:304] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.098269 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:37.103250 30349 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.103085 30354 raft_consensus.cc:493] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.104363 30354 raft_consensus.cc:515] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.109398 30595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.111399 30354 leader_election.cc:290] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:37.097586 30720 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.111821 29833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc233ddc2b6d41f395652a5c5010c88b" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.115458 29914 raft_consensus.cc:1240] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Rejecting Update request from peer d08b1a18bb214fa1b08560d4d8149deb for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:37.116219 30595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f49c45a4a5479d8609fe95f8486b97" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.113234 29833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f49c45a4a5479d8609fe95f8486b97" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.117949 29833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.118125 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.118170 29881 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.120373 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:37.142271 29955 consensus_peers.cc:597] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:37.122007 30592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.122622 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.137504 30349 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:37.127187 29838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:37.094590 30722 raft_consensus.cc:2749] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.149152 30725 raft_consensus.cc:493] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.127661 29986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.149767 30725 raft_consensus.cc:515] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.150890 30747 raft_consensus.cc:2804] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:37.151075 30720 raft_consensus.cc:493] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.151399 30747 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.151749 30720 raft_consensus.cc:515] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.151994 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.152731 29909 raft_consensus.cc:2468] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 1.
I20260709 13:10:37.151861 30747 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.153863 30720 leader_election.cc:290] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.148475 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.148414 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.094863 30511 raft_consensus.cc:2804] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:37.157155 30511 raft_consensus.cc:493] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.157585 30511 raft_consensus.cc:3060] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.158468 29985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86eb4af5624466bb2d9395a36632d71" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.161321 30549 raft_consensus.cc:493] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.161689 30707 leader_election.cc:290] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.161837 30549 raft_consensus.cc:515] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.130044 30589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714cbaa854d430fb09d82c5d878cff7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.121882 30597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.130570 30743 raft_consensus.cc:493] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:37.164187 30549 leader_election.cc:290] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:37.164547 30743 raft_consensus.cc:515] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:37.154107 30598 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.166647 30743 leader_election.cc:290] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.155448 30725 leader_election.cc:290] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.162182 29881 leader_election.cc:304] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.168351 30725 raft_consensus.cc:2804] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:37.168756 30725 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.169086 30725 raft_consensus.cc:3060] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.131680 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.170202 29881 leader_election.cc:304] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.170593 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:37.176721 29807 consensus_peers.cc:597] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:37.169569 30722 raft_consensus.cc:493] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.181171 29881 leader_election.cc:304] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.195467 30722 raft_consensus.cc:515] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:37.192307 30742 raft_consensus.cc:2804] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:37.209856 30548 tablet_bootstrap.cc:492] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:37.210186 29955 leader_election.cc:304] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.221698 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:37.222514 29915 raft_consensus.cc:2393] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 75162022240a4fe1a113333e77964ca3 in current term 2: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
W20260709 13:10:37.231495 29807 leader_election.cc:343] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:37.235114 30747 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.237419 30747 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:37.228997 30733 raft_consensus.cc:493] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:37.238328 30733 raft_consensus.cc:515] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.239987 30548 tablet_bootstrap.cc:492] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:37.240499 30733 leader_election.cc:290] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.241080 30548 ts_tablet_manager.cc:1403] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.414s	user 0.020s	sys 0.006s
I20260709 13:10:37.241227 29838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:37.231148 30743 raft_consensus.cc:493] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:37.242743 30743 raft_consensus.cc:515] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:37.244814 30732 raft_consensus.cc:2749] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.258884 30743 leader_election.cc:290] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
W20260709 13:10:37.258785 29807 leader_election.cc:343] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:37.261023 29807 leader_election.cc:304] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.261785 30722 leader_election.cc:290] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:37.262144 30640 raft_consensus.cc:2749] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.229723 30742 raft_consensus.cc:493] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.264503 30742 raft_consensus.cc:3060] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.267057 30743 raft_consensus.cc:493] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:37.269163 30743 raft_consensus.cc:515] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:37.210594 30594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.250737 30733 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:37.271009 30733 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:37.255882 29955 leader_election.cc:304] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
W20260709 13:10:37.270550 29807 consensus_peers.cc:597] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:37.272863 30548 raft_consensus.cc:359] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:37.273862 30548 raft_consensus.cc:740] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:37.274753 30548 consensus_queue.cc:260] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:37.279865 30548 ts_tablet_manager.cc:1434] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 13:10:37.281359 30548 tablet_bootstrap.cc:492] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:37.286685 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.287027 29985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.289608 30733 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:37.284273 29955 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.291373 30746 raft_consensus.cc:2749] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.292841 29985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.293639 30511 raft_consensus.cc:515] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.294791 29990 raft_consensus.cc:1240] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Rejecting Update request from peer d08b1a18bb214fa1b08560d4d8149deb for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:37.296934 29985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.299228 29807 leader_election.cc:304] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
W20260709 13:10:37.303962 29863 debug-util.cc:398] Leaking SignalData structure 0x7b080026be20 after lost signal to thread 19970
W20260709 13:10:37.305285 29863 debug-util.cc:398] Leaking SignalData structure 0x7b08001a3380 after lost signal to thread 29699
W20260709 13:10:37.303097 29955 leader_election.cc:343] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:37.308265 29955 leader_election.cc:304] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.310122 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.278818 30722 raft_consensus.cc:2804] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:10:37.313004 29881 leader_election.cc:343] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:37.313611 29881 leader_election.cc:304] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.314105 29807 leader_election.cc:304] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.319619 30549 raft_consensus.cc:2749] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.322863 30512 raft_consensus.cc:2749] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.315376 30747 raft_consensus.cc:493] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.325090 30747 raft_consensus.cc:515] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.325690 30675 raft_consensus.cc:2749] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.329048 30727 raft_consensus.cc:2749] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.315928 30722 raft_consensus.cc:493] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.329895 30722 raft_consensus.cc:3060] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.323153 30743 leader_election.cc:290] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.281224 30725 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.346144 30556 consensus_queue.cc:1059] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 13:10:37.352734 29863 debug-util.cc:398] Leaking SignalData structure 0x7b08000eb940 after lost signal to thread 30625
W20260709 13:10:37.353973 30754 log.cc:927] Time spent T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb: Append to log took a long time: real 0.146s	user 0.001s	sys 0.002s
I20260709 13:10:37.361927 30511 leader_election.cc:290] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:37.338531 30747 leader_election.cc:290] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:37.369787 30722 raft_consensus.cc:515] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.373481 30639 raft_consensus.cc:3055] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:37.386965 29836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:37.388598 30720 raft_consensus.cc:2804] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:37.382367 29807 leader_election.cc:304] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:37.400148 30556 raft_consensus.cc:2749] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.401145 30720 raft_consensus.cc:493] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:37.401628 30720 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.402037 29807 leader_election.cc:304] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.402809 30530 raft_consensus.cc:2749] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.397862 30725 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:37.398499 30639 raft_consensus.cc:740] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:37.403239 29807 leader_election.cc:304] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.403983 30639 consensus_queue.cc:260] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.418602 30530 raft_consensus.cc:2749] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.419451 30639 raft_consensus.cc:3060] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.423242 29985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.412277 29989 raft_consensus.cc:2393] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74191176370841578d396cc01f230f06 in current term 2: Already voted for candidate 75162022240a4fe1a113333e77964ca3 in this term.
I20260709 13:10:37.412869 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.398793 30742 raft_consensus.cc:515] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.433202 30742 leader_election.cc:290] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:37.410826 30720 raft_consensus.cc:515] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.437577 29881 leader_election.cc:304] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.438704 30725 raft_consensus.cc:2804] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:37.438871 30720 leader_election.cc:290] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:37.439095 30725 raft_consensus.cc:493] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:37.441437 30725 raft_consensus.cc:3060] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.441534 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:37.442245 29908 raft_consensus.cc:3060] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.414113 30722 leader_election.cc:290] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:37.450548 29908 raft_consensus.cc:2468] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:37.451627 30725 raft_consensus.cc:515] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.459576 29836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:37.461043 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:37.461671 29911 raft_consensus.cc:1240] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Rejecting Update request from peer d08b1a18bb214fa1b08560d4d8149deb for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:37.461685 29916 raft_consensus.cc:3060] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.462653 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.466027 30592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.467306 30596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:37.467623 30589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:37.470005 30597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:37.471920 30599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:37.472738 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.477138 29916 raft_consensus.cc:2468] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:37.480770 30725 leader_election.cc:290] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:37.483024 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.483560 29915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac2292232bc434b88b2786ed56f7adb" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.486310 29912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388e95cfde842949825ab8dbdc03e91" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.487174 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ef88786080e40e1a8a8b623e9c30106" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.487350 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.479095 30548 tablet_bootstrap.cc:492] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:37.488859 29913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.489504 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.491115 29989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac2292232bc434b88b2786ed56f7adb" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.491643 29985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.492149 29986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ef88786080e40e1a8a8b623e9c30106" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.493391 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.559851 29842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.504302 29836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.493875 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.494987 29988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.529621 29837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388e95cfde842949825ab8dbdc03e91" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.506081 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.440028 29881 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.500567 29987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4a836f7bfb414eb5bba33a3be51d4f" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.607081 29981 raft_consensus.cc:1240] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Rejecting Update request from peer d08b1a18bb214fa1b08560d4d8149deb for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:37.608114 29881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39603, user_credentials={real_user=slave}} blocked reactor thread for 168154us
I20260709 13:10:37.575670 29916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.608845 29881 leader_election.cc:304] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.609782 30742 raft_consensus.cc:2749] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.530215 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7968a480124ac9ad9734c6ffc0dbbb" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.541287 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.612612 30548 tablet_bootstrap.cc:492] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:37.622790 30548 ts_tablet_manager.cc:1403] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.342s	user 0.020s	sys 0.003s
I20260709 13:10:37.507668 29838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.542049 29833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.624277 29955 leader_election.cc:304] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.517020 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5481f26841ab45efa35ded9872e310ee" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.625099 30746 raft_consensus.cc:2749] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.626579 29838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cfa1a88ac2b4cacb00c00d882d4cf1b" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.628032 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252766f9503842f48be95fdd3e3035c2" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.629168 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:37.630686 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.632274 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90385323444f41d096dd3842e8b2bd6d" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:37.632650 30548 raft_consensus.cc:359] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:37.502123 29984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.503197 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e73d2d9e7449d4b87ea5ee4324c2c9" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:37.616070 30544 leader_election.cc:304] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.617285 30725 raft_consensus.cc:2749] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.636756 29807 leader_election.cc:304] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.633438 30548 raft_consensus.cc:740] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:37.637120 29838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:37.660954 30639 raft_consensus.cc:2749] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.661150 30743 raft_consensus.cc:2749] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.636642 29840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:37.574805 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4a836f7bfb414eb5bba33a3be51d4f" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:37.633404 29955 leader_election.cc:304] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.663719 29955 leader_election.cc:304] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.663940 30722 raft_consensus.cc:2804] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:37.629424 29833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02b4b0aee0c426ba054d6eac3eda348" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
W20260709 13:10:37.662271 30544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.194:39603, user_credentials={real_user=slave}} blocked reactor thread for 46428us
I20260709 13:10:37.664688 30548 consensus_queue.cc:260] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:37.665796 30717 raft_consensus.cc:2804] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:37.666325 30717 raft_consensus.cc:697] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:37.667301 30722 raft_consensus.cc:697] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:37.668185 30722 consensus_queue.cc:237] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.668820 30544 leader_election.cc:304] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.669656 30743 raft_consensus.cc:2749] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.670832 30544 leader_election.cc:304] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.670598 30717 consensus_queue.cc:237] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.671744 30743 raft_consensus.cc:2749] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.671977 30548 ts_tablet_manager.cc:1434] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.001s
I20260709 13:10:37.672899 30548 tablet_bootstrap.cc:492] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:37.673367 30544 leader_election.cc:304] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.674171 30743 raft_consensus.cc:2749] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.674621 30544 leader_election.cc:304] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.675477 30743 raft_consensus.cc:2749] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.675918 30544 leader_election.cc:304] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.676806 30743 raft_consensus.cc:2749] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:37.618223 29881 leader_election.cc:343] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:37.683039 29881 leader_election.cc:304] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.684623 29807 leader_election.cc:304] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.687485 30725 raft_consensus.cc:2749] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.684430 30530 consensus_queue.cc:1059] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:37.702910 29881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.193:35599, user_credentials={real_user=slave}} blocked reactor thread for 84710.6us
I20260709 13:10:37.703764 29881 leader_election.cc:304] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.705404 30725 raft_consensus.cc:2749] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.706984 30556 raft_consensus.cc:2749] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.708174 30530 raft_consensus.cc:3055] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:37.708595 30530 raft_consensus.cc:740] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:37.708385 30762 consensus_queue.cc:1059] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:37.709440 30530 consensus_queue.cc:260] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.710449 30762 raft_consensus.cc:3055] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:37.710734 30530 raft_consensus.cc:3060] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.710886 30762 raft_consensus.cc:740] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:37.711148 30544 leader_election.cc:304] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.711782 30762 consensus_queue.cc:260] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.712603 30544 leader_election.cc:304] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.713011 30762 raft_consensus.cc:3060] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:37.716521 29808 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:37.717304 30721 raft_consensus.cc:2749] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.718128 30721 raft_consensus.cc:2749] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.721570 29910 raft_consensus.cc:1240] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Rejecting Update request from peer d08b1a18bb214fa1b08560d4d8149deb for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:37.725003 29881 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.725384 30544 leader_election.cc:304] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.726902 30743 raft_consensus.cc:2749] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.728258 30544 leader_election.cc:304] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.730219 30743 raft_consensus.cc:2749] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.730549 30742 raft_consensus.cc:2749] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.730726 30544 leader_election.cc:304] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.732115 30743 raft_consensus.cc:2749] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.732188 30544 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.733145 30743 raft_consensus.cc:2749] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.733599 30544 leader_election.cc:304] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.736258 30743 raft_consensus.cc:2749] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.737270 29910 raft_consensus.cc:1240] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Rejecting Update request from peer d08b1a18bb214fa1b08560d4d8149deb for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:37.743917 29910 raft_consensus.cc:1240] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Rejecting Update request from peer d08b1a18bb214fa1b08560d4d8149deb for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:37.746637 29910 raft_consensus.cc:1240] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Rejecting Update request from peer d08b1a18bb214fa1b08560d4d8149deb for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:37.751488 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:37.752120 29664 catalog_manager.cc:5697] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 1 to 2, leader changed from d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194) to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 2 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:37.754320 29664 catalog_manager.cc:5697] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 1 to 2, leader changed from d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194) to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 2 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:37.734935 30544 leader_election.cc:304] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265; no voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3
I20260709 13:10:37.786357 30743 raft_consensus.cc:2749] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:37.787472 30544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 52637.3us
I20260709 13:10:37.790836 30548 tablet_bootstrap.cc:492] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:37.797622 30701 consensus_queue.cc:1059] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:37.798717 30701 raft_consensus.cc:3055] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:37.799281 30595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:37.799089 30701 raft_consensus.cc:740] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:37.811766 30762 consensus_queue.cc:1059] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:37.812928 30639 consensus_queue.cc:1059] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:37.813915 30556 raft_consensus.cc:3055] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:37.814280 30556 raft_consensus.cc:740] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:37.815048 30556 consensus_queue.cc:260] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.821158 30548 tablet_bootstrap.cc:492] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:37.822016 30548 ts_tablet_manager.cc:1403] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.149s	user 0.022s	sys 0.000s
I20260709 13:10:37.816501 29881 leader_election.cc:304] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.826721 29881 leader_election.cc:304] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.827685 30548 raft_consensus.cc:359] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.829761 29839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:37.830540 29881 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.829011 29907 raft_consensus.cc:1240] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Rejecting Update request from peer d08b1a18bb214fa1b08560d4d8149deb for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:37.832329 29881 leader_election.cc:304] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.833935 30720 raft_consensus.cc:2749] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.828425 30548 raft_consensus.cc:740] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:37.825681 30556 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.829078 30725 raft_consensus.cc:2749] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.836762 29881 leader_election.cc:304] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.827234 30742 raft_consensus.cc:2749] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.834645 30549 raft_consensus.cc:2749] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:37.840586 30548 consensus_queue.cc:260] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.847071 29881 leader_election.cc:304] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.848026 30147 raft_consensus.cc:2749] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.848451 29881 leader_election.cc:304] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.849401 30147 raft_consensus.cc:2749] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.849700 29881 leader_election.cc:304] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.834374 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:37.849473 30548 ts_tablet_manager.cc:1434] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:10:37.845934 30720 raft_consensus.cc:2749] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.855639 30548 tablet_bootstrap.cc:492] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:37.853706 30747 raft_consensus.cc:2749] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.831897 30701 consensus_queue.cc:260] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.862176 30639 raft_consensus.cc:3055] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:37.872118 30639 raft_consensus.cc:740] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:37.871253 30701 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.889070 29881 leader_election.cc:304] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:37.919490 30726 consensus_queue.cc:1059] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:37.920745 30699 raft_consensus.cc:3055] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:37.921259 30699 raft_consensus.cc:740] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:37.922698 30699 consensus_queue.cc:260] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.912606 30639 consensus_queue.cc:260] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:37.899348 30720 raft_consensus.cc:2749] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:37.951599 30639 raft_consensus.cc:3060] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.962529 29911 raft_consensus.cc:1240] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Rejecting Update request from peer 75162022240a4fe1a113333e77964ca3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:37.965405 30759 consensus_queue.cc:1059] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.966398 30759 raft_consensus.cc:3055] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:37.966749 30759 raft_consensus.cc:740] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:37.967731 30759 consensus_queue.cc:260] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:37.968504 29834 raft_consensus.cc:2468] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 3.
I20260709 13:10:37.969070 30759 raft_consensus.cc:3060] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.951853 30699 raft_consensus.cc:3060] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.971652 29881 leader_election.cc:304] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:37.974335 30720 raft_consensus.cc:2804] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:37.975060 30720 raft_consensus.cc:697] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 3 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:37.983889 30720 consensus_queue.cc:237] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:38.016317 30548 tablet_bootstrap.cc:492] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:38.017258 30548 tablet_bootstrap.cc:492] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:38.018041 30548 ts_tablet_manager.cc:1403] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.163s	user 0.021s	sys 0.007s
I20260709 13:10:38.020696 30548 raft_consensus.cc:359] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:38.021458 30548 raft_consensus.cc:740] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:38.022279 30548 consensus_queue.cc:260] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:38.087962 30548 ts_tablet_manager.cc:1434] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.069s	user 0.010s	sys 0.001s
I20260709 13:10:38.090845 30548 tablet_bootstrap.cc:492] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:38.134640 29661 catalog_manager.cc:5697] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 2 to 3, leader changed from d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194) to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:38.177597 29834 raft_consensus.cc:1275] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:38.183728 29834 raft_consensus.cc:1275] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:38.226208 30717 consensus_queue.cc:1048] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:38.228458 30717 consensus_queue.cc:1048] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:38.288396 29910 raft_consensus.cc:1275] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:38.346305 30548 tablet_bootstrap.cc:492] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:38.355567 30548 tablet_bootstrap.cc:492] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:38.352303 30735 consensus_queue.cc:1048] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:10:38.402494 30548 ts_tablet_manager.cc:1403] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.313s	user 0.034s	sys 0.073s
I20260709 13:10:38.416637 30548 raft_consensus.cc:359] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:38.417582 30548 raft_consensus.cc:740] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:38.418490 30548 consensus_queue.cc:260] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:38.428205 30548 ts_tablet_manager.cc:1434] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.001s
I20260709 13:10:38.429401 30548 tablet_bootstrap.cc:492] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:38.437812 29910 raft_consensus.cc:1275] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:38.483206 29833 raft_consensus.cc:1275] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:38.487811 30742 consensus_queue.cc:1048] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:38.489414 30717 consensus_queue.cc:1048] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:38.593420 30592 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.670269 30548 tablet_bootstrap.cc:492] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:38.672372 30548 tablet_bootstrap.cc:492] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:38.676755 30548 ts_tablet_manager.cc:1403] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.248s	user 0.014s	sys 0.015s
I20260709 13:10:38.682161 30548 raft_consensus.cc:359] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:38.684504 30548 raft_consensus.cc:740] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:38.688616 30548 consensus_queue.cc:260] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:38.692137 30548 ts_tablet_manager.cc:1434] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 13:10:38.693197 30548 tablet_bootstrap.cc:492] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:38.697896 30592 raft_consensus.cc:1275] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:38.700788 30742 consensus_queue.cc:1048] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:38.786926 30548 tablet_bootstrap.cc:492] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:38.787949 30548 tablet_bootstrap.cc:492] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:38.788578 30548 ts_tablet_manager.cc:1403] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.002s
I20260709 13:10:38.791229 30548 raft_consensus.cc:359] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:38.800738 30548 raft_consensus.cc:740] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:38.801517 30548 consensus_queue.cc:260] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:38.809675 30548 ts_tablet_manager.cc:1434] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:10:38.811451 30548 tablet_bootstrap.cc:492] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:38.819679 30717 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.820403 30717 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:38.824051 30717 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:38.829234 30548 tablet_bootstrap.cc:492] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:38.831261 30548 tablet_bootstrap.cc:492] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:38.833076 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:38.832916 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:38.833961 29910 raft_consensus.cc:2468] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:38.834156 30548 ts_tablet_manager.cc:1403] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.004s
I20260709 13:10:38.833961 29834 raft_consensus.cc:2468] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 2.
I20260709 13:10:38.837313 30548 raft_consensus.cc:359] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:38.838061 30548 raft_consensus.cc:740] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:38.838933 30548 consensus_queue.cc:260] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:38.841250 30548 ts_tablet_manager.cc:1434] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:38.843019 30548 tablet_bootstrap.cc:492] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:38.842726 29955 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:38.845470 30759 raft_consensus.cc:2804] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:38.845914 30759 raft_consensus.cc:493] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.846309 30759 raft_consensus.cc:3060] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.858873 30759 raft_consensus.cc:515] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:38.861815 30759 leader_election.cc:290] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:38.863253 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:38.868269 29834 raft_consensus.cc:3060] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.868506 30548 tablet_bootstrap.cc:492] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:38.869329 30548 tablet_bootstrap.cc:492] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:38.869801 30548 ts_tablet_manager.cc:1403] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.003s
I20260709 13:10:38.866060 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9fb82d905c441bb50d70e1ce080213" candidate_uuid: "75162022240a4fe1a113333e77964ca3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:38.871605 29910 raft_consensus.cc:3060] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.872229 30548 raft_consensus.cc:359] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:38.872974 30548 raft_consensus.cc:740] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:38.873694 30548 consensus_queue.cc:260] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
W20260709 13:10:38.867515 29807 consensus_peers.cc:597] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:38.876145 30548 ts_tablet_manager.cc:1434] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:10:38.878015 29834 raft_consensus.cc:2468] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 3.
I20260709 13:10:38.882947 29955 leader_election.cc:304] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:38.883813 30759 raft_consensus.cc:2804] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:38.885999 30759 raft_consensus.cc:697] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 3 LEADER]: Becoming Leader. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:38.884413 30548 tablet_bootstrap.cc:492] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:38.901513 30759 consensus_queue.cc:237] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:38.901600 29910 raft_consensus.cc:2468] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75162022240a4fe1a113333e77964ca3 in term 3.
W20260709 13:10:38.916659 29955 consensus_peers.cc:597] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:38.924656 30548 tablet_bootstrap.cc:492] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:38.925783 30548 tablet_bootstrap.cc:492] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:38.926458 30548 ts_tablet_manager.cc:1403] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.042s	user 0.021s	sys 0.009s
I20260709 13:10:38.929224 30548 raft_consensus.cc:359] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:38.929920 30548 raft_consensus.cc:740] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:38.929569 29661 catalog_manager.cc:5697] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 reported cstate change: term changed from 1 to 3, leader changed from d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194) to 75162022240a4fe1a113333e77964ca3 (127.19.127.196). New cstate: current_term: 3 leader_uuid: "75162022240a4fe1a113333e77964ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:38.930629 30548 consensus_queue.cc:260] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:38.933018 30548 ts_tablet_manager.cc:1434] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:38.936729 30548 tablet_bootstrap.cc:492] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:38.969640 30548 tablet_bootstrap.cc:492] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:38.970620 30548 tablet_bootstrap.cc:492] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:38.971271 30548 ts_tablet_manager.cc:1403] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.003s
I20260709 13:10:38.973963 30548 raft_consensus.cc:359] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:38.974669 30548 raft_consensus.cc:740] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:38.982545 30548 consensus_queue.cc:260] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:38.991052 30548 ts_tablet_manager.cc:1434] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.003s
I20260709 13:10:38.992046 30548 tablet_bootstrap.cc:492] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap starting.
I20260709 13:10:39.031062 30548 tablet_bootstrap.cc:492] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:39.031870 30548 tablet_bootstrap.cc:492] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: Bootstrap complete.
I20260709 13:10:39.032440 30548 ts_tablet_manager.cc:1403] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.005s
I20260709 13:10:39.035061 30548 raft_consensus.cc:359] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:39.035749 30548 raft_consensus.cc:740] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Initialized, Role: FOLLOWER
I20260709 13:10:39.036662 30548 consensus_queue.cc:260] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:39.039119 30548 ts_tablet_manager.cc:1434] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:10:39.230397 30743 raft_consensus.cc:493] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:39.231048 30743 raft_consensus.cc:515] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.233224 30743 leader_election.cc:290] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:39.234098 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:39.234401 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:39.234892 29834 raft_consensus.cc:2468] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:39.235222 29907 raft_consensus.cc:2393] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 2: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:39.236320 30544 leader_election.cc:304] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:39.237108 30743 raft_consensus.cc:2804] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:39.237442 30743 raft_consensus.cc:493] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:39.237753 30743 raft_consensus.cc:3060] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:39.245684 30743 raft_consensus.cc:515] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.248167 30743 leader_election.cc:290] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:39.249498 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:39.251262 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1207ccf2285f42dc901a1db9e48e39f3" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:39.252148 29907 raft_consensus.cc:2393] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 2: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:39.259334 30701 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.259958 30701 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.261642 29834 raft_consensus.cc:2468] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:39.262643 30701 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:39.263330 30544 leader_election.cc:304] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06
I20260709 13:10:39.264122 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:39.264701 30800 raft_consensus.cc:2804] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:39.264230 30589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:39.265139 29907 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 3.
I20260709 13:10:39.265357 30800 raft_consensus.cc:697] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:39.265578 30589 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:39.266705 30800 consensus_queue.cc:237] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.267680 29807 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:39.271762 30639 raft_consensus.cc:2804] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:39.272153 30639 raft_consensus.cc:493] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.272473 30639 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:39.282814 30639 raft_consensus.cc:515] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.285151 30639 leader_election.cc:290] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 4 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:39.286942 30589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:39.287737 30589 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:10:39.291755 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ff57a4871b4c2197e7c7f41230d429" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:39.292577 29907 raft_consensus.cc:3060] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:39.300206 30589 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 4.
I20260709 13:10:39.305630 29807 leader_election.cc:304] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:39.313683 30701 raft_consensus.cc:2804] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:39.313898 29907 raft_consensus.cc:2468] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 4.
I20260709 13:10:39.314302 30701 raft_consensus.cc:697] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 4 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:39.316143 29665 catalog_manager.cc:5697] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 1 to 2, leader changed from d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.325671 30701 consensus_queue.cc:237] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.357751 29663 catalog_manager.cc:5697] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.388412 30701 raft_consensus.cc:493] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:39.388976 30701 raft_consensus.cc:515] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:39.391633 30701 leader_election.cc:290] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:39.392344 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:39.392680 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3" is_pre_election: true
I20260709 13:10:39.393281 29907 raft_consensus.cc:2393] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 2: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:39.393488 29990 raft_consensus.cc:2468] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:39.395134 29807 leader_election.cc:304] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06
I20260709 13:10:39.396263 30701 raft_consensus.cc:2804] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:39.396669 30701 raft_consensus.cc:493] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75162022240a4fe1a113333e77964ca3)
I20260709 13:10:39.397105 30701 raft_consensus.cc:3060] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:39.401127 30802 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.401991 30802 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.404040 30802 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:39.404824 30701 raft_consensus.cc:515] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:39.413983 29907 raft_consensus.cc:1275] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:10:39.418006 30701 leader_election.cc:290] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:39.414278 30592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265" is_pre_election: true
I20260709 13:10:39.415207 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:39.420143 29910 raft_consensus.cc:2468] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:39.423032 30759 consensus_queue.cc:1048] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:39.421387 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:39.426774 29807 leader_election.cc:304] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:39.430583 30800 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:39.419073 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a52b7d43c054ac485deaf19144dced6" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:39.439013 29907 raft_consensus.cc:2393] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d08b1a18bb214fa1b08560d4d8149deb in current term 2: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:39.440320 30800 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.443080 30800 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:39.444031 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb" is_pre_election: true
I20260709 13:10:39.445067 29834 raft_consensus.cc:2468] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:39.448215 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06" is_pre_election: true
I20260709 13:10:39.449551 29907 raft_consensus.cc:2393] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 in current term 2: Already voted for candidate 74191176370841578d396cc01f230f06 in this term.
I20260709 13:10:39.449820 30701 raft_consensus.cc:2804] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:39.450331 30701 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.450675 30701 raft_consensus.cc:3060] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:39.451378 30544 leader_election.cc:304] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:39.453925 30800 raft_consensus.cc:2804] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:39.454919 30800 raft_consensus.cc:493] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d08b1a18bb214fa1b08560d4d8149deb)
I20260709 13:10:39.455509 30800 raft_consensus.cc:3060] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:39.472635 30701 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.474740 29834 raft_consensus.cc:1275] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Refusing update from remote peer 75162022240a4fe1a113333e77964ca3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:10:39.476434 30759 consensus_queue.cc:1048] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:39.476542 30592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
I20260709 13:10:39.477604 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:39.478374 29907 raft_consensus.cc:3060] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:39.476435 30701 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 election: Requested vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:39.509807 29990 raft_consensus.cc:2468] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:39.519083 29807 leader_election.cc:304] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75162022240a4fe1a113333e77964ca3, d08b1a18bb214fa1b08560d4d8149deb; no voters: 74191176370841578d396cc01f230f06
I20260709 13:10:39.521409 30701 raft_consensus.cc:2804] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:39.523772 30701 raft_consensus.cc:697] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:39.524685 30701 consensus_queue.cc:237] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:39.528278 30800 raft_consensus.cc:515] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.529817 30592 raft_consensus.cc:3060] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:39.531809 29837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:39.532663 29837 raft_consensus.cc:2368] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 for earlier term 2. Current term is 3.
I20260709 13:10:39.533385 29907 raft_consensus.cc:2468] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 3.
I20260709 13:10:39.534003 30544 leader_election.cc:400] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Vote denied by peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603) with higher term. Message: Invalid argument: T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 for earlier term 2. Current term is 3.
I20260709 13:10:39.534358 30544 leader_election.cc:403] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:10:39.534287 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1697952144bae8ddfeab6f69d13f4" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:39.535120 29911 raft_consensus.cc:2368] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 for earlier term 2. Current term is 3.
I20260709 13:10:39.539096 30544 leader_election.cc:400] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Vote denied by peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257) with higher term. Message: Invalid argument: T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 for earlier term 2. Current term is 3.
I20260709 13:10:39.544605 29807 leader_election.cc:304] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:39.538442 30800 leader_election.cc:290] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:39.549116 30802 raft_consensus.cc:2804] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:39.556113 30802 raft_consensus.cc:697] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 3 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:39.557123 30802 consensus_queue.cc:237] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:39.561906 30592 raft_consensus.cc:2468] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 3.
I20260709 13:10:39.563809 30743 raft_consensus.cc:2749] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603) with higher term. Message: Invalid argument: T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ba28923c4534c78903f759f0a4e1265 for earlier term 2. Current term is 3.
I20260709 13:10:39.581244 29665 catalog_manager.cc:5697] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 2 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.607430 29665 catalog_manager.cc:5697] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.654352 29833 raft_consensus.cc:1275] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:39.655906 30743 consensus_queue.cc:1048] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:39.682964 29910 raft_consensus.cc:1275] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:39.693537 30800 consensus_queue.cc:1048] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:39.742871 30592 raft_consensus.cc:1275] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 4 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:39.744601 30639 consensus_queue.cc:1048] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:39.775719 29910 raft_consensus.cc:1275] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 4 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:39.779379 30802 consensus_queue.cc:1048] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:39.913019 29910 raft_consensus.cc:1275] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:39.915460 30639 consensus_queue.cc:1048] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:39.934271 29990 raft_consensus.cc:1275] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:39.937057 30701 consensus_queue.cc:1048] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.028872 30592 raft_consensus.cc:1275] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:10:40.031476 30802 consensus_queue.cc:1048] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:40.047895 29910 raft_consensus.cc:1275] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:10:40.052510 30639 consensus_queue.cc:1048] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:44.233546 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:44.310259 29983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e543a21ae8243058d010255b40860e9"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:44.311877 29983 raft_consensus.cc:606] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Received request to transfer leadership to TS 74191176370841578d396cc01f230f06
I20260709 13:10:44.334847 29833 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef"
dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:47686
I20260709 13:10:44.335461 29833 raft_consensus.cc:606] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:44.346481 29983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:44.347201 29983 raft_consensus.cc:606] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Received request to transfer leadership to TS 74191176370841578d396cc01f230f06
I20260709 13:10:44.352535 29990 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:44.353232 29990 raft_consensus.cc:606] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:44.360713 30836 raft_consensus.cc:993] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:44.361065 29983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:44.361294 30836 raft_consensus.cc:1081] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:44.361773 29983 raft_consensus.cc:606] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Received request to transfer leadership to TS 74191176370841578d396cc01f230f06
I20260709 13:10:44.365173 30589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44744
I20260709 13:10:44.365927 30589 raft_consensus.cc:493] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.367007 30589 raft_consensus.cc:3060] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.365959 29990 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:44.368409 29990 raft_consensus.cc:606] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:44.373389 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:10:44.375155 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:44.377543 30589 raft_consensus.cc:515] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:44.380125 30589 leader_election.cc:290] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:44.380247 29833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:44.380949 29833 raft_consensus.cc:3055] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:44.381237 29833 raft_consensus.cc:740] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:44.382321 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21581ff708354cb0ad0ed6ae935d4eef" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:44.381984 29833 consensus_queue.cc:260] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:44.382875 29911 raft_consensus.cc:3060] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.383522 29833 raft_consensus.cc:3060] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.393132 29911 raft_consensus.cc:2468] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:44.394351 30544 leader_election.cc:304] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:44.394510 29833 raft_consensus.cc:2468] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:44.395303 30828 raft_consensus.cc:2804] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:44.396742 30828 raft_consensus.cc:697] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:44.397529 30828 consensus_queue.cc:237] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:44.410297 29661 catalog_manager.cc:5697] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 1 to 2, leader changed from d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:44.436151 30830 raft_consensus.cc:993] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:44.436692 30830 raft_consensus.cc:1081] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:44.438606 30589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:10:44.439168 30589 raft_consensus.cc:493] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.439594 30589 raft_consensus.cc:3060] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.444224 30589 raft_consensus.cc:515] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:44.445994 30589 leader_election.cc:290] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:44.447238 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:44.447587 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4c252a57bc4facbe5d2e0ceb4c80ae" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:44.448110 29983 raft_consensus.cc:3055] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:44.448174 29911 raft_consensus.cc:3060] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.448658 29983 raft_consensus.cc:740] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:44.449695 29983 consensus_queue.cc:260] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:44.452847 29983 raft_consensus.cc:3060] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.457669 29911 raft_consensus.cc:2468] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:44.459894 30544 leader_election.cc:304] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:44.460742 30828 raft_consensus.cc:2804] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:44.461210 30828 raft_consensus.cc:697] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:44.462020 30828 consensus_queue.cc:237] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:44.471662 30825 raft_consensus.cc:993] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:44.472236 30825 raft_consensus.cc:1081] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:44.485684 29983 raft_consensus.cc:2468] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:44.506394 29661 catalog_manager.cc:5697] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:44.508781 30598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:10:44.509405 30598 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.509784 30598 raft_consensus.cc:3060] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.517787 30598 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:44.520570 30598 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:44.521206 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:44.521844 29911 raft_consensus.cc:3060] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.522240 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac962e63d404b2a89aa0afd301ce161" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:44.522887 29983 raft_consensus.cc:3055] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:44.523175 29983 raft_consensus.cc:740] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:44.524063 29983 consensus_queue.cc:260] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:44.525537 29983 raft_consensus.cc:3060] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.529206 29911 raft_consensus.cc:2468] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 3.
I20260709 13:10:44.530476 30544 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:44.531479 30828 raft_consensus.cc:2804] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:44.532153 30828 raft_consensus.cc:697] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 3 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:44.533306 30828 consensus_queue.cc:237] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:44.563171 29983 raft_consensus.cc:2468] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 3.
I20260709 13:10:44.572917 30825 raft_consensus.cc:993] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3: : Instructing follower 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:44.573446 30825 raft_consensus.cc:1081] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Signalling peer 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:44.578277 29911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3"
dest_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:60766
I20260709 13:10:44.578881 29911 raft_consensus.cc:493] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.579211 29911 raft_consensus.cc:3060] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.580574 30825 raft_consensus.cc:993] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3: : Instructing follower 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:44.581049 30825 raft_consensus.cc:1081] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Signalling peer 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:44.583034 29910 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0"
dest_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:60766
I20260709 13:10:44.583660 29910 raft_consensus.cc:493] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.584008 29910 raft_consensus.cc:3060] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.591795 29911 raft_consensus.cc:515] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:44.591581 29910 raft_consensus.cc:515] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:44.597023 29911 leader_election.cc:290] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:44.598050 29910 leader_election.cc:290] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:44.601876 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:44.602252 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:44.602646 29983 raft_consensus.cc:3055] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:44.602875 29990 raft_consensus.cc:3055] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:44.603015 29983 raft_consensus.cc:740] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:44.603227 29990 raft_consensus.cc:740] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:44.604067 29990 consensus_queue.cc:260] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:44.605584 29990 raft_consensus.cc:3060] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.604067 29983 consensus_queue.cc:260] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:44.618438 29990 raft_consensus.cc:2468] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 3.
I20260709 13:10:44.772840 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d182f1fe9db6430d8ff9a0c29b11c3f3" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:44.772938 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7255b6d0c90146d7bb85a327da87cec0" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:44.773561 29841 raft_consensus.cc:3060] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.773576 29834 raft_consensus.cc:3060] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:44.785745 29841 raft_consensus.cc:2468] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:44.795395 29834 raft_consensus.cc:2468] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 3.
I20260709 13:10:44.835453 29661 catalog_manager.cc:5697] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 2 to 3, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 3 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:44.838593 29911 raft_consensus.cc:1275] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:44.840665 29834 raft_consensus.cc:1275] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:44.843207 30837 consensus_queue.cc:1048] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:44.844941 30837 consensus_queue.cc:1048] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:44.923202 29914 raft_consensus.cc:1275] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:10:44.925652 30828 consensus_queue.cc:1048] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:44.976581 29983 raft_consensus.cc:3060] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.997457 29907 raft_consensus.cc:1275] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:45.015522 30828 consensus_queue.cc:1048] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:45.083418 29990 raft_consensus.cc:1275] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:10:45.088861 30843 raft_consensus.cc:993] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3: : Instructing follower 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:45.089367 30843 raft_consensus.cc:1081] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Signalling peer 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:45.078907 29881 leader_election.cc:304] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:45.097999 30863 raft_consensus.cc:2804] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:45.098555 30863 raft_consensus.cc:697] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 3 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:45.099570 29911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e543a21ae8243058d010255b40860e9"
dest_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:60766
I20260709 13:10:45.099469 30863 consensus_queue.cc:237] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:45.100173 29911 raft_consensus.cc:493] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:45.101609 29911 raft_consensus.cc:3060] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:45.105316 30544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:40257, user_credentials={real_user=slave}} blocked reactor thread for 61273.1us
I20260709 13:10:45.110307 29911 raft_consensus.cc:515] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:45.119596 29911 leader_election.cc:290] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:45.112378 29986 raft_consensus.cc:1275] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:45.099875 29983 raft_consensus.cc:2468] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:45.115820 30828 consensus_queue.cc:1048] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:45.129451 29881 leader_election.cc:304] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:45.132752 30824 raft_consensus.cc:2804] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:45.137002 30824 raft_consensus.cc:697] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:45.137874 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543a21ae8243058d010255b40860e9" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:45.138594 29834 raft_consensus.cc:3060] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:45.138871 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543a21ae8243058d010255b40860e9" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:45.139542 29983 raft_consensus.cc:3055] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:45.140008 29983 raft_consensus.cc:740] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:45.144953 30824 consensus_queue.cc:237] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:45.140957 29983 consensus_queue.cc:260] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:45.158905 29983 raft_consensus.cc:3060] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:45.171425 29834 raft_consensus.cc:2468] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:45.176352 30828 consensus_queue.cc:1048] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:45.176846 29983 raft_consensus.cc:2468] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:45.181138 29881 leader_election.cc:304] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:45.183270 30863 raft_consensus.cc:2804] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:45.183854 30863 raft_consensus.cc:697] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:45.184796 30863 consensus_queue.cc:237] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:45.188357 29663 catalog_manager.cc:5697] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 2 to 3, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:45.191954 29663 catalog_manager.cc:5697] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:45.225440 29661 catalog_manager.cc:5697] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:45.422698 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:45.463409 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:45.515731 29907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "125fc2e410394efba24979999a6785b7"
dest_uuid: "74191176370841578d396cc01f230f06"
mode: GRACEFUL
new_leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
 from {username='slave'} at 127.0.0.1:50186
I20260709 13:10:45.516319 29907 raft_consensus.cc:606] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Received request to transfer leadership to TS d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:45.519462 29986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:45.520112 29986 raft_consensus.cc:606] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:45.521303 29990 raft_consensus.cc:1275] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:45.522712 30867 consensus_queue.cc:1048] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:45.523571 29834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de"
dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:47686
I20260709 13:10:45.524188 29834 raft_consensus.cc:606] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:45.527695 29986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:45.528342 29986 raft_consensus.cc:606] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:45.532928 29907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51"
dest_uuid: "74191176370841578d396cc01f230f06"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:50186
I20260709 13:10:45.533564 29907 raft_consensus.cc:606] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:45.537122 29841 raft_consensus.cc:1275] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:45.537246 29986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:45.537968 29986 raft_consensus.cc:606] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:45.539212 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:10:45.540959 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:45.541635 30864 consensus_queue.cc:1048] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:45.605788 30851 raft_consensus.cc:993] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:45.606267 30863 raft_consensus.cc:1081] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:45.607800 30598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44728
I20260709 13:10:45.608420 30598 raft_consensus.cc:493] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:45.608711 30598 raft_consensus.cc:3060] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:45.613466 30598 raft_consensus.cc:515] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:45.615178 30598 leader_election.cc:290] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:45.616341 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:45.616341 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fecd61828184c7e9cba33adb56ccd51" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:45.617286 29907 raft_consensus.cc:3055] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:45.617307 29834 raft_consensus.cc:3060] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:45.617707 29907 raft_consensus.cc:740] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:45.619601 29907 consensus_queue.cc:260] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:45.621047 29907 raft_consensus.cc:3060] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:45.629758 29907 raft_consensus.cc:2468] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 3.
I20260709 13:10:45.631074 30544 leader_election.cc:304] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:45.632026 29834 raft_consensus.cc:2468] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 3.
I20260709 13:10:45.632236 30828 raft_consensus.cc:2804] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:45.632889 30828 raft_consensus.cc:697] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 3 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:45.633682 30828 consensus_queue.cc:237] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:45.635483 30850 raft_consensus.cc:993] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
W20260709 13:10:45.635607 29863 debug-util.cc:398] Leaking SignalData structure 0x7b08000b6be0 after lost signal to thread 19970
I20260709 13:10:45.636245 30825 raft_consensus.cc:1081] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
W20260709 13:10:45.636965 29863 debug-util.cc:398] Leaking SignalData structure 0x7b0800249820 after lost signal to thread 29699
W20260709 13:10:45.638553 29863 debug-util.cc:398] Leaking SignalData structure 0x7b08002d10a0 after lost signal to thread 29866
W20260709 13:10:45.640374 29863 debug-util.cc:398] Leaking SignalData structure 0x7b0800198660 after lost signal to thread 30014
W20260709 13:10:45.641666 29863 debug-util.cc:398] Leaking SignalData structure 0x7b0800345dc0 after lost signal to thread 30625
I20260709 13:10:45.642042 30598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:10:45.643039 30598 raft_consensus.cc:493] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:45.643411 30598 raft_consensus.cc:3060] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:45.643813 29834 raft_consensus.cc:1275] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:45.644585 29665 catalog_manager.cc:5697] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 2 to 3, leader changed from 74191176370841578d396cc01f230f06 (127.19.127.195) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 3 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:45.645532 30863 consensus_queue.cc:1048] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:45.661998 29986 raft_consensus.cc:1275] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:45.660051 30598 raft_consensus.cc:515] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:45.663885 30863 consensus_queue.cc:1048] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:45.665701 30598 leader_election.cc:290] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:45.668790 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:45.669688 29990 raft_consensus.cc:3055] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:45.670065 29990 raft_consensus.cc:740] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:45.677109 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e8c425db9e4e479c00e6f131fbafe2" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:45.678158 29907 raft_consensus.cc:3060] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:45.694542 29907 raft_consensus.cc:2468] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:45.695900 30544 leader_election.cc:304] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:45.670950 29990 consensus_queue.cc:260] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:45.696878 30828 raft_consensus.cc:2804] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:45.697402 30828 raft_consensus.cc:697] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:45.697839 29990 raft_consensus.cc:3060] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:45.698369 30828 consensus_queue.cc:237] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:45.723460 29990 raft_consensus.cc:2468] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:45.737049 29665 catalog_manager.cc:5697] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:45.758215 29834 raft_consensus.cc:1275] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:45.764753 30863 consensus_queue.cc:1048] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:45.773185 30836 raft_consensus.cc:993] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:45.773703 30836 raft_consensus.cc:1081] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:45.777410 29990 raft_consensus.cc:1275] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:45.780645 30851 consensus_queue.cc:1048] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:45.782500 30598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44744
I20260709 13:10:45.783066 30598 raft_consensus.cc:493] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:45.783355 30598 raft_consensus.cc:3060] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:45.792158 30598 raft_consensus.cc:515] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:45.795122 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:45.795915 29990 raft_consensus.cc:3060] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:45.803894 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91cac27fc9443b86fc3821776294de" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:45.804606 29834 raft_consensus.cc:3055] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:45.804962 29834 raft_consensus.cc:740] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:45.805686 29834 consensus_queue.cc:260] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:45.806823 29834 raft_consensus.cc:3060] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:45.814240 30598 leader_election.cc:290] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:45.818537 29990 raft_consensus.cc:2468] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:45.819772 30544 leader_election.cc:304] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:45.820657 30828 raft_consensus.cc:2804] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:45.821161 30828 raft_consensus.cc:697] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:45.821973 30828 consensus_queue.cc:237] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:45.845976 29661 catalog_manager.cc:5697] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 1 to 2, leader changed from d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:45.863154 29834 raft_consensus.cc:2468] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:45.993185 30864 raft_consensus.cc:993] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06: : Instructing follower d08b1a18bb214fa1b08560d4d8149deb to start an election
I20260709 13:10:45.993724 30863 raft_consensus.cc:1081] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Signalling peer d08b1a18bb214fa1b08560d4d8149deb to start an election
I20260709 13:10:45.995523 29841 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "125fc2e410394efba24979999a6785b7"
dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
 from {username='slave'} at 127.0.0.1:43782
I20260709 13:10:45.996040 29841 raft_consensus.cc:493] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:45.996424 29841 raft_consensus.cc:3060] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.005409 29841 raft_consensus.cc:515] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:46.007858 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125fc2e410394efba24979999a6785b7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:46.008358 29911 raft_consensus.cc:3055] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:46.008630 29911 raft_consensus.cc:740] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:46.009281 29911 consensus_queue.cc:260] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:46.010309 29911 raft_consensus.cc:3060] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.019923 29841 leader_election.cc:290] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:46.020895 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125fc2e410394efba24979999a6785b7" candidate_uuid: "d08b1a18bb214fa1b08560d4d8149deb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:46.021950 29990 raft_consensus.cc:3060] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.020869 29911 raft_consensus.cc:2468] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:46.032934 29807 leader_election.cc:304] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:46.033600 29990 raft_consensus.cc:2468] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d08b1a18bb214fa1b08560d4d8149deb in term 2.
I20260709 13:10:46.034446 30888 raft_consensus.cc:2804] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:46.035039 30888 raft_consensus.cc:697] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Becoming Leader. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:46.036115 30888 consensus_queue.cc:237] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:46.041152 30825 raft_consensus.cc:993] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:46.041754 30889 raft_consensus.cc:1081] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:46.051697 29665 catalog_manager.cc:5697] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb reported cstate change: term changed from 1 to 2, leader changed from 74191176370841578d396cc01f230f06 (127.19.127.195) to d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194). New cstate: current_term: 2 leader_uuid: "d08b1a18bb214fa1b08560d4d8149deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.052851 30589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:10:46.053438 30589 raft_consensus.cc:493] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:46.053831 30589 raft_consensus.cc:3060] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.062093 30589 raft_consensus.cc:515] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:46.064306 30589 leader_election.cc:290] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:46.065593 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:46.065934 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caad1999ec5545d9a8eb2ade21997cab" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:46.066274 29834 raft_consensus.cc:3060] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.066461 29990 raft_consensus.cc:3055] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:46.066793 29990 raft_consensus.cc:740] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:46.069799 29990 consensus_queue.cc:260] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:46.071502 29990 raft_consensus.cc:3060] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.073014 30889 raft_consensus.cc:993] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:46.073836 30825 raft_consensus.cc:1081] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:46.075362 29834 raft_consensus.cc:2468] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:46.078771 30544 leader_election.cc:304] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:46.079488 30589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:10:46.079809 30828 raft_consensus.cc:2804] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:46.080197 30589 raft_consensus.cc:493] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:46.079826 29990 raft_consensus.cc:2468] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:46.080720 30828 raft_consensus.cc:697] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:46.080789 30589 raft_consensus.cc:3060] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.081964 30828 consensus_queue.cc:237] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:46.087498 30589 raft_consensus.cc:515] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:46.089627 30589 leader_election.cc:290] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:46.090394 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:46.090754 29990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6c48a3ec224675bd35d17f96f8fb13" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:46.091070 29911 raft_consensus.cc:3060] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.091360 29990 raft_consensus.cc:3055] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:46.091818 29990 raft_consensus.cc:740] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:46.092702 29990 consensus_queue.cc:260] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:46.093942 29990 raft_consensus.cc:3060] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.096652 29665 catalog_manager.cc:5697] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.102597 29990 raft_consensus.cc:2468] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:46.102597 29911 raft_consensus.cc:2468] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:46.105647 30544 leader_election.cc:304] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:46.106509 30859 raft_consensus.cc:2804] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:46.106997 30859 raft_consensus.cc:697] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:46.107951 30859 consensus_queue.cc:237] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:46.108904 29911 raft_consensus.cc:1275] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.112138 30828 consensus_queue.cc:1048] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:46.143548 29665 catalog_manager.cc:5697] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:46.170518 29983 raft_consensus.cc:1275] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.173386 30828 consensus_queue.cc:1048] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:46.233827 29834 raft_consensus.cc:1275] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:46.236687 29910 raft_consensus.cc:1275] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:46.239464 30861 consensus_queue.cc:1048] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:46.240233 30828 consensus_queue.cc:1048] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:46.307096 29834 raft_consensus.cc:1275] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.326524 29983 raft_consensus.cc:1275] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.329430 30861 consensus_queue.cc:1048] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:46.335626 30861 consensus_queue.cc:1048] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:46.541975 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:46.566619 29907 raft_consensus.cc:1275] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.568409 30861 consensus_queue.cc:1048] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:46.582017 29990 raft_consensus.cc:1275] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.584941 29990 raft_consensus.cc:1275] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.586315 30861 consensus_queue.cc:1048] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:46.586509 30859 consensus_queue.cc:1048] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:46.595466 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:46.611224 29907 raft_consensus.cc:1275] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.613230 30882 consensus_queue.cc:1048] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:46.636219 29834 raft_consensus.cc:1275] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.637687 30828 consensus_queue.cc:1048] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:46.644691 29983 raft_consensus.cc:1275] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer d08b1a18bb214fa1b08560d4d8149deb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:46.651719 30836 consensus_queue.cc:1048] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:46.686627 29990 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "737f2cb09b0a48b1987133f213820765"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:46.687361 29990 raft_consensus.cc:606] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Received request to transfer leadership to TS 74191176370841578d396cc01f230f06
I20260709 13:10:46.694587 30850 raft_consensus.cc:993] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3: : Instructing follower 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:46.693802 29834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd"
dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:47686
I20260709 13:10:46.695484 29834 raft_consensus.cc:606] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:46.695151 30889 raft_consensus.cc:1081] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Signalling peer 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:46.697680 29914 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "737f2cb09b0a48b1987133f213820765"
dest_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:60766
I20260709 13:10:46.698236 29914 raft_consensus.cc:493] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:46.698396 29983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:46.698577 29914 raft_consensus.cc:3060] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.698998 29983 raft_consensus.cc:606] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Received request to transfer leadership to TS 74191176370841578d396cc01f230f06
I20260709 13:10:46.701799 29983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:46.702291 29983 raft_consensus.cc:606] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:46.704787 29983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df180e07f62245aa9c117b1558657534"
dest_uuid: "75162022240a4fe1a113333e77964ca3"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:10:46.705286 29983 raft_consensus.cc:606] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:46.705330 29914 raft_consensus.cc:515] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:46.706413 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:46.707391 29914 leader_election.cc:290] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:46.707891 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f2cb09b0a48b1987133f213820765" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:46.708456 29834 raft_consensus.cc:3060] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.708639 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f2cb09b0a48b1987133f213820765" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:46.708968 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:46.709360 29983 raft_consensus.cc:3055] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:46.709748 29983 raft_consensus.cc:740] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:46.710453 29983 consensus_queue.cc:260] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:46.711747 29983 raft_consensus.cc:3060] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.719838 29983 raft_consensus.cc:2468] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:46.721935 29881 leader_election.cc:304] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, 75162022240a4fe1a113333e77964ca3; no voters: 
I20260709 13:10:46.722766 30851 raft_consensus.cc:2804] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:46.723268 30851 raft_consensus.cc:697] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:46.724149 30851 consensus_queue.cc:237] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:46.735013 29834 raft_consensus.cc:2468] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:46.747341 29665 catalog_manager.cc:5697] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.927795 30889 raft_consensus.cc:993] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:46.928309 30889 raft_consensus.cc:1081] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:46.929736 30589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:10:46.930214 30589 raft_consensus.cc:493] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:46.930505 30589 raft_consensus.cc:3060] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.936371 30589 raft_consensus.cc:515] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:46.937876 30589 leader_election.cc:290] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:46.938689 29914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:46.938967 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced82bb96f8a45e790d075986ff66cce" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:46.939378 29914 raft_consensus.cc:3060] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.939529 29983 raft_consensus.cc:3055] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:46.939890 29983 raft_consensus.cc:740] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:46.940653 29983 consensus_queue.cc:260] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:46.941881 29983 raft_consensus.cc:3060] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.946236 29914 raft_consensus.cc:2468] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:46.947536 30544 leader_election.cc:304] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06; no voters: 
I20260709 13:10:46.948199 30828 raft_consensus.cc:2804] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:46.948309 29983 raft_consensus.cc:2468] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:46.948653 30828 raft_consensus.cc:697] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:46.949504 30828 consensus_queue.cc:237] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:46.958364 29665 catalog_manager.cc:5697] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:46.974638 30836 raft_consensus.cc:993] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:46.975111 30882 raft_consensus.cc:1081] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:46.976629 30589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44744
I20260709 13:10:46.977104 30589 raft_consensus.cc:493] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:46.977373 30589 raft_consensus.cc:3060] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.983054 30589 raft_consensus.cc:515] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:46.984638 30589 leader_election.cc:290] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:46.985714 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:46.986176 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8807cef4b9f940359ad32d99dd28b5cd" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:46.986373 29983 raft_consensus.cc:3060] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.986742 29841 raft_consensus.cc:3055] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:46.987063 29841 raft_consensus.cc:740] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d08b1a18bb214fa1b08560d4d8149deb, State: Running, Role: LEADER
I20260709 13:10:46.987820 29841 consensus_queue.cc:260] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:46.989050 29841 raft_consensus.cc:3060] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.995872 29841 raft_consensus.cc:2468] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:46.998674 30544 leader_election.cc:304] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:46.999502 30828 raft_consensus.cc:2804] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:46.999969 30828 raft_consensus.cc:697] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:47.001153 29983 raft_consensus.cc:2468] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 2.
I20260709 13:10:47.000873 30828 consensus_queue.cc:237] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:47.017047 29665 catalog_manager.cc:5697] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 1 to 2, leader changed from d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:47.121486 30850 raft_consensus.cc:993] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:47.122140 30889 raft_consensus.cc:1081] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:47.124207 30589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df180e07f62245aa9c117b1558657534"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:10:47.124742 30589 raft_consensus.cc:493] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:47.125166 30589 raft_consensus.cc:3060] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:47.131888 30589 raft_consensus.cc:515] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:47.133759 30589 leader_election.cc:290] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 election: Requested vote from peers 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:47.134361 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:47.135427 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df180e07f62245aa9c117b1558657534" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:47.135850 29983 raft_consensus.cc:3055] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:47.135979 29841 raft_consensus.cc:3060] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:47.136224 29983 raft_consensus.cc:740] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:47.137427 29983 consensus_queue.cc:260] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:47.138720 29983 raft_consensus.cc:3060] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:47.143007 29841 raft_consensus.cc:2468] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 3.
I20260709 13:10:47.144245 30544 leader_election.cc:304] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:47.145078 30897 raft_consensus.cc:2804] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:47.145566 30897 raft_consensus.cc:697] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 3 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:47.145869 29983 raft_consensus.cc:2468] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 3.
I20260709 13:10:47.146430 30897 consensus_queue.cc:237] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:47.152264 30850 raft_consensus.cc:993] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3: : Instructing follower 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:47.152772 30914 raft_consensus.cc:1081] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Signalling peer 74191176370841578d396cc01f230f06 to start an election
I20260709 13:10:47.154330 29907 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9"
dest_uuid: "74191176370841578d396cc01f230f06"
 from {username='slave'} at 127.0.0.1:60766
I20260709 13:10:47.154826 29907 raft_consensus.cc:493] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:47.158275 29907 raft_consensus.cc:3060] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:47.157842 29665 catalog_manager.cc:5697] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 2 to 3, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 3 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:47.165614 29907 raft_consensus.cc:515] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:47.168923 29907 leader_election.cc:290] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 75162022240a4fe1a113333e77964ca3 (127.19.127.196:46707)
I20260709 13:10:47.169941 29983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75162022240a4fe1a113333e77964ca3"
I20260709 13:10:47.170343 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc1a922f421149f48209b8e771fb3ca9" candidate_uuid: "74191176370841578d396cc01f230f06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:47.170701 29983 raft_consensus.cc:3055] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:47.171007 29841 raft_consensus.cc:3060] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:47.171092 29983 raft_consensus.cc:740] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75162022240a4fe1a113333e77964ca3, State: Running, Role: LEADER
I20260709 13:10:47.172045 29983 consensus_queue.cc:260] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:47.173192 29983 raft_consensus.cc:3060] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:47.177522 29841 raft_consensus.cc:2468] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:47.178645 29881 leader_election.cc:304] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74191176370841578d396cc01f230f06, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:47.179513 30863 raft_consensus.cc:2804] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:47.180011 30863 raft_consensus.cc:697] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Becoming Leader. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:47.180583 29983 raft_consensus.cc:2468] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74191176370841578d396cc01f230f06 in term 2.
I20260709 13:10:47.180837 30863 consensus_queue.cc:237] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } }
I20260709 13:10:47.191088 29665 catalog_manager.cc:5697] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 reported cstate change: term changed from 1 to 2, leader changed from 75162022240a4fe1a113333e77964ca3 (127.19.127.196) to 74191176370841578d396cc01f230f06 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "74191176370841578d396cc01f230f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:47.219424 29834 raft_consensus.cc:1275] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.220700 30863 consensus_queue.cc:1048] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.231547 29983 raft_consensus.cc:1275] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.237228 30920 consensus_queue.cc:1048] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:47.393692 29983 raft_consensus.cc:1275] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.395094 30897 consensus_queue.cc:1048] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.411144 29841 raft_consensus.cc:1275] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.412505 30897 consensus_queue.cc:1048] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.496955 29990 raft_consensus.cc:1275] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.498248 30828 consensus_queue.cc:1048] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.507896 29907 raft_consensus.cc:1275] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.510919 30861 consensus_queue.cc:1048] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:47.667202 29834 raft_consensus.cc:1275] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:47.668470 30897 consensus_queue.cc:1048] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:47.684377 29834 raft_consensus.cc:1275] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.687630 29990 raft_consensus.cc:1275] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:47.688794 30920 consensus_queue.cc:1048] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.694144 30861 consensus_queue.cc:1048] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:47.705055 29983 raft_consensus.cc:1275] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Refusing update from remote peer 74191176370841578d396cc01f230f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.707454 30851 consensus_queue.cc:1048] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.710031 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:47.767082 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:47.812249 29914 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297"
dest_uuid: "74191176370841578d396cc01f230f06"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:50186
I20260709 13:10:47.812810 29914 raft_consensus.cc:606] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:47.815965 29914 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e388ba02893f419997372341b6f561d8"
dest_uuid: "74191176370841578d396cc01f230f06"
mode: GRACEFUL
new_leader_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:50186
I20260709 13:10:47.816557 29914 raft_consensus.cc:606] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 3 LEADER]: Received request to transfer leadership to TS 2ba28923c4534c78903f759f0a4e1265
I20260709 13:10:47.817701 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:47.819581 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:48.054101 30863 raft_consensus.cc:993] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:48.054603 30851 raft_consensus.cc:1081] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:48.056190 30598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44728
I20260709 13:10:48.056699 30598 raft_consensus.cc:493] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:48.057119 30598 raft_consensus.cc:3060] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:48.067410 30598 raft_consensus.cc:515] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:48.070322 29841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:48.070914 29841 raft_consensus.cc:3060] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:48.070946 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55c176c0f054ab4ac3e880739d7c297" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:48.071677 29907 raft_consensus.cc:3055] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:48.071959 29907 raft_consensus.cc:740] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:48.069656 30598 leader_election.cc:290] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:48.072679 29907 consensus_queue.cc:260] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:48.075719 29907 raft_consensus.cc:3060] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:48.078248 29841 raft_consensus.cc:2468] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 3.
I20260709 13:10:48.079372 30544 leader_election.cc:304] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:48.080061 30828 raft_consensus.cc:2804] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:48.080673 30828 raft_consensus.cc:697] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 3 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:48.081571 30828 consensus_queue.cc:237] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:48.082571 29907 raft_consensus.cc:2468] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 3.
I20260709 13:10:48.086426 30851 raft_consensus.cc:993] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06: : Instructing follower 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:48.087885 30863 raft_consensus.cc:1081] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 3 LEADER]: Signalling peer 2ba28923c4534c78903f759f0a4e1265 to start an election
I20260709 13:10:48.089627 30598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e388ba02893f419997372341b6f561d8"
dest_uuid: "2ba28923c4534c78903f759f0a4e1265"
 from {username='slave'} at 127.0.0.1:44728
I20260709 13:10:48.090167 30598 raft_consensus.cc:493] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:48.090520 30598 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:48.093979 29663 catalog_manager.cc:5697] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 2 to 3, leader changed from 74191176370841578d396cc01f230f06 (127.19.127.195) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 3 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.098083 30598 raft_consensus.cc:515] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:48.100219 30598 leader_election.cc:290] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 4 election: Requested vote from peers d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603), 74191176370841578d396cc01f230f06 (127.19.127.195:40257)
I20260709 13:10:48.102242 29834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d08b1a18bb214fa1b08560d4d8149deb"
I20260709 13:10:48.102839 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e388ba02893f419997372341b6f561d8" candidate_uuid: "2ba28923c4534c78903f759f0a4e1265" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "74191176370841578d396cc01f230f06"
I20260709 13:10:48.103758 29907 raft_consensus.cc:3055] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:48.104038 29907 raft_consensus.cc:740] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 74191176370841578d396cc01f230f06, State: Running, Role: LEADER
I20260709 13:10:48.102854 29834 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:48.108222 29907 consensus_queue.cc:260] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:48.120049 29907 raft_consensus.cc:3060] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:48.123896 29834 raft_consensus.cc:2468] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 4.
I20260709 13:10:48.125208 30544 leader_election.cc:304] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb; no voters: 
I20260709 13:10:48.126129 30897 raft_consensus.cc:2804] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:48.126632 30897 raft_consensus.cc:697] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 4 LEADER]: Becoming Leader. State: Replica: 2ba28923c4534c78903f759f0a4e1265, State: Running, Role: LEADER
I20260709 13:10:48.127264 29907 raft_consensus.cc:2468] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba28923c4534c78903f759f0a4e1265 in term 4.
I20260709 13:10:48.127549 30897 consensus_queue.cc:237] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:48.200984 29663 catalog_manager.cc:5697] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 reported cstate change: term changed from 3 to 4, leader changed from 74191176370841578d396cc01f230f06 (127.19.127.195) to 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193). New cstate: current_term: 4 leader_uuid: "2ba28923c4534c78903f759f0a4e1265" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.463990 29841 raft_consensus.cc:1275] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:48.467274 30828 consensus_queue.cc:1048] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:10:48.513629 29914 raft_consensus.cc:1275] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:48.516893 30828 consensus_queue.cc:1048] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:48.703315 29914 raft_consensus.cc:1275] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 4 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:48.705057 30946 consensus_queue.cc:1048] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:48.715924 29841 raft_consensus.cc:1275] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 4 FOLLOWER]: Refusing update from remote peer 2ba28923c4534c78903f759f0a4e1265: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:48.718502 30861 consensus_queue.cc:1048] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:48.820791 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:48.875530 19967 tablet_server.cc:179] TabletServer@127.19.127.193:35599 shutting down...
W20260709 13:10:48.912082 29807 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 94 similar messages]
W20260709 13:10:48.923705 29955 consensus_peers.cc:597] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:48.933658 29807 consensus_peers.cc:597] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:48.934824 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:10:48.935994 29807 consensus_peers.cc:597] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:48.935997 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.939384 19967 raft_consensus.cc:2243] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:48.941308 19967 raft_consensus.cc:2272] T 21581ff708354cb0ad0ed6ae935d4eef P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.944504 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.945222 19967 raft_consensus.cc:2243] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.945685 19967 raft_consensus.cc:2272] T 8405db80d77d4b8c9737d33de3ca1f74 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.948264 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.949000 19967 raft_consensus.cc:2243] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.949448 19967 raft_consensus.cc:2272] T c6b780417ded422690ec3d46dda49a07 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.951902 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.952545 19967 raft_consensus.cc:2243] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:48.953739 19967 raft_consensus.cc:2272] T 6d91cac27fc9443b86fc3821776294de P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.957089 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.957882 19967 raft_consensus.cc:2243] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:48.959004 19967 raft_consensus.cc:2272] T ced82bb96f8a45e790d075986ff66cce P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.961453 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.962136 19967 raft_consensus.cc:2243] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.962556 19967 raft_consensus.cc:2272] T 244d599ec58f4e51ac20679e26a3c2b1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.965816 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.966931 19967 raft_consensus.cc:2243] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.967684 19967 raft_consensus.cc:2272] T ce794c74850e4dc6845e02da675fa199 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.971271 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.972363 19967 raft_consensus.cc:2243] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.972980 19967 raft_consensus.cc:2272] T 9f721871c07c4c57b9c2a1185604e336 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:48.973526 29955 consensus_peers.cc:597] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:48.976912 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.977978 19967 raft_consensus.cc:2243] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.978601 19967 raft_consensus.cc:2272] T dc233ddc2b6d41f395652a5c5010c88b P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.983345 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.984313 19967 raft_consensus.cc:2243] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.984889 19967 raft_consensus.cc:2272] T 277299b9ea1b456eb943ddf7be26fe05 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:48.986418 29881 consensus_peers.cc:597] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:48.988377 29955 consensus_peers.cc:597] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:48.988924 19967 tablet_replica.cc:333] stopping tablet replica
W20260709 13:10:48.989171 29955 consensus_peers.cc:597] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:48.989864 19967 raft_consensus.cc:2243] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.990387 19967 raft_consensus.cc:2272] T 95db6009685e4a3d88d098ea07105ad7 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.992899 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.993559 19967 raft_consensus.cc:2243] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.993973 19967 raft_consensus.cc:2272] T 0bbc73ad9e6848228c0d2cae81843b7a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:48.996724 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:48.997301 19967 raft_consensus.cc:2243] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:48.997751 19967 raft_consensus.cc:2272] T 0f4a836f7bfb414eb5bba33a3be51d4f P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.000280 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.000926 19967 raft_consensus.cc:2243] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.001566 19967 raft_consensus.cc:2272] T 99a9fd11d5a348389207646896abb152 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.005358 19967 tablet_replica.cc:333] stopping tablet replica
W20260709 13:10:49.005786 29955 consensus_peers.cc:597] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.006238 19967 raft_consensus.cc:2243] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.006918 19967 raft_consensus.cc:2272] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.009325 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.010001 19967 raft_consensus.cc:2243] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.010442 19967 raft_consensus.cc:2272] T 523db1e9cca74ffb938f22ae9105dbe1 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.013744 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.014642 19967 raft_consensus.cc:2243] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.015234 19967 raft_consensus.cc:2272] T e9a1697952144bae8ddfeab6f69d13f4 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.017627 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.018415 19967 raft_consensus.cc:2243] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.018972 19967 raft_consensus.cc:2272] T d9f49c45a4a5479d8609fe95f8486b97 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.022038 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.022696 19967 raft_consensus.cc:2243] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.023216 19967 raft_consensus.cc:2272] T b1ff57a4871b4c2197e7c7f41230d429 P 2ba28923c4534c78903f759f0a4e1265 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.025940 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.026654 19967 raft_consensus.cc:2243] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.027216 19967 raft_consensus.cc:2272] T 20470bec53744d58b8955d49e2e7ea2a P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.029843 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.030577 19967 raft_consensus.cc:2243] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.031075 19967 raft_consensus.cc:2272] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.034266 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.034920 19967 raft_consensus.cc:2243] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.035435 19967 raft_consensus.cc:2272] T 0ef88786080e40e1a8a8b623e9c30106 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.038627 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.039331 19967 raft_consensus.cc:2243] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.040241 19967 raft_consensus.cc:2272] T a55c176c0f054ab4ac3e880739d7c297 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.043706 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.044560 19967 raft_consensus.cc:2243] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.045113 19967 raft_consensus.cc:2272] T 6ac2292232bc434b88b2786ed56f7adb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.048177 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.048934 19967 raft_consensus.cc:2243] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.049568 19967 raft_consensus.cc:2272] T 6714cbaa854d430fb09d82c5d878cff7 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.053257 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.053957 19967 raft_consensus.cc:2243] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.055335 19967 raft_consensus.cc:2272] T df180e07f62245aa9c117b1558657534 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.056057 29881 consensus_peers.cc:597] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.058764 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.059633 19967 raft_consensus.cc:2243] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.060940 19967 raft_consensus.cc:2272] T 1b6c48a3ec224675bd35d17f96f8fb13 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.064218 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.065104 19967 raft_consensus.cc:2243] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.065640 19967 raft_consensus.cc:2272] T 70e5a681df5b42e98dac6b49a96773fe P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.068773 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.069379 19967 raft_consensus.cc:2243] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.069808 19967 raft_consensus.cc:2272] T 252766f9503842f48be95fdd3e3035c2 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.072249 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.072803 19967 raft_consensus.cc:2243] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.073985 19967 raft_consensus.cc:2272] T 9fecd61828184c7e9cba33adb56ccd51 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.077558 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.078403 19967 raft_consensus.cc:2243] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.078969 19967 raft_consensus.cc:2272] T e86eb4af5624466bb2d9395a36632d71 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.079782 29881 consensus_peers.cc:597] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.082083 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.083038 19967 raft_consensus.cc:2243] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.083738 19967 raft_consensus.cc:2272] T 5481f26841ab45efa35ded9872e310ee P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.084995 29955 consensus_peers.cc:597] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.087085 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.088156 19967 raft_consensus.cc:2243] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.088725 19967 raft_consensus.cc:2272] T 8a6337639944433881ce49be8f337030 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.092348 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.092952 19967 raft_consensus.cc:2243] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.093487 19967 raft_consensus.cc:2272] T 96641d56c48649d993e0eec66d091ee5 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.096114 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.096688 19967 raft_consensus.cc:2243] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.097723 19967 raft_consensus.cc:2272] T 74e8c425db9e4e479c00e6f131fbafe2 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.100581 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.101234 19967 raft_consensus.cc:2243] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.102070 19967 raft_consensus.cc:2272] T caad1999ec5545d9a8eb2ade21997cab P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.104663 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.105376 19967 raft_consensus.cc:2243] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.106516 19967 raft_consensus.cc:2272] T fac962e63d404b2a89aa0afd301ce161 P 2ba28923c4534c78903f759f0a4e1265 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.109122 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.109642 19967 raft_consensus.cc:2243] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.110070 19967 raft_consensus.cc:2272] T 6d7968a480124ac9ad9734c6ffc0dbbb P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.111781 29807 consensus_peers.cc:597] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.113173 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.113988 19967 raft_consensus.cc:2243] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.114542 19967 raft_consensus.cc:2272] T e388e95cfde842949825ab8dbdc03e91 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.115420 29807 consensus_peers.cc:597] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.116962 29807 consensus_peers.cc:597] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.117722 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.118546 19967 raft_consensus.cc:2243] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.119642 19967 raft_consensus.cc:2272] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.121788 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.122462 19967 raft_consensus.cc:2243] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.123472 19967 raft_consensus.cc:2272] T e388ba02893f419997372341b6f561d8 P 2ba28923c4534c78903f759f0a4e1265 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.127468 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.128041 19967 raft_consensus.cc:2243] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.128479 19967 raft_consensus.cc:2272] T 90385323444f41d096dd3842e8b2bd6d P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.131520 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.132153 19967 raft_consensus.cc:2243] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.133261 19967 raft_consensus.cc:2272] T 1207ccf2285f42dc901a1db9e48e39f3 P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.134721 29955 consensus_peers.cc:597] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.136622 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.137373 19967 raft_consensus.cc:2243] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.137851 19967 raft_consensus.cc:2272] T af3bf2e1f12f46c59f56c808bcfd5e78 P 2ba28923c4534c78903f759f0a4e1265 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.140877 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:10:49.141516 19967 raft_consensus.cc:2243] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.142433 19967 raft_consensus.cc:2272] T 8807cef4b9f940359ad32d99dd28b5cd P 2ba28923c4534c78903f759f0a4e1265 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.184162 29881 consensus_peers.cc:597] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.185907 29807 consensus_peers.cc:597] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.222985 29807 consensus_peers.cc:597] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.228752 19967 tablet_server.cc:196] TabletServer@127.19.127.193:35599 shutdown complete.
W20260709 13:10:49.231921 29955 consensus_peers.cc:597] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.240604 29955 consensus_peers.cc:597] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.242322 29807 consensus_peers.cc:597] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.249652 29881 consensus_peers.cc:597] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.250331 29881 consensus_peers.cc:597] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.252173 29955 consensus_peers.cc:597] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.256208 29881 consensus_peers.cc:597] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.258638 29955 consensus_peers.cc:597] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.289757 29807 consensus_peers.cc:597] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.331629 29807 consensus_peers.cc:597] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.333681 29955 consensus_peers.cc:597] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.333933 19967 tablet_server.cc:179] TabletServer@127.19.127.194:0 shutting down...
W20260709 13:10:49.339861 29881 consensus_peers.cc:597] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.343272 29881 consensus_peers.cc:597] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.346853 29807 consensus_peers.cc:597] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.388437 29955 consensus_peers.cc:597] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.399699 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:49.400578 19967 tablet_replica.cc:333] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.401579 19967 raft_consensus.cc:2243] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.402691 19967 raft_consensus.cc:2272] T 244d599ec58f4e51ac20679e26a3c2b1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.405185 19967 tablet_replica.cc:333] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.405828 19967 raft_consensus.cc:2243] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.406275 19967 raft_consensus.cc:2272] T 8b1bb074fa4f46b48be16325eabd6d7d P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.408368 19967 tablet_replica.cc:333] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.409061 19967 raft_consensus.cc:2243] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.409534 19967 raft_consensus.cc:2272] T 9fecd61828184c7e9cba33adb56ccd51 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.412482 19967 tablet_replica.cc:333] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.413112 19967 raft_consensus.cc:2243] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.413587 19967 raft_consensus.cc:2272] T 8a6337639944433881ce49be8f337030 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.415731 19967 tablet_replica.cc:333] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.416430 19967 raft_consensus.cc:2243] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.416896 19967 raft_consensus.cc:2272] T 9f721871c07c4c57b9c2a1185604e336 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.418876 19967 tablet_replica.cc:333] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.419577 19967 raft_consensus.cc:2243] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.420531 19967 raft_consensus.cc:2272] T 523db1e9cca74ffb938f22ae9105dbe1 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.422451 19967 tablet_replica.cc:333] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.423051 19967 raft_consensus.cc:2243] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.423954 19967 raft_consensus.cc:2272] T dc233ddc2b6d41f395652a5c5010c88b P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.426405 19967 tablet_replica.cc:333] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.427067 19967 raft_consensus.cc:2243] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.427584 19967 raft_consensus.cc:2272] T cc1a922f421149f48209b8e771fb3ca9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.429185 29881 consensus_peers.cc:597] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.430444 19967 tablet_replica.cc:333] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.431438 19967 raft_consensus.cc:2243] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.432489 19967 raft_consensus.cc:2272] T e9a1697952144bae8ddfeab6f69d13f4 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.432723 29881 consensus_peers.cc:597] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.434630 19967 tablet_replica.cc:333] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.435343 19967 raft_consensus.cc:2243] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.435848 19967 raft_consensus.cc:2272] T 8807cef4b9f940359ad32d99dd28b5cd P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.437999 19967 tablet_replica.cc:333] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.438603 19967 raft_consensus.cc:2243] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.439625 19967 raft_consensus.cc:2272] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.441911 19967 tablet_replica.cc:333] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.442454 19967 raft_consensus.cc:2243] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.442842 19967 raft_consensus.cc:2272] T 1207ccf2285f42dc901a1db9e48e39f3 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.444917 19967 tablet_replica.cc:333] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.445714 19967 raft_consensus.cc:2243] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.446242 19967 raft_consensus.cc:2272] T e388ba02893f419997372341b6f561d8 P d08b1a18bb214fa1b08560d4d8149deb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.449371 19967 tablet_replica.cc:333] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.450179 19967 raft_consensus.cc:2243] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.451246 19967 raft_consensus.cc:2272] T 90385323444f41d096dd3842e8b2bd6d P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.454226 19967 tablet_replica.cc:333] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.454968 19967 raft_consensus.cc:2243] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:49.454959 29807 consensus_peers.cc:597] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb -> Peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Couldn't send request to peer 2ba28923c4534c78903f759f0a4e1265. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.455679 19967 raft_consensus.cc:2272] T 4f9fb82d905c441bb50d70e1ce080213 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.458729 19967 tablet_replica.cc:333] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.459579 19967 raft_consensus.cc:2243] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.460614 19967 raft_consensus.cc:2272] T e02b4b0aee0c426ba054d6eac3eda348 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.463536 19967 tablet_replica.cc:333] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.464311 19967 raft_consensus.cc:2243] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.464867 19967 raft_consensus.cc:2272] T 6714cbaa854d430fb09d82c5d878cff7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.467865 19967 tablet_replica.cc:333] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.468586 19967 raft_consensus.cc:2243] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.469149 19967 raft_consensus.cc:2272] T a55c176c0f054ab4ac3e880739d7c297 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.472801 19967 tablet_replica.cc:333] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.473706 19967 raft_consensus.cc:2243] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.474435 19967 raft_consensus.cc:2272] T df180e07f62245aa9c117b1558657534 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.476389 19967 tablet_replica.cc:333] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.476979 19967 raft_consensus.cc:2243] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.477415 19967 raft_consensus.cc:2272] T d4e21634252d40b196da2d7ac6d5faa5 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.479764 19967 tablet_replica.cc:333] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.480580 19967 raft_consensus.cc:2243] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.481048 19967 raft_consensus.cc:2272] T e86eb4af5624466bb2d9395a36632d71 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.482343 29955 consensus_peers.cc:597] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.483672 19967 tablet_replica.cc:333] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.484449 19967 raft_consensus.cc:2243] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.484894 19967 raft_consensus.cc:2272] T 5481f26841ab45efa35ded9872e310ee P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.486791 19967 tablet_replica.cc:333] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.487386 19967 raft_consensus.cc:2243] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.488443 19967 raft_consensus.cc:2272] T 125fc2e410394efba24979999a6785b7 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.491410 19967 tablet_replica.cc:333] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.492208 19967 raft_consensus.cc:2243] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.492802 19967 raft_consensus.cc:2272] T 6d91cac27fc9443b86fc3821776294de P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.495962 19967 tablet_replica.cc:333] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.496713 19967 raft_consensus.cc:2243] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.497288 19967 raft_consensus.cc:2272] T 21581ff708354cb0ad0ed6ae935d4eef P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.500243 19967 tablet_replica.cc:333] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.500831 19967 raft_consensus.cc:2243] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.501852 19967 raft_consensus.cc:2272] T 2a52b7d43c054ac485deaf19144dced6 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.504380 19967 tablet_replica.cc:333] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.504931 19967 raft_consensus.cc:2243] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.505400 19967 raft_consensus.cc:2272] T 7255b6d0c90146d7bb85a327da87cec0 P d08b1a18bb214fa1b08560d4d8149deb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.508193 19967 tablet_replica.cc:333] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.508973 19967 raft_consensus.cc:2243] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.509557 19967 raft_consensus.cc:2272] T 737f2cb09b0a48b1987133f213820765 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.512451 19967 tablet_replica.cc:333] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.513195 19967 raft_consensus.cc:2243] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.514091 19967 raft_consensus.cc:2272] T c6b780417ded422690ec3d46dda49a07 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.516521 19967 tablet_replica.cc:333] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.517140 19967 raft_consensus.cc:2243] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.517702 19967 raft_consensus.cc:2272] T 95db6009685e4a3d88d098ea07105ad7 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.520212 19967 tablet_replica.cc:333] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.521070 19967 raft_consensus.cc:2243] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.522214 19967 raft_consensus.cc:2272] T e388e95cfde842949825ab8dbdc03e91 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.525362 19967 tablet_replica.cc:333] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.526212 19967 raft_consensus.cc:2243] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.527312 19967 raft_consensus.cc:2272] T d9f49c45a4a5479d8609fe95f8486b97 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.530359 19967 tablet_replica.cc:333] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.531196 19967 raft_consensus.cc:2243] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.532372 19967 raft_consensus.cc:2272] T b1ff57a4871b4c2197e7c7f41230d429 P d08b1a18bb214fa1b08560d4d8149deb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.534746 19967 tablet_replica.cc:333] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.535409 19967 raft_consensus.cc:2243] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.535889 19967 raft_consensus.cc:2272] T 8170c15c8bf942b596d738aa14bb6f80 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.538532 19967 tablet_replica.cc:333] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.539110 19967 raft_consensus.cc:2243] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.539649 19967 raft_consensus.cc:2272] T caad1999ec5545d9a8eb2ade21997cab P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.541371 29881 consensus_peers.cc:597] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.542382 19967 tablet_replica.cc:333] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.543146 19967 raft_consensus.cc:2243] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.543700 19967 raft_consensus.cc:2272] T 0e543a21ae8243058d010255b40860e9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.545899 19967 tablet_replica.cc:333] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.546453 19967 raft_consensus.cc:2243] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.547323 19967 raft_consensus.cc:2272] T 6d7968a480124ac9ad9734c6ffc0dbbb P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.549398 19967 tablet_replica.cc:333] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.549942 19967 raft_consensus.cc:2243] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.550420 19967 raft_consensus.cc:2272] T 45a7caaa2edf4bafacd9da8900ebeeb2 P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.552456 19967 tablet_replica.cc:333] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.552973 19967 raft_consensus.cc:2243] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.553421 19967 raft_consensus.cc:2272] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.555470 19967 tablet_replica.cc:333] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.555968 19967 raft_consensus.cc:2243] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.556411 19967 raft_consensus.cc:2272] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.558531 19967 tablet_replica.cc:333] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.559058 19967 raft_consensus.cc:2243] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.559530 19967 raft_consensus.cc:2272] T d182f1fe9db6430d8ff9a0c29b11c3f3 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.561384 19967 tablet_replica.cc:333] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.561880 19967 raft_consensus.cc:2243] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.562336 19967 raft_consensus.cc:2272] T 99a9fd11d5a348389207646896abb152 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.564531 19967 tablet_replica.cc:333] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.565081 19967 raft_consensus.cc:2243] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.565603 19967 raft_consensus.cc:2272] T 70e5a681df5b42e98dac6b49a96773fe P d08b1a18bb214fa1b08560d4d8149deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.567809 19967 tablet_replica.cc:333] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb: stopping tablet replica
I20260709 13:10:49.568454 19967 raft_consensus.cc:2243] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.569205 19967 raft_consensus.cc:2272] T 252766f9503842f48be95fdd3e3035c2 P d08b1a18bb214fa1b08560d4d8149deb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.574848 29955 consensus_peers.cc:597] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.594992 29955 consensus_peers.cc:597] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.606055 29881 consensus_peers.cc:597] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.608666 29881 consensus_peers.cc:597] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.645716 29881 consensus_peers.cc:597] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.652596 29955 consensus_peers.cc:597] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.659498 29881 consensus_peers.cc:597] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.664029 29955 consensus_peers.cc:597] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.671463 29955 consensus_peers.cc:597] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.679805 29955 consensus_peers.cc:597] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.711102 19967 tablet_server.cc:196] TabletServer@127.19.127.194:0 shutdown complete.
W20260709 13:10:49.741977 29955 consensus_peers.cc:597] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.745549 29955 consensus_peers.cc:597] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:49.765374 29881 consensus_peers.cc:597] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 -> Peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Couldn't send request to peer d08b1a18bb214fa1b08560d4d8149deb. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.834545 19967 tablet_server.cc:179] TabletServer@127.19.127.195:0 shutting down...
I20260709 13:10:49.903391 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:49.904135 19967 tablet_replica.cc:333] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.904788 19967 raft_consensus.cc:2243] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.905359 19967 raft_consensus.cc:2272] T 9fecd61828184c7e9cba33adb56ccd51 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.907610 19967 tablet_replica.cc:333] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.908224 19967 raft_consensus.cc:2243] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.908701 19967 raft_consensus.cc:2272] T 8b1bb074fa4f46b48be16325eabd6d7d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.910868 19967 tablet_replica.cc:333] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.911598 19967 raft_consensus.cc:2243] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.912083 19967 raft_consensus.cc:2272] T 74e8c425db9e4e479c00e6f131fbafe2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.914408 19967 tablet_replica.cc:333] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.915058 19967 raft_consensus.cc:2243] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.916021 19967 raft_consensus.cc:2272] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.918522 19967 tablet_replica.cc:333] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.919132 19967 raft_consensus.cc:2243] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.919637 19967 raft_consensus.cc:2272] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.921744 19967 tablet_replica.cc:333] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.922480 19967 raft_consensus.cc:2243] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.923472 19967 raft_consensus.cc:2272] T 9f721871c07c4c57b9c2a1185604e336 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.926532 19967 tablet_replica.cc:333] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.927239 19967 raft_consensus.cc:2243] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.927778 19967 raft_consensus.cc:2272] T ced82bb96f8a45e790d075986ff66cce P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.929802 19967 tablet_replica.cc:333] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.930408 19967 raft_consensus.cc:2243] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.930863 19967 raft_consensus.cc:2272] T e9a1697952144bae8ddfeab6f69d13f4 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.933018 19967 tablet_replica.cc:333] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.933611 19967 raft_consensus.cc:2243] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.934048 19967 raft_consensus.cc:2272] T dc233ddc2b6d41f395652a5c5010c88b P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.936106 19967 tablet_replica.cc:333] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.936640 19967 raft_consensus.cc:2243] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.937443 19967 raft_consensus.cc:2272] T cc1a922f421149f48209b8e771fb3ca9 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.939550 19967 tablet_replica.cc:333] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.940096 19967 raft_consensus.cc:2243] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.940510 19967 raft_consensus.cc:2272] T 96641d56c48649d993e0eec66d091ee5 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.942443 19967 tablet_replica.cc:333] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.943019 19967 raft_consensus.cc:2243] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.943871 19967 raft_consensus.cc:2272] T 8170c15c8bf942b596d738aa14bb6f80 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.945761 19967 tablet_replica.cc:333] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.946355 19967 raft_consensus.cc:2243] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.946806 19967 raft_consensus.cc:2272] T 1b6c48a3ec224675bd35d17f96f8fb13 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.948763 19967 tablet_replica.cc:333] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.949298 19967 raft_consensus.cc:2243] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.950104 19967 raft_consensus.cc:2272] T d4e21634252d40b196da2d7ac6d5faa5 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.951977 19967 tablet_replica.cc:333] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.952592 19967 raft_consensus.cc:2243] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.953074 19967 raft_consensus.cc:2272] T 4f9fb82d905c441bb50d70e1ce080213 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.955037 19967 tablet_replica.cc:333] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.955641 19967 raft_consensus.cc:2243] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.956444 19967 raft_consensus.cc:2272] T ce794c74850e4dc6845e02da675fa199 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.958495 19967 tablet_replica.cc:333] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.959040 19967 raft_consensus.cc:2243] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.959514 19967 raft_consensus.cc:2272] T 1207ccf2285f42dc901a1db9e48e39f3 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.961511 19967 tablet_replica.cc:333] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.962337 19967 raft_consensus.cc:2243] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.962859 19967 raft_consensus.cc:2272] T e388ba02893f419997372341b6f561d8 P 74191176370841578d396cc01f230f06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.964876 19967 tablet_replica.cc:333] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.965441 19967 raft_consensus.cc:2243] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.965889 19967 raft_consensus.cc:2272] T 90385323444f41d096dd3842e8b2bd6d P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.968120 19967 tablet_replica.cc:333] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.968420 30914 raft_consensus.cc:493] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:49.969019 19967 raft_consensus.cc:2243] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.968992 30914 raft_consensus.cc:515] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:49.969795 19967 raft_consensus.cc:2272] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.971505 30914 leader_election.cc:290] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:49.972810 19967 tablet_replica.cc:333] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.973685 19967 raft_consensus.cc:2243] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.974575 19967 raft_consensus.cc:2272] T 0ef88786080e40e1a8a8b623e9c30106 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.975174 29955 leader_election.cc:336] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
W20260709 13:10:49.975909 29955 leader_election.cc:336] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:49.976248 29955 leader_election.cc:304] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:49.976842 30914 raft_consensus.cc:2749] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:49.977484 19967 tablet_replica.cc:333] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.978209 19967 raft_consensus.cc:2243] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.979225 19967 raft_consensus.cc:2272] T 277299b9ea1b456eb943ddf7be26fe05 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.979302 29955 consensus_peers.cc:597] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 -> Peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Couldn't send request to peer 74191176370841578d396cc01f230f06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.981693 19967 tablet_replica.cc:333] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.982435 19967 raft_consensus.cc:2243] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.982915 19967 raft_consensus.cc:2272] T e388e95cfde842949825ab8dbdc03e91 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.985044 19967 tablet_replica.cc:333] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.985783 19967 raft_consensus.cc:2243] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.986321 19967 raft_consensus.cc:2272] T 125fc2e410394efba24979999a6785b7 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:49.988749 29955 consensus_peers.cc:597] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 -> Peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Couldn't send request to peer 74191176370841578d396cc01f230f06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:49.989032 19967 tablet_replica.cc:333] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.989833 19967 raft_consensus.cc:2243] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.990365 19967 raft_consensus.cc:2272] T 2a52b7d43c054ac485deaf19144dced6 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.993244 19967 tablet_replica.cc:333] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.994056 19967 raft_consensus.cc:2243] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.995045 19967 raft_consensus.cc:2272] T 7255b6d0c90146d7bb85a327da87cec0 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.997983 19967 tablet_replica.cc:333] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:49.998792 19967 raft_consensus.cc:2243] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:49.999867 19967 raft_consensus.cc:2272] T af3bf2e1f12f46c59f56c808bcfd5e78 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.002767 19967 tablet_replica.cc:333] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.003546 19967 raft_consensus.cc:2243] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.004118 19967 raft_consensus.cc:2272] T d9f49c45a4a5479d8609fe95f8486b97 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.006788 19967 tablet_replica.cc:333] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.007445 19967 raft_consensus.cc:2243] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.007915 19967 raft_consensus.cc:2272] T 20470bec53744d58b8955d49e2e7ea2a P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.009990 19967 tablet_replica.cc:333] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.010676 19967 raft_consensus.cc:2243] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.011193 19967 raft_consensus.cc:2272] T b1ff57a4871b4c2197e7c7f41230d429 P 74191176370841578d396cc01f230f06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.013190 19967 tablet_replica.cc:333] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.013717 19967 raft_consensus.cc:2243] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.014130 19967 raft_consensus.cc:2272] T 0bbc73ad9e6848228c0d2cae81843b7a P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.016515 19967 tablet_replica.cc:333] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.017289 19967 raft_consensus.cc:2243] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.018364 19967 raft_consensus.cc:2272] T 8405db80d77d4b8c9737d33de3ca1f74 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.021034 19967 tablet_replica.cc:333] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.021615 19967 raft_consensus.cc:2243] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.022176 19967 raft_consensus.cc:2272] T 21581ff708354cb0ad0ed6ae935d4eef P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:50.022523 29955 consensus_peers.cc:597] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 -> Peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Couldn't send request to peer 74191176370841578d396cc01f230f06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.024411 19967 tablet_replica.cc:333] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.025079 19967 raft_consensus.cc:2243] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.025615 19967 raft_consensus.cc:2272] T 6d7968a480124ac9ad9734c6ffc0dbbb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.025666 30863 raft_consensus.cc:493] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:50.026202 30863 raft_consensus.cc:515] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } } peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } }
I20260709 13:10:50.027859 30863 leader_election.cc:290] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:50.028376 19967 tablet_replica.cc:333] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.029292 19967 raft_consensus.cc:2243] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.029891 19967 raft_consensus.cc:2272] T fac962e63d404b2a89aa0afd301ce161 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:50.032529 29881 leader_election.cc:336] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
I20260709 13:10:50.032548 19967 tablet_replica.cc:333] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06: stopping tablet replica
W20260709 13:10:50.033231 29881 leader_election.cc:336] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111)
I20260709 13:10:50.033456 19967 raft_consensus.cc:2243] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.033553 29881 leader_election.cc:304] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74191176370841578d396cc01f230f06; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:50.034371 30863 raft_consensus.cc:2749] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:50.034754 19967 raft_consensus.cc:2272] T 737f2cb09b0a48b1987133f213820765 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.036990 19967 tablet_replica.cc:333] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.037559 19967 raft_consensus.cc:2243] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.038360 19967 raft_consensus.cc:2272] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.040488 19967 tablet_replica.cc:333] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.041057 19967 raft_consensus.cc:2243] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.041469 19967 raft_consensus.cc:2272] T e02b4b0aee0c426ba054d6eac3eda348 P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.043382 19967 tablet_replica.cc:333] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.043867 19967 raft_consensus.cc:2243] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.044651 19967 raft_consensus.cc:2272] T 6714cbaa854d430fb09d82c5d878cff7 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.047319 19967 tablet_replica.cc:333] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.047936 19967 raft_consensus.cc:2243] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.048497 19967 raft_consensus.cc:2272] T a55c176c0f054ab4ac3e880739d7c297 P 74191176370841578d396cc01f230f06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.051198 19967 tablet_replica.cc:333] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.051888 19967 raft_consensus.cc:2243] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.052405 19967 raft_consensus.cc:2272] T 6ac2292232bc434b88b2786ed56f7adb P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.054996 19967 tablet_replica.cc:333] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.055557 19967 raft_consensus.cc:2243] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.056274 19967 raft_consensus.cc:2272] T 0e543a21ae8243058d010255b40860e9 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.058319 19967 tablet_replica.cc:333] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.058739 19967 raft_consensus.cc:2243] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.059181 19967 raft_consensus.cc:2272] T 252766f9503842f48be95fdd3e3035c2 P 74191176370841578d396cc01f230f06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.061167 19967 tablet_replica.cc:333] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06: stopping tablet replica
I20260709 13:10:50.061509 30914 raft_consensus.cc:493] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:50.061648 19967 raft_consensus.cc:2243] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.062326 19967 raft_consensus.cc:2272] T 0f4a836f7bfb414eb5bba33a3be51d4f P 74191176370841578d396cc01f230f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.062143 30914 raft_consensus.cc:515] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:50.064394 30914 leader_election.cc:290] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
I20260709 13:10:50.064972 30850 raft_consensus.cc:493] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:50.065446 30850 raft_consensus.cc:515] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:50.067030 30850 leader_election.cc:290] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
W20260709 13:10:50.071866 29955 consensus_peers.cc:597] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 -> Peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Couldn't send request to peer 74191176370841578d396cc01f230f06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:50.072511 29955 leader_election.cc:336] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:10:50.075758 29955 leader_election.cc:336] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
I20260709 13:10:50.076224 29955 leader_election.cc:304] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:50.076836 30850 raft_consensus.cc:2749] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:50.077034 29955 leader_election.cc:336] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
W20260709 13:10:50.077876 29955 leader_election.cc:336] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111)
I20260709 13:10:50.078329 29955 leader_election.cc:304] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:50.079250 30956 raft_consensus.cc:2749] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:50.093650 29955 consensus_peers.cc:597] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 -> Peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Couldn't send request to peer 74191176370841578d396cc01f230f06. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.148928 19967 tablet_server.cc:196] TabletServer@127.19.127.195:0 shutdown complete.
I20260709 13:10:50.202488 30850 raft_consensus.cc:493] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:50.203164 30850 raft_consensus.cc:515] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:50.205363 30850 leader_election.cc:290] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
W20260709 13:10:50.210620 29955 leader_election.cc:336] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
W20260709 13:10:50.211467 29955 leader_election.cc:336] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Network error: Client connection negotiation failed: client connection to 127.19.127.195:40257: connect: Connection refused (error 111)
I20260709 13:10:50.211921 29955 leader_election.cc:304] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:50.212651 30850 raft_consensus.cc:2749] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:50.228572 19967 tablet_server.cc:179] TabletServer@127.19.127.196:0 shutting down...
I20260709 13:10:50.234165 30850 raft_consensus.cc:493] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:50.234859 30850 raft_consensus.cc:515] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74191176370841578d396cc01f230f06" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40257 } } peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } }
I20260709 13:10:50.236929 30850 leader_election.cc:290] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 74191176370841578d396cc01f230f06 (127.19.127.195:40257), 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599)
W20260709 13:10:50.241340 29955 leader_election.cc:336] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
W20260709 13:10:50.242112 29955 leader_election.cc:336] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Network error: Client connection negotiation failed: client connection to 127.19.127.195:40257: connect: Connection refused (error 111)
I20260709 13:10:50.242532 29955 leader_election.cc:304] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, 74191176370841578d396cc01f230f06
I20260709 13:10:50.243335 30850 raft_consensus.cc:2749] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:50.282750 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:50.283696 19967 tablet_replica.cc:333] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.284740 19967 raft_consensus.cc:2243] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.285799 19967 raft_consensus.cc:2272] T 8b1bb074fa4f46b48be16325eabd6d7d P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.288535 19967 tablet_replica.cc:333] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.289198 19967 raft_consensus.cc:2243] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.290058 19967 raft_consensus.cc:2272] T 20470bec53744d58b8955d49e2e7ea2a P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.292537 19967 tablet_replica.cc:333] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.293151 19967 raft_consensus.cc:2243] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:50.293247 29955 consensus_peers.cc:597] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 -> Peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Couldn't send request to peer 74191176370841578d396cc01f230f06. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.195:40257: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.293689 19967 raft_consensus.cc:2272] T 74e8c425db9e4e479c00e6f131fbafe2 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.295794 19967 tablet_replica.cc:333] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.296437 19967 raft_consensus.cc:2243] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.297271 19967 raft_consensus.cc:2272] T 70e5a681df5b42e98dac6b49a96773fe P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.299259 19967 tablet_replica.cc:333] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.299926 19967 raft_consensus.cc:2243] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.300375 19967 raft_consensus.cc:2272] T 8c4c252a57bc4facbe5d2e0ceb4c80ae P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.302366 19967 tablet_replica.cc:333] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.302963 19967 raft_consensus.cc:2243] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.303478 19967 raft_consensus.cc:2272] T ced82bb96f8a45e790d075986ff66cce P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.305503 19967 tablet_replica.cc:333] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.306120 19967 raft_consensus.cc:2243] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.306588 19967 raft_consensus.cc:2272] T 523db1e9cca74ffb938f22ae9105dbe1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.308874 19967 tablet_replica.cc:333] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.309650 19967 raft_consensus.cc:2243] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.310681 19967 raft_consensus.cc:2272] T 8a6337639944433881ce49be8f337030 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:50.310954 29955 consensus_peers.cc:597] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 -> Peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Couldn't send request to peer 74191176370841578d396cc01f230f06. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.195:40257: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.313006 19967 tablet_replica.cc:333] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.313759 19967 raft_consensus.cc:2243] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.314548 19967 raft_consensus.cc:2272] T 96641d56c48649d993e0eec66d091ee5 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.316685 19967 tablet_replica.cc:333] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.317260 19967 raft_consensus.cc:2243] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.317698 19967 raft_consensus.cc:2272] T 8807cef4b9f940359ad32d99dd28b5cd P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.319684 19967 tablet_replica.cc:333] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.320278 19967 raft_consensus.cc:2243] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.320762 19967 raft_consensus.cc:2272] T cc1a922f421149f48209b8e771fb3ca9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.323024 19967 tablet_replica.cc:333] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.323647 19967 raft_consensus.cc:2243] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.324107 19967 raft_consensus.cc:2272] T 1b6c48a3ec224675bd35d17f96f8fb13 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.326109 19967 tablet_replica.cc:333] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.326661 19967 raft_consensus.cc:2243] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.327106 19967 raft_consensus.cc:2272] T 6ebcaf79fc9947e5a4a8e8ed8eb3dab7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.329069 19967 tablet_replica.cc:333] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.329645 19967 raft_consensus.cc:2243] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.330065 19967 raft_consensus.cc:2272] T d4e21634252d40b196da2d7ac6d5faa5 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.331993 19967 tablet_replica.cc:333] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.332552 19967 raft_consensus.cc:2243] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.332986 30850 raft_consensus.cc:493] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2ba28923c4534c78903f759f0a4e1265)
I20260709 13:10:50.333133 19967 raft_consensus.cc:2272] T 244d599ec58f4e51ac20679e26a3c2b1 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.333451 30850 raft_consensus.cc:515] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75162022240a4fe1a113333e77964ca3" member_type: VOTER last_known_addr { host: "127.19.127.196" port: 46707 } } peers { permanent_uuid: "2ba28923c4534c78903f759f0a4e1265" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 35599 } } peers { permanent_uuid: "d08b1a18bb214fa1b08560d4d8149deb" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39603 } }
I20260709 13:10:50.335587 30850 leader_election.cc:290] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599), d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603)
I20260709 13:10:50.335940 19967 tablet_replica.cc:333] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.336869 19967 raft_consensus.cc:2243] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.337457 19967 raft_consensus.cc:2272] T ce794c74850e4dc6845e02da675fa199 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.340277 19967 tablet_replica.cc:333] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.341030 19967 raft_consensus.cc:2243] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:50.341308 29955 leader_election.cc:336] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2ba28923c4534c78903f759f0a4e1265 (127.19.127.193:35599): Network error: Client connection negotiation failed: client connection to 127.19.127.193:35599: connect: Connection refused (error 111)
I20260709 13:10:50.341552 19967 raft_consensus.cc:2272] T af3bf2e1f12f46c59f56c808bcfd5e78 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:50.342933 29955 leader_election.cc:336] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d08b1a18bb214fa1b08560d4d8149deb (127.19.127.194:39603): Network error: Client connection negotiation failed: client connection to 127.19.127.194:39603: connect: Connection refused (error 111)
I20260709 13:10:50.343405 29955 leader_election.cc:304] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75162022240a4fe1a113333e77964ca3; no voters: 2ba28923c4534c78903f759f0a4e1265, d08b1a18bb214fa1b08560d4d8149deb
I20260709 13:10:50.344239 30850 raft_consensus.cc:2749] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:50.344321 19967 tablet_replica.cc:333] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
W20260709 13:10:50.344627 29955 consensus_peers.cc:597] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 -> Peer 74191176370841578d396cc01f230f06 (127.19.127.195:40257): Couldn't send request to peer 74191176370841578d396cc01f230f06. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.195:40257: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.345175 19967 raft_consensus.cc:2243] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.346020 19967 raft_consensus.cc:2272] T b4e73d2d9e7449d4b87ea5ee4324c2c9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.348114 19967 tablet_replica.cc:333] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.348703 19967 raft_consensus.cc:2243] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.349130 19967 raft_consensus.cc:2272] T 0ef88786080e40e1a8a8b623e9c30106 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.351094 19967 tablet_replica.cc:333] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.351797 19967 raft_consensus.cc:2243] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.352631 19967 raft_consensus.cc:2272] T 0cfa1a88ac2b4cacb00c00d882d4cf1b P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.354610 19967 tablet_replica.cc:333] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.355181 19967 raft_consensus.cc:2243] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.355793 19967 raft_consensus.cc:2272] T 2a52b7d43c054ac485deaf19144dced6 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.358017 19967 tablet_replica.cc:333] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.358702 19967 raft_consensus.cc:2243] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.359434 19967 raft_consensus.cc:2272] T 7255b6d0c90146d7bb85a327da87cec0 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.361632 19967 tablet_replica.cc:333] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.362210 19967 raft_consensus.cc:2243] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.362663 19967 raft_consensus.cc:2272] T 125fc2e410394efba24979999a6785b7 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.364651 19967 tablet_replica.cc:333] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.365216 19967 raft_consensus.cc:2243] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.365617 19967 raft_consensus.cc:2272] T 6d91cac27fc9443b86fc3821776294de P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.367609 19967 tablet_replica.cc:333] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.368136 19967 raft_consensus.cc:2243] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.368916 19967 raft_consensus.cc:2272] T 4f9fb82d905c441bb50d70e1ce080213 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.370875 19967 tablet_replica.cc:333] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.371474 19967 raft_consensus.cc:2243] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.371897 19967 raft_consensus.cc:2272] T 277299b9ea1b456eb943ddf7be26fe05 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.373776 19967 tablet_replica.cc:333] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.374344 19967 raft_consensus.cc:2243] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.374773 19967 raft_consensus.cc:2272] T fac962e63d404b2a89aa0afd301ce161 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.376857 19967 tablet_replica.cc:333] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.377436 19967 raft_consensus.cc:2243] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.377875 19967 raft_consensus.cc:2272] T c6b780417ded422690ec3d46dda49a07 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.379978 19967 tablet_replica.cc:333] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.380527 19967 raft_consensus.cc:2243] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.380985 19967 raft_consensus.cc:2272] T 737f2cb09b0a48b1987133f213820765 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.382854 19967 tablet_replica.cc:333] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.383428 19967 raft_consensus.cc:2243] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.383863 19967 raft_consensus.cc:2272] T d182f1fe9db6430d8ff9a0c29b11c3f3 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.385793 19967 tablet_replica.cc:333] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.386356 19967 raft_consensus.cc:2243] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.387157 19967 raft_consensus.cc:2272] T 99a9fd11d5a348389207646896abb152 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.389232 19967 tablet_replica.cc:333] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.389780 19967 raft_consensus.cc:2243] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.390221 19967 raft_consensus.cc:2272] T 45a7caaa2edf4bafacd9da8900ebeeb2 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.392436 19967 tablet_replica.cc:333] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.393034 19967 raft_consensus.cc:2243] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.393481 19967 raft_consensus.cc:2272] T e02b4b0aee0c426ba054d6eac3eda348 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.395431 19967 tablet_replica.cc:333] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.395992 19967 raft_consensus.cc:2243] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.396435 19967 raft_consensus.cc:2272] T df180e07f62245aa9c117b1558657534 P 75162022240a4fe1a113333e77964ca3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.398403 19967 tablet_replica.cc:333] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.398921 19967 raft_consensus.cc:2243] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.399703 19967 raft_consensus.cc:2272] T 6ac2292232bc434b88b2786ed56f7adb P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.401695 19967 tablet_replica.cc:333] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.402230 19967 raft_consensus.cc:2243] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.403038 19967 raft_consensus.cc:2272] T 0bbc73ad9e6848228c0d2cae81843b7a P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.404978 19967 tablet_replica.cc:333] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.405511 19967 raft_consensus.cc:2243] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.406294 19967 raft_consensus.cc:2272] T 95db6009685e4a3d88d098ea07105ad7 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.408274 19967 tablet_replica.cc:333] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.408809 19967 raft_consensus.cc:2243] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.409595 19967 raft_consensus.cc:2272] T e86eb4af5624466bb2d9395a36632d71 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.411636 19967 tablet_replica.cc:333] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.412160 19967 raft_consensus.cc:2243] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.412904 19967 raft_consensus.cc:2272] T 5481f26841ab45efa35ded9872e310ee P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.414753 19967 tablet_replica.cc:333] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.415259 19967 raft_consensus.cc:2243] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.416067 19967 raft_consensus.cc:2272] T 0f4a836f7bfb414eb5bba33a3be51d4f P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.418044 19967 tablet_replica.cc:333] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.418546 19967 raft_consensus.cc:2243] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.418975 19967 raft_consensus.cc:2272] T 8405db80d77d4b8c9737d33de3ca1f74 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.420971 19967 tablet_replica.cc:333] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.421463 19967 raft_consensus.cc:2243] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.421868 19967 raft_consensus.cc:2272] T 8170c15c8bf942b596d738aa14bb6f80 P 75162022240a4fe1a113333e77964ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.423765 19967 tablet_replica.cc:333] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.424234 19967 raft_consensus.cc:2243] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.424639 19967 raft_consensus.cc:2272] T caad1999ec5545d9a8eb2ade21997cab P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.426542 19967 tablet_replica.cc:333] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3: stopping tablet replica
I20260709 13:10:50.426993 19967 raft_consensus.cc:2243] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:50.427459 19967 raft_consensus.cc:2272] T 0e543a21ae8243058d010255b40860e9 P 75162022240a4fe1a113333e77964ca3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.496168 19967 tablet_server.cc:196] TabletServer@127.19.127.196:0 shutdown complete.
I20260709 13:10:50.563531 19967 master.cc:562] Master@127.19.127.254:37437 shutting down...
I20260709 13:10:50.590929 19967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:50.591744 19967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:50.592103 19967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 917e8e70948348e9b76b5d231e54f10e: stopping tablet replica
I20260709 13:10:50.614979 19967 master.cc:584] Master@127.19.127.254:37437 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (29013 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:10:50.708031 19967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.127.254:36679
I20260709 13:10:50.709122 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:50.714890 30963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.716305 30966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.716452 30964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:50.717334 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:50.718370 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:50.718546 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:50.718655 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602650718645 us; error 0 us; skew 500 ppm
I20260709 13:10:50.719111 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:50.721580 19967 webserver.cc:533] Webserver started at http://127.19.127.254:38011/ using document root <none> and password file <none>
I20260709 13:10:50.722074 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:50.722230 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:50.722450 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:50.723929 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/master-0-root/instance:
uuid: "1643a4acf6ac4697b36e1e59f4998df8"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-s3n6"
I20260709 13:10:50.728225 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:10:50.731734 30971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.732494 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:10:50.732821 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/master-0-root
uuid: "1643a4acf6ac4697b36e1e59f4998df8"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-s3n6"
I20260709 13:10:50.733112 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:50.741847 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:50.743069 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:50.782606 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.254:36679
I20260709 13:10:50.782727 31022 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.254:36679 every 8 connection(s)
I20260709 13:10:50.786609 31023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.799523 31023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8: Bootstrap starting.
I20260709 13:10:50.806594 31023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.811376 31023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8: No bootstrap required, opened a new log
I20260709 13:10:50.813550 31023 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1643a4acf6ac4697b36e1e59f4998df8" member_type: VOTER }
I20260709 13:10:50.813934 31023 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.814108 31023 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1643a4acf6ac4697b36e1e59f4998df8, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.814615 31023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1643a4acf6ac4697b36e1e59f4998df8" member_type: VOTER }
I20260709 13:10:50.814998 31023 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:50.815191 31023 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:50.815560 31023 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.820300 31023 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1643a4acf6ac4697b36e1e59f4998df8" member_type: VOTER }
I20260709 13:10:50.820807 31023 leader_election.cc:304] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1643a4acf6ac4697b36e1e59f4998df8; no voters: 
I20260709 13:10:50.821897 31023 leader_election.cc:290] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:50.822414 31026 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.823885 31026 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 1 LEADER]: Becoming Leader. State: Replica: 1643a4acf6ac4697b36e1e59f4998df8, State: Running, Role: LEADER
I20260709 13:10:50.824601 31026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1643a4acf6ac4697b36e1e59f4998df8" member_type: VOTER }
I20260709 13:10:50.824915 31023 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:50.829748 31027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1643a4acf6ac4697b36e1e59f4998df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1643a4acf6ac4697b36e1e59f4998df8" member_type: VOTER } }
I20260709 13:10:50.830286 31028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1643a4acf6ac4697b36e1e59f4998df8. Latest consensus state: current_term: 1 leader_uuid: "1643a4acf6ac4697b36e1e59f4998df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1643a4acf6ac4697b36e1e59f4998df8" member_type: VOTER } }
I20260709 13:10:50.830704 31027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:50.831192 31028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:50.834004 31033 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:50.838994 31033 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:50.843032 19967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:50.848881 31033 catalog_manager.cc:1383] Generated new cluster ID: 8a0c065b9a844d20b7618122d6ab4472
I20260709 13:10:50.849189 31033 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:50.872454 31033 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:50.873802 31033 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:50.888707 31033 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8: Generated new TSK 0
I20260709 13:10:50.889338 31033 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:50.910205 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:50.916833 31044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.917721 31045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:50.919135 31047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:50.920187 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:50.921249 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:50.921432 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:50.921562 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602650921551 us; error 0 us; skew 500 ppm
I20260709 13:10:50.922097 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:50.924625 19967 webserver.cc:533] Webserver started at http://127.19.127.193:40669/ using document root <none> and password file <none>
I20260709 13:10:50.925129 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:50.925325 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:50.925619 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:50.926807 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-0-root/instance:
uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-s3n6"
I20260709 13:10:50.931443 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:10:50.934971 31052 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.935828 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:50.936154 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-s3n6"
I20260709 13:10:50.936419 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:50.962162 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:50.963506 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:50.965090 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:50.967519 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:50.967754 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:50.968015 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:50.968222 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:51.013075 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:46785
I20260709 13:10:51.013201 31114 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:46785 every 8 connection(s)
I20260709 13:10:51.018513 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:51.026098 31119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:51.027021 31120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:51.029660 31122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:51.030556 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:51.031965 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:51.032187 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:51.032407 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602651032388 us; error 0 us; skew 500 ppm
I20260709 13:10:51.032948 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:51.036649 19967 webserver.cc:533] Webserver started at http://127.19.127.194:41109/ using document root <none> and password file <none>
I20260709 13:10:51.037331 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:51.037581 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:51.037925 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:51.038214 31115 heartbeater.cc:344] Connected to a master server at 127.19.127.254:36679
I20260709 13:10:51.038714 31115 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:51.039546 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-1-root/instance:
uuid: "f75e2042be8e4dd48c0d83db4569c794"
format_stamp: "Formatted at 2026-07-09 13:10:51 on dist-test-slave-s3n6"
I20260709 13:10:51.039695 31115 heartbeater.cc:507] Master 127.19.127.254:36679 requested a full tablet report, sending...
I20260709 13:10:51.042129 30988 ts_manager.cc:194] Registered new tserver with Master: a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785)
I20260709 13:10:51.044056 30988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60776
I20260709 13:10:51.045243 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:10:51.049561 31127 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:51.050391 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:10:51.050750 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-1-root
uuid: "f75e2042be8e4dd48c0d83db4569c794"
format_stamp: "Formatted at 2026-07-09 13:10:51 on dist-test-slave-s3n6"
I20260709 13:10:51.051090 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:51.079149 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:51.080487 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:51.082023 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:51.084448 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:51.084686 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:51.084939 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:51.085110 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:51.130076 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.194:44527
I20260709 13:10:51.130208 31189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.194:44527 every 8 connection(s)
I20260709 13:10:51.136170 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:51.145020 31194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:51.146272 31193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:51.147473 31196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:51.148097 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:51.149257 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:51.149537 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:51.149761 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602651149719 us; error 0 us; skew 500 ppm
I20260709 13:10:51.150044 31190 heartbeater.cc:344] Connected to a master server at 127.19.127.254:36679
I20260709 13:10:51.150401 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:51.150475 31190 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:51.151654 31190 heartbeater.cc:507] Master 127.19.127.254:36679 requested a full tablet report, sending...
I20260709 13:10:51.153677 19967 webserver.cc:533] Webserver started at http://127.19.127.195:44277/ using document root <none> and password file <none>
I20260709 13:10:51.154126 30988 ts_manager.cc:194] Registered new tserver with Master: f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:51.154410 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:51.154728 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:51.155042 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:51.156101 30988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60780
I20260709 13:10:51.156688 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-2-root/instance:
uuid: "91ce733e5cc24d65b53fb2e607d8e541"
format_stamp: "Formatted at 2026-07-09 13:10:51 on dist-test-slave-s3n6"
I20260709 13:10:51.161792 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:10:51.165586 31201 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:51.166469 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:10:51.166815 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-2-root
uuid: "91ce733e5cc24d65b53fb2e607d8e541"
format_stamp: "Formatted at 2026-07-09 13:10:51 on dist-test-slave-s3n6"
I20260709 13:10:51.167160 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:51.180883 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:51.182266 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:51.183987 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:51.186537 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:51.186793 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:51.187060 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:51.187261 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:51.232558 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.195:38317
I20260709 13:10:51.232657 31263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.195:38317 every 8 connection(s)
I20260709 13:10:51.248813 31264 heartbeater.cc:344] Connected to a master server at 127.19.127.254:36679
I20260709 13:10:51.249266 31264 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:51.250046 31264 heartbeater.cc:507] Master 127.19.127.254:36679 requested a full tablet report, sending...
I20260709 13:10:51.252117 30988 ts_manager.cc:194] Registered new tserver with Master: 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:51.252766 19967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016274929s
I20260709 13:10:51.253989 30988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60794
I20260709 13:10:51.281996 30988 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60804:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:51.285035 30988 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:51.383355 31154 tablet_service.cc:1511] Processing CreateTablet for tablet 4be225566b9c4cdd87b16f24336912b9 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:51.383996 31155 tablet_service.cc:1511] Processing CreateTablet for tablet 690a0dd835254d60b27ba503878f564c (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:51.385241 31154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4be225566b9c4cdd87b16f24336912b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.386736 31155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 690a0dd835254d60b27ba503878f564c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.388106 31151 tablet_service.cc:1511] Processing CreateTablet for tablet fb50ff0d20914f53895e3e0aa1619d73 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:51.389876 31151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb50ff0d20914f53895e3e0aa1619d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.391181 31153 tablet_service.cc:1511] Processing CreateTablet for tablet b1885678eb9a4afdb5508e135f0eb6ff (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:51.389884 31152 tablet_service.cc:1511] Processing CreateTablet for tablet 577ed9a02a104b7aa8e47067a9d33129 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:51.392804 31080 tablet_service.cc:1511] Processing CreateTablet for tablet 690a0dd835254d60b27ba503878f564c (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:51.394265 31080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 690a0dd835254d60b27ba503878f564c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.394574 31075 tablet_service.cc:1511] Processing CreateTablet for tablet bca443435e19492d9a937605286180b3 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:51.396106 31153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1885678eb9a4afdb5508e135f0eb6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.396411 31075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bca443435e19492d9a937605286180b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.396495 31079 tablet_service.cc:1511] Processing CreateTablet for tablet 4be225566b9c4cdd87b16f24336912b9 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:51.397964 31079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4be225566b9c4cdd87b16f24336912b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.398132 31152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577ed9a02a104b7aa8e47067a9d33129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.400559 31078 tablet_service.cc:1511] Processing CreateTablet for tablet b1885678eb9a4afdb5508e135f0eb6ff (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:51.402048 31078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1885678eb9a4afdb5508e135f0eb6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.404552 31077 tablet_service.cc:1511] Processing CreateTablet for tablet 577ed9a02a104b7aa8e47067a9d33129 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:51.407466 31076 tablet_service.cc:1511] Processing CreateTablet for tablet fb50ff0d20914f53895e3e0aa1619d73 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:51.407964 31279 tablet_bootstrap.cc:492] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:51.409006 31076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb50ff0d20914f53895e3e0aa1619d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.410763 31077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577ed9a02a104b7aa8e47067a9d33129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.431322 31227 tablet_service.cc:1511] Processing CreateTablet for tablet b1885678eb9a4afdb5508e135f0eb6ff (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:51.432183 31229 tablet_service.cc:1511] Processing CreateTablet for tablet 690a0dd835254d60b27ba503878f564c (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:51.432910 31227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1885678eb9a4afdb5508e135f0eb6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.433571 31229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 690a0dd835254d60b27ba503878f564c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.433826 31279 tablet_bootstrap.cc:654] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.436228 31226 tablet_service.cc:1511] Processing CreateTablet for tablet 577ed9a02a104b7aa8e47067a9d33129 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:51.436784 31228 tablet_service.cc:1511] Processing CreateTablet for tablet 4be225566b9c4cdd87b16f24336912b9 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:51.437309 31074 tablet_service.cc:1511] Processing CreateTablet for tablet 728f8355951f41aab19eebe146d1cf0d (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:51.438747 31074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 728f8355951f41aab19eebe146d1cf0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.447711 31226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577ed9a02a104b7aa8e47067a9d33129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.448382 31155 tablet_service.cc:1511] Processing CreateTablet for tablet bca443435e19492d9a937605286180b3 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:51.450395 31155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bca443435e19492d9a937605286180b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.453581 31073 tablet_service.cc:1511] Processing CreateTablet for tablet 66dc6ecb9c4c40b19c759263267cf5e7 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:51.453601 31150 tablet_service.cc:1511] Processing CreateTablet for tablet 728f8355951f41aab19eebe146d1cf0d (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:51.455407 31150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 728f8355951f41aab19eebe146d1cf0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.456555 31225 tablet_service.cc:1511] Processing CreateTablet for tablet fb50ff0d20914f53895e3e0aa1619d73 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:51.458110 31073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66dc6ecb9c4c40b19c759263267cf5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.464665 31224 tablet_service.cc:1511] Processing CreateTablet for tablet bca443435e19492d9a937605286180b3 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:51.465090 31223 tablet_service.cc:1511] Processing CreateTablet for tablet 728f8355951f41aab19eebe146d1cf0d (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:51.458077 31225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb50ff0d20914f53895e3e0aa1619d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.467079 31223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 728f8355951f41aab19eebe146d1cf0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.470528 31224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bca443435e19492d9a937605286180b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.471206 31149 tablet_service.cc:1511] Processing CreateTablet for tablet 66dc6ecb9c4c40b19c759263267cf5e7 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:51.472698 31228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4be225566b9c4cdd87b16f24336912b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.472741 31149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66dc6ecb9c4c40b19c759263267cf5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.473543 31222 tablet_service.cc:1511] Processing CreateTablet for tablet 66dc6ecb9c4c40b19c759263267cf5e7 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:51.474963 31222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66dc6ecb9c4c40b19c759263267cf5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.478981 31078 tablet_service.cc:1511] Processing CreateTablet for tablet 75e9c2083400466995f59a7c4758e58e (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:51.480463 31078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e9c2083400466995f59a7c4758e58e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.527307 31221 tablet_service.cc:1511] Processing CreateTablet for tablet 75e9c2083400466995f59a7c4758e58e (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:51.528916 31221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e9c2083400466995f59a7c4758e58e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.541632 31149 tablet_service.cc:1511] Processing CreateTablet for tablet 75e9c2083400466995f59a7c4758e58e (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:51.543111 31149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e9c2083400466995f59a7c4758e58e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.570397 31280 tablet_bootstrap.cc:492] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:51.573249 31279 tablet_bootstrap.cc:492] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
I20260709 13:10:51.573895 31279 ts_tablet_manager.cc:1403] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.166s	user 0.035s	sys 0.007s
I20260709 13:10:51.577273 31279 raft_consensus.cc:359] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.578076 31279 raft_consensus.cc:385] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.578395 31279 raft_consensus.cc:740] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.579342 31279 consensus_queue.cc:260] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.584376 31153 tablet_service.cc:1511] Processing CreateTablet for tablet 8e6935f128074cf6bcc6a2294094f231 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:51.584676 31075 tablet_service.cc:1511] Processing CreateTablet for tablet 8e6935f128074cf6bcc6a2294094f231 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:51.585871 31153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6935f128074cf6bcc6a2294094f231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.592377 31075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6935f128074cf6bcc6a2294094f231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.604831 31190 heartbeater.cc:499] Master 127.19.127.254:36679 was elected leader, sending a full tablet report...
I20260709 13:10:51.607585 31280 tablet_bootstrap.cc:654] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.618643 31227 tablet_service.cc:1511] Processing CreateTablet for tablet 8e6935f128074cf6bcc6a2294094f231 (DEFAULT_TABLE table=test-workload [id=4dcf2ab12bcc4c2ab4113932bab868d1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:51.621025 31282 tablet_bootstrap.cc:492] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:51.621233 31227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6935f128074cf6bcc6a2294094f231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.622686 31279 ts_tablet_manager.cc:1434] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.006s
I20260709 13:10:51.623863 31279 tablet_bootstrap.cc:492] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:51.629161 31282 tablet_bootstrap.cc:654] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.643694 31279 tablet_bootstrap.cc:654] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.648272 31282 tablet_bootstrap.cc:492] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:51.648785 31282 ts_tablet_manager.cc:1403] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.005s
I20260709 13:10:51.650341 31279 tablet_bootstrap.cc:492] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
I20260709 13:10:51.651037 31279 ts_tablet_manager.cc:1403] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.027s	user 0.002s	sys 0.010s
I20260709 13:10:51.651479 31282 raft_consensus.cc:359] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.651882 31280 tablet_bootstrap.cc:492] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:51.652143 31282 raft_consensus.cc:385] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.652545 31282 raft_consensus.cc:740] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.652575 31280 ts_tablet_manager.cc:1403] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.082s	user 0.004s	sys 0.013s
I20260709 13:10:51.653661 31282 consensus_queue.cc:260] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.654640 31279 raft_consensus.cc:359] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.655565 31279 raft_consensus.cc:385] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.655962 31264 heartbeater.cc:499] Master 127.19.127.254:36679 was elected leader, sending a full tablet report...
I20260709 13:10:51.655905 31279 raft_consensus.cc:740] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.656479 31280 raft_consensus.cc:359] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.657428 31282 ts_tablet_manager.cc:1434] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.005s
I20260709 13:10:51.656900 31279 consensus_queue.cc:260] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.657397 31280 raft_consensus.cc:385] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.658159 31280 raft_consensus.cc:740] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.659771 31279 ts_tablet_manager.cc:1434] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:10:51.659797 31280 consensus_queue.cc:260] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.661171 31279 tablet_bootstrap.cc:492] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:51.658805 31282 tablet_bootstrap.cc:492] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:51.669319 31282 tablet_bootstrap.cc:654] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.670048 31279 tablet_bootstrap.cc:654] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.672688 31280 ts_tablet_manager.cc:1434] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:10:51.673704 31115 heartbeater.cc:499] Master 127.19.127.254:36679 was elected leader, sending a full tablet report...
I20260709 13:10:51.676004 31280 tablet_bootstrap.cc:492] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:51.685302 31280 tablet_bootstrap.cc:654] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.691772 31279 tablet_bootstrap.cc:492] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
I20260709 13:10:51.692440 31279 ts_tablet_manager.cc:1403] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.018s
I20260709 13:10:51.692865 31280 tablet_bootstrap.cc:492] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:51.693126 31282 tablet_bootstrap.cc:492] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:51.693588 31280 ts_tablet_manager.cc:1403] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:10:51.693815 31282 ts_tablet_manager.cc:1403] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.000s
I20260709 13:10:51.695885 31279 raft_consensus.cc:359] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.696880 31279 raft_consensus.cc:385] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.697266 31279 raft_consensus.cc:740] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.697046 31280 raft_consensus.cc:359] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.698129 31280 raft_consensus.cc:385] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.698652 31280 raft_consensus.cc:740] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.698331 31279 consensus_queue.cc:260] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.699023 31282 raft_consensus.cc:359] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.699939 31282 raft_consensus.cc:385] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.699571 31280 consensus_queue.cc:260] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.700325 31282 raft_consensus.cc:740] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.700857 31279 ts_tablet_manager.cc:1434] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:10:51.701452 31282 consensus_queue.cc:260] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.703161 31280 ts_tablet_manager.cc:1434] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:10:51.704100 31282 ts_tablet_manager.cc:1434] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:10:51.704365 31280 tablet_bootstrap.cc:492] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:51.704388 31279 tablet_bootstrap.cc:492] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:51.709868 31282 tablet_bootstrap.cc:492] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:51.712001 31280 tablet_bootstrap.cc:654] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.712004 31279 tablet_bootstrap.cc:654] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.718269 31282 tablet_bootstrap.cc:654] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.733372 31280 tablet_bootstrap.cc:492] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:51.733649 31282 tablet_bootstrap.cc:492] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:51.734310 31280 ts_tablet_manager.cc:1403] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.006s
I20260709 13:10:51.735292 31282 ts_tablet_manager.cc:1403] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.007s
I20260709 13:10:51.736941 31279 tablet_bootstrap.cc:492] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
I20260709 13:10:51.737569 31279 ts_tablet_manager.cc:1403] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.006s
I20260709 13:10:51.737692 31280 raft_consensus.cc:359] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.738576 31280 raft_consensus.cc:385] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.738938 31280 raft_consensus.cc:740] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.739837 31282 raft_consensus.cc:359] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.739832 31280 consensus_queue.cc:260] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.740804 31282 raft_consensus.cc:385] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.741127 31279 raft_consensus.cc:359] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.742360 31279 raft_consensus.cc:385] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.742731 31279 raft_consensus.cc:740] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.743026 31280 ts_tablet_manager.cc:1434] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:10:51.743636 31279 consensus_queue.cc:260] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.744460 31280 tablet_bootstrap.cc:492] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:51.746060 31279 ts_tablet_manager.cc:1434] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:10:51.747133 31279 tablet_bootstrap.cc:492] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:51.741199 31282 raft_consensus.cc:740] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.750566 31282 consensus_queue.cc:260] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.753044 31280 tablet_bootstrap.cc:654] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.753866 31282 ts_tablet_manager.cc:1434] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:10:51.755196 31282 tablet_bootstrap.cc:492] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:51.765543 31279 tablet_bootstrap.cc:654] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.770442 31287 raft_consensus.cc:493] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.771262 31287 raft_consensus.cc:515] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.776417 31283 raft_consensus.cc:493] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.777555 31287 leader_election.cc:290] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:51.778860 31283 raft_consensus.cc:515] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.792431 31282 tablet_bootstrap.cc:654] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.806396 31280 tablet_bootstrap.cc:492] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:51.795929 31283 leader_election.cc:290] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:51.809444 31280 ts_tablet_manager.cc:1403] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.065s	user 0.025s	sys 0.011s
I20260709 13:10:51.812014 31279 tablet_bootstrap.cc:492] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
I20260709 13:10:51.812999 31279 ts_tablet_manager.cc:1403] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.066s	user 0.015s	sys 0.009s
I20260709 13:10:51.813162 31283 raft_consensus.cc:493] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.813601 31280 raft_consensus.cc:359] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.813951 31283 raft_consensus.cc:515] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.814553 31280 raft_consensus.cc:385] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.815104 31280 raft_consensus.cc:740] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.816146 31280 consensus_queue.cc:260] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.816527 31279 raft_consensus.cc:359] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.817162 31283 leader_election.cc:290] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:51.817430 31279 raft_consensus.cc:385] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.817938 31279 raft_consensus.cc:740] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.818344 31288 raft_consensus.cc:493] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.818943 31279 consensus_queue.cc:260] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.819008 31288 raft_consensus.cc:515] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.825677 31280 ts_tablet_manager.cc:1434] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:10:51.828033 31280 tablet_bootstrap.cc:492] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
W20260709 13:10:51.838609 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:51.832803 31288 leader_election.cc:290] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:51.859239 31090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bca443435e19492d9a937605286180b3" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:51.860507 31090 raft_consensus.cc:2468] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 0.
I20260709 13:10:51.861335 31282 tablet_bootstrap.cc:492] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:51.861948 31282 ts_tablet_manager.cc:1403] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.107s	user 0.023s	sys 0.004s
I20260709 13:10:51.862627 31202 leader_election.cc:304] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:10:51.863648 31287 raft_consensus.cc:2804] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.864001 31287 raft_consensus.cc:493] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.864334 31287 raft_consensus.cc:3060] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.865358 31282 raft_consensus.cc:359] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.866451 31282 raft_consensus.cc:385] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.866852 31282 raft_consensus.cc:740] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.869556 31282 consensus_queue.cc:260] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.873001 31287 raft_consensus.cc:515] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.877478 31090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bca443435e19492d9a937605286180b3" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:51.878505 31090 raft_consensus.cc:3060] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.880995 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:51.881341 31088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:51.882350 31088 raft_consensus.cc:2468] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 0.
I20260709 13:10:51.883569 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:51.883854 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
W20260709 13:10:51.885758 31129 leader_election.cc:343] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.885844 31128 leader_election.cc:304] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:51.889319 31304 raft_consensus.cc:2804] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.889810 31304 raft_consensus.cc:493] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.890208 31304 raft_consensus.cc:3060] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:51.893229 31128 leader_election.cc:343] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:51.893442 31129 leader_election.cc:343] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.886231 31280 tablet_bootstrap.cc:654] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.876433 31282 ts_tablet_manager.cc:1434] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 13:10:51.895555 31128 leader_election.cc:304] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f75e2042be8e4dd48c0d83db4569c794; no voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe
I20260709 13:10:51.896865 31282 tablet_bootstrap.cc:492] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:51.895658 31165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bca443435e19492d9a937605286180b3" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794" is_pre_election: true
I20260709 13:10:51.898231 31283 raft_consensus.cc:2749] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.899111 31165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bca443435e19492d9a937605286180b3" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794" is_pre_election: true
I20260709 13:10:51.899502 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bca443435e19492d9a937605286180b3" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:10:51.900803 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bca443435e19492d9a937605286180b3" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
W20260709 13:10:51.900875 31055 leader_election.cc:343] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.901875 31239 raft_consensus.cc:2393] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a89ddf02312a44619eae5b9d6f7ecdbe in current term 1: Already voted for candidate 91ce733e5cc24d65b53fb2e607d8e541 in this term.
W20260709 13:10:51.902475 31204 leader_election.cc:343] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.903384 31282 tablet_bootstrap.cc:654] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:51.903484 31204 leader_election.cc:343] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.904618 31054 leader_election.cc:304] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794
I20260709 13:10:51.904539 31304 raft_consensus.cc:515] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.906754 31279 ts_tablet_manager.cc:1434] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.093s	user 0.014s	sys 0.020s
I20260709 13:10:51.907932 31279 tablet_bootstrap.cc:492] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:51.908921 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:51.909127 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:51.909972 31089 raft_consensus.cc:3060] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:51.910871 31129 leader_election.cc:343] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.895520 31287 leader_election.cc:290] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:51.909186 31304 leader_election.cc:290] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:51.918505 31279 tablet_bootstrap.cc:654] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.921379 31282 tablet_bootstrap.cc:492] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:51.921617 31090 raft_consensus.cc:2468] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 1.
I20260709 13:10:51.922231 31282 ts_tablet_manager.cc:1403] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.005s
I20260709 13:10:51.923156 31202 leader_election.cc:304] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: f75e2042be8e4dd48c0d83db4569c794
I20260709 13:10:51.924539 31287 raft_consensus.cc:2804] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.925021 31287 raft_consensus.cc:697] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:10:51.925148 31279 tablet_bootstrap.cc:492] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
I20260709 13:10:51.925695 31279 ts_tablet_manager.cc:1403] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.004s
I20260709 13:10:51.925925 31287 consensus_queue.cc:237] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.929046 31279 raft_consensus.cc:359] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.922240 31288 raft_consensus.cc:2749] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.929893 31279 raft_consensus.cc:385] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.930253 31279 raft_consensus.cc:740] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.930557 31089 raft_consensus.cc:2468] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
I20260709 13:10:51.931195 31279 consensus_queue.cc:260] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.933425 31128 leader_election.cc:304] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:10:51.934254 31279 ts_tablet_manager.cc:1434] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:10:51.935386 31283 raft_consensus.cc:2804] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.935482 31304 raft_consensus.cc:493] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.936398 31304 raft_consensus.cc:515] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.941623 31090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:51.942145 31304 leader_election.cc:290] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:51.926775 31282 raft_consensus.cc:359] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.944093 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:51.944217 31282 raft_consensus.cc:385] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.944854 31239 raft_consensus.cc:2468] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 0.
I20260709 13:10:51.942350 31090 raft_consensus.cc:2468] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 0.
I20260709 13:10:51.946419 31129 leader_election.cc:304] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:51.947237 31279 tablet_bootstrap.cc:492] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:51.947516 31304 raft_consensus.cc:493] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.948046 31304 raft_consensus.cc:515] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.949144 31313 raft_consensus.cc:2804] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.949862 31313 raft_consensus.cc:493] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.950392 31313 raft_consensus.cc:3060] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.951388 31283 raft_consensus.cc:697] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:51.951694 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4be225566b9c4cdd87b16f24336912b9" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:51.952462 31090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4be225566b9c4cdd87b16f24336912b9" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:51.952406 31283 consensus_queue.cc:237] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.952659 31239 raft_consensus.cc:2468] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 0.
W20260709 13:10:51.954227 31128 leader_election.cc:343] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.954821 31304 leader_election.cc:290] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:51.955077 31129 leader_election.cc:304] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: a89ddf02312a44619eae5b9d6f7ecdbe
I20260709 13:10:51.956665 31304 raft_consensus.cc:2804] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.957234 31304 raft_consensus.cc:493] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.957686 31304 raft_consensus.cc:3060] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.957777 31280 tablet_bootstrap.cc:492] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:51.958732 31280 ts_tablet_manager.cc:1403] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.131s	user 0.022s	sys 0.025s
I20260709 13:10:51.961340 31279 tablet_bootstrap.cc:654] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.963022 31280 raft_consensus.cc:359] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.944849 31282 raft_consensus.cc:740] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.971513 31280 raft_consensus.cc:385] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.972046 31280 raft_consensus.cc:740] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.972183 31282 consensus_queue.cc:260] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.973318 31280 consensus_queue.cc:260] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.974678 31282 ts_tablet_manager.cc:1434] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 13:10:51.975931 31282 tablet_bootstrap.cc:492] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:51.976754 31280 ts_tablet_manager.cc:1434] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:10:51.978358 31280 tablet_bootstrap.cc:492] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:51.983259 31313 raft_consensus.cc:515] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.986603 31090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:51.987222 31090 raft_consensus.cc:3060] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.976791 30988 catalog_manager.cc:5697] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:51.995530 31280 tablet_bootstrap.cc:654] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.991375 30987 catalog_manager.cc:5697] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.996238 31304 raft_consensus.cc:515] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:51.990075 31288 raft_consensus.cc:493] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.997946 31288 raft_consensus.cc:515] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.000057 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4be225566b9c4cdd87b16f24336912b9" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:52.001674 31239 raft_consensus.cc:3060] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.002938 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794" is_pre_election: true
I20260709 13:10:52.002496 31288 leader_election.cc:290] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.006177 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4be225566b9c4cdd87b16f24336912b9" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:52.007117 31089 raft_consensus.cc:3060] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.007807 31238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
W20260709 13:10:52.009781 31054 leader_election.cc:343] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:52.012238 31055 leader_election.cc:343] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.012890 31055 leader_election.cc:304] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794
I20260709 13:10:51.994975 31090 raft_consensus.cc:2468] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
I20260709 13:10:52.015029 31128 leader_election.cc:304] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:52.016013 31283 raft_consensus.cc:2804] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.016585 31239 raft_consensus.cc:2468] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
I20260709 13:10:52.017287 31288 raft_consensus.cc:2749] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.018201 31129 leader_election.cc:304] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:52.032218 31304 leader_election.cc:290] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.034794 31304 raft_consensus.cc:2804] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.035836 31304 raft_consensus.cc:697] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:52.035141 31283 raft_consensus.cc:697] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:52.039842 31313 leader_election.cc:290] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.040369 31304 consensus_queue.cc:237] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.041597 31282 tablet_bootstrap.cc:654] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.041921 31279 tablet_bootstrap.cc:492] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
I20260709 13:10:52.042582 31279 ts_tablet_manager.cc:1403] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.005s
I20260709 13:10:52.044734 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:52.045559 31239 raft_consensus.cc:3060] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.045706 31089 raft_consensus.cc:2468] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
I20260709 13:10:52.041703 31283 consensus_queue.cc:237] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.048491 31319 raft_consensus.cc:493] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.049553 31319 raft_consensus.cc:515] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.050468 31280 tablet_bootstrap.cc:492] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:52.051080 31280 ts_tablet_manager.cc:1403] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.073s	user 0.022s	sys 0.021s
I20260709 13:10:52.053066 31238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:52.053803 31238 raft_consensus.cc:2468] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 0.
I20260709 13:10:52.048424 31279 raft_consensus.cc:359] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.054509 31279 raft_consensus.cc:385] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.054836 31279 raft_consensus.cc:740] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.055641 31279 consensus_queue.cc:260] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.057212 31319 leader_election.cc:290] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.060341 31280 raft_consensus.cc:359] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.062016 31280 raft_consensus.cc:385] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.062438 31129 leader_election.cc:304] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:52.062404 31280 raft_consensus.cc:740] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.068158 31239 raft_consensus.cc:2468] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
I20260709 13:10:52.066159 31319 raft_consensus.cc:2804] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.062733 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:52.071168 31319 raft_consensus.cc:493] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.074815 31319 raft_consensus.cc:3060] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:52.075654 31128 leader_election.cc:343] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.078511 31282 tablet_bootstrap.cc:492] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:52.079025 31282 ts_tablet_manager.cc:1403] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.103s	user 0.008s	sys 0.015s
I20260709 13:10:52.071844 31280 consensus_queue.cc:260] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.081779 31280 ts_tablet_manager.cc:1434] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.003s
I20260709 13:10:52.074147 31279 ts_tablet_manager.cc:1434] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:10:52.083019 31280 tablet_bootstrap.cc:492] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:52.082227 31282 raft_consensus.cc:359] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.087862 31279 tablet_bootstrap.cc:492] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:52.088044 31282 raft_consensus.cc:385] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.088462 31282 raft_consensus.cc:740] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.088910 30987 catalog_manager.cc:5697] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.089433 31282 consensus_queue.cc:260] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.092176 31282 ts_tablet_manager.cc:1434] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:10:52.093372 31282 tablet_bootstrap.cc:492] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:52.095221 31279 tablet_bootstrap.cc:654] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.095225 31280 tablet_bootstrap.cc:654] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.101203 31282 tablet_bootstrap.cc:654] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.119570 30987 catalog_manager.cc:5697] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.128863 31319 raft_consensus.cc:515] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.138839 31319 leader_election.cc:290] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.139676 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:52.140058 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:52.140674 31239 raft_consensus.cc:3060] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.143378 31279 tablet_bootstrap.cc:492] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
W20260709 13:10:52.143504 31128 leader_election.cc:343] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.144024 31279 ts_tablet_manager.cc:1403] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.056s	user 0.018s	sys 0.034s
I20260709 13:10:52.146373 31279 raft_consensus.cc:359] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.148237 31279 raft_consensus.cc:385] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.148792 31239 raft_consensus.cc:2468] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
I20260709 13:10:52.148921 31279 raft_consensus.cc:740] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.149039 31282 tablet_bootstrap.cc:492] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:52.150211 31129 leader_election.cc:304] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: a89ddf02312a44619eae5b9d6f7ecdbe
I20260709 13:10:52.150223 31282 ts_tablet_manager.cc:1403] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.057s	user 0.024s	sys 0.019s
I20260709 13:10:52.151453 31319 raft_consensus.cc:2804] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.154979 31319 raft_consensus.cc:697] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:52.155771 31319 consensus_queue.cc:237] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.156840 31282 raft_consensus.cc:359] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.157490 31282 raft_consensus.cc:385] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.157755 31282 raft_consensus.cc:740] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.152637 31280 tablet_bootstrap.cc:492] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:52.158540 31282 consensus_queue.cc:260] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.151583 31279 consensus_queue.cc:260] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.159679 31280 ts_tablet_manager.cc:1403] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.077s	user 0.013s	sys 0.044s
I20260709 13:10:52.160996 31279 ts_tablet_manager.cc:1434] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:10:52.161151 31282 ts_tablet_manager.cc:1434] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:52.162113 31279 tablet_bootstrap.cc:492] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:52.162348 31282 tablet_bootstrap.cc:492] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:52.163630 31280 raft_consensus.cc:359] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.164500 31280 raft_consensus.cc:385] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.164865 31280 raft_consensus.cc:740] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.166028 31280 consensus_queue.cc:260] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.169255 31280 ts_tablet_manager.cc:1434] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:10:52.173038 31279 tablet_bootstrap.cc:654] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.173280 31280 tablet_bootstrap.cc:492] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:52.174803 30987 catalog_manager.cc:5697] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.178386 31319 raft_consensus.cc:493] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.179167 31319 raft_consensus.cc:515] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.181016 31280 tablet_bootstrap.cc:654] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.183718 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:52.184652 31089 raft_consensus.cc:2468] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 0.
I20260709 13:10:52.185434 31319 leader_election.cc:290] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.185659 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:52.186147 31128 leader_election.cc:304] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 
W20260709 13:10:52.187327 31129 leader_election.cc:343] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.190721 31319 raft_consensus.cc:2804] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.191759 31319 raft_consensus.cc:493] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.193033 31282 tablet_bootstrap.cc:654] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.196002 31280 tablet_bootstrap.cc:492] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:52.196762 31280 ts_tablet_manager.cc:1403] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.008s
I20260709 13:10:52.200508 31280 raft_consensus.cc:359] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.201733 31280 raft_consensus.cc:385] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.192130 31319 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.207669 31279 tablet_bootstrap.cc:492] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
I20260709 13:10:52.208117 31279 ts_tablet_manager.cc:1403] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.046s	user 0.024s	sys 0.010s
I20260709 13:10:52.208077 31280 raft_consensus.cc:740] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.210466 31280 consensus_queue.cc:260] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.210701 31279 raft_consensus.cc:359] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.211400 31279 raft_consensus.cc:385] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.211676 31279 raft_consensus.cc:740] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.212642 31280 ts_tablet_manager.cc:1434] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:10:52.212455 31279 consensus_queue.cc:260] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.214918 31280 tablet_bootstrap.cc:492] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:52.214918 31279 ts_tablet_manager.cc:1434] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:52.216063 31279 tablet_bootstrap.cc:492] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794: Bootstrap starting.
I20260709 13:10:52.219394 31282 tablet_bootstrap.cc:492] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:52.219974 31282 ts_tablet_manager.cc:1403] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.020s
I20260709 13:10:52.221305 31319 raft_consensus.cc:515] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.222815 31282 raft_consensus.cc:359] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.223537 31282 raft_consensus.cc:385] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.223613 31279 tablet_bootstrap.cc:654] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.223872 31282 raft_consensus.cc:740] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.224483 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:52.224742 31282 consensus_queue.cc:260] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.225149 31089 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.221305 31280 tablet_bootstrap.cc:654] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.231653 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:52.233412 31089 raft_consensus.cc:2468] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
W20260709 13:10:52.233887 31129 leader_election.cc:343] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.234776 31319 leader_election.cc:290] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.234920 31128 leader_election.cc:304] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:10:52.238983 31319 raft_consensus.cc:2804] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.239698 31282 ts_tablet_manager.cc:1434] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:10:52.239980 31319 raft_consensus.cc:697] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:52.240814 31282 tablet_bootstrap.cc:492] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:52.241340 31319 consensus_queue.cc:237] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.247701 31282 tablet_bootstrap.cc:654] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.257442 31280 tablet_bootstrap.cc:492] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:52.257965 31282 tablet_bootstrap.cc:492] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:52.258026 31279 tablet_bootstrap.cc:492] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794: No bootstrap required, opened a new log
I20260709 13:10:52.258571 31282 ts_tablet_manager.cc:1403] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:10:52.258735 31279 ts_tablet_manager.cc:1403] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794: Time spent bootstrapping tablet: real 0.043s	user 0.027s	sys 0.012s
I20260709 13:10:52.258754 30986 catalog_manager.cc:5697] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.257969 31280 ts_tablet_manager.cc:1403] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.004s
I20260709 13:10:52.261564 31282 raft_consensus.cc:359] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.261832 31279 raft_consensus.cc:359] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.262296 31282 raft_consensus.cc:385] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.262651 31279 raft_consensus.cc:385] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.262753 31282 raft_consensus.cc:740] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.263078 31279 raft_consensus.cc:740] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.263796 31282 consensus_queue.cc:260] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.263934 31279 consensus_queue.cc:260] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.265473 31280 raft_consensus.cc:359] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.266168 31279 ts_tablet_manager.cc:1434] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:52.266227 31280 raft_consensus.cc:385] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.267426 31280 raft_consensus.cc:740] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.267882 31282 ts_tablet_manager.cc:1434] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:10:52.268249 31280 consensus_queue.cc:260] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.268903 31282 tablet_bootstrap.cc:492] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541: Bootstrap starting.
I20260709 13:10:52.270555 31280 ts_tablet_manager.cc:1434] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:10:52.271588 31280 tablet_bootstrap.cc:492] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:52.275588 31282 tablet_bootstrap.cc:654] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.277251 31280 tablet_bootstrap.cc:654] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.281211 31282 tablet_bootstrap.cc:492] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541: No bootstrap required, opened a new log
I20260709 13:10:52.281728 31282 ts_tablet_manager.cc:1403] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 13:10:52.283990 31287 raft_consensus.cc:493] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.284359 31282 raft_consensus.cc:359] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.284520 31287 raft_consensus.cc:515] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.285133 31282 raft_consensus.cc:385] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.285470 31282 raft_consensus.cc:740] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.286206 31282 consensus_queue.cc:260] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.287007 31287 leader_election.cc:290] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:52.288667 31282 ts_tablet_manager.cc:1434] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:52.289137 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:52.289939 31089 raft_consensus.cc:2468] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 0.
I20260709 13:10:52.291731 31202 leader_election.cc:304] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:10:52.293347 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794" is_pre_election: true
I20260709 13:10:52.294090 31164 raft_consensus.cc:2468] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 0.
I20260709 13:10:52.294570 31280 tablet_bootstrap.cc:492] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: No bootstrap required, opened a new log
I20260709 13:10:52.295145 31280 ts_tablet_manager.cc:1403] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.000s
I20260709 13:10:52.297633 31287 raft_consensus.cc:2804] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.298084 31287 raft_consensus.cc:493] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.298485 31287 raft_consensus.cc:3060] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.298207 31280 raft_consensus.cc:359] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.298934 31280 raft_consensus.cc:385] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.299224 31280 raft_consensus.cc:740] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.299919 31280 consensus_queue.cc:260] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.301945 31280 ts_tablet_manager.cc:1434] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:52.306186 31287 raft_consensus.cc:515] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.308300 31287 leader_election.cc:290] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:52.309192 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:52.309418 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:10:52.309791 31089 raft_consensus.cc:3060] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.310103 31164 raft_consensus.cc:3060] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.315860 31164 raft_consensus.cc:2468] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 1.
I20260709 13:10:52.316586 31089 raft_consensus.cc:2468] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 1.
I20260709 13:10:52.317123 31204 leader_election.cc:304] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:52.318003 31287 raft_consensus.cc:2804] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.318508 31287 raft_consensus.cc:697] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:10:52.319345 31287 consensus_queue.cc:237] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.320008 31308 consensus_queue.cc:1048] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.332545 31164 raft_consensus.cc:3060] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.337353 30988 catalog_manager.cc:5697] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:52.349696 31343 consensus_queue.cc:1048] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.385382 31343 raft_consensus.cc:493] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.385877 31343 raft_consensus.cc:515] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.387694 31343 leader_election.cc:290] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:52.388732 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e9c2083400466995f59a7c4758e58e" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:52.388824 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e9c2083400466995f59a7c4758e58e" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794" is_pre_election: true
I20260709 13:10:52.389446 31089 raft_consensus.cc:2468] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 0.
I20260709 13:10:52.389532 31164 raft_consensus.cc:2468] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 0.
I20260709 13:10:52.390507 31202 leader_election.cc:304] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:10:52.391263 31343 raft_consensus.cc:2804] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.391638 31343 raft_consensus.cc:493] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.391918 31343 raft_consensus.cc:3060] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.397886 31343 raft_consensus.cc:515] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.399730 31343 leader_election.cc:290] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:52.400558 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e9c2083400466995f59a7c4758e58e" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:52.400771 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e9c2083400466995f59a7c4758e58e" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:10:52.401140 31089 raft_consensus.cc:3060] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.401432 31164 raft_consensus.cc:3060] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.406728 31089 raft_consensus.cc:2468] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 1.
I20260709 13:10:52.406807 31164 raft_consensus.cc:2468] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 1.
I20260709 13:10:52.407970 31202 leader_election.cc:304] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:10:52.408728 31343 raft_consensus.cc:2804] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.409258 31343 raft_consensus.cc:697] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:10:52.409994 31343 consensus_queue.cc:237] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.417042 30986 catalog_manager.cc:5697] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:52.424376 31288 raft_consensus.cc:493] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.424949 31288 raft_consensus.cc:515] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.426963 31288 leader_election.cc:290] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.427711 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794" is_pre_election: true
I20260709 13:10:52.428051 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:52.428498 31164 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a89ddf02312a44619eae5b9d6f7ecdbe in term 0.
I20260709 13:10:52.428676 31239 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a89ddf02312a44619eae5b9d6f7ecdbe in term 0.
I20260709 13:10:52.429587 31055 leader_election.cc:304] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:52.430312 31288 raft_consensus.cc:2804] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.430709 31288 raft_consensus.cc:493] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.430994 31288 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.436442 31288 raft_consensus.cc:515] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.438349 31288 leader_election.cc:290] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 election: Requested vote from peers f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.439152 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:10:52.439455 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:52.439859 31164 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.439998 31239 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.447080 31239 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a89ddf02312a44619eae5b9d6f7ecdbe in term 1.
I20260709 13:10:52.447080 31164 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a89ddf02312a44619eae5b9d6f7ecdbe in term 1.
I20260709 13:10:52.448477 31054 leader_election.cc:304] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:10:52.449362 31288 raft_consensus.cc:2804] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.450316 31288 raft_consensus.cc:697] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 LEADER]: Becoming Leader. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Running, Role: LEADER
I20260709 13:10:52.451056 31288 consensus_queue.cc:237] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.459419 30986 catalog_manager.cc:5697] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe reported cstate change: term changed from 0 to 1, leader changed from <none> to a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193). New cstate: current_term: 1 leader_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.507898 31288 raft_consensus.cc:493] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.508502 31288 raft_consensus.cc:515] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.510373 31288 leader_election.cc:290] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.511359 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794" is_pre_election: true
I20260709 13:10:52.511605 31239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:52.512441 31239 raft_consensus.cc:2393] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a89ddf02312a44619eae5b9d6f7ecdbe in current term 1: Already voted for candidate f75e2042be8e4dd48c0d83db4569c794 in this term.
I20260709 13:10:52.513468 31054 leader_election.cc:304] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794
I20260709 13:10:52.514200 31288 raft_consensus.cc:3060] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.517347 31319 consensus_queue.cc:1048] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.520161 31288 raft_consensus.cc:2749] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.524530 31319 consensus_queue.cc:1048] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.530390 31238 raft_consensus.cc:3060] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.535337 31318 raft_consensus.cc:493] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.535969 31318 raft_consensus.cc:515] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.538944 31318 leader_election.cc:290] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.546355 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:52.547132 31089 raft_consensus.cc:2468] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 0.
I20260709 13:10:52.540678 31237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:52.548498 31237 raft_consensus.cc:2468] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 0.
I20260709 13:10:52.550683 31343 raft_consensus.cc:493] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.551218 31343 raft_consensus.cc:515] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.551721 31129 leader_election.cc:304] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:52.552719 31304 raft_consensus.cc:2804] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.553125 31304 raft_consensus.cc:493] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.544922 31283 consensus_queue.cc:1048] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.553462 31304 raft_consensus.cc:3060] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.553540 31343 leader_election.cc:290] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:52.550042 31319 consensus_queue.cc:1048] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:10:52.554296 31089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:52.555156 31089 raft_consensus.cc:2393] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ce733e5cc24d65b53fb2e607d8e541 in current term 1: Already voted for candidate f75e2042be8e4dd48c0d83db4569c794 in this term.
I20260709 13:10:52.556869 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794" is_pre_election: true
I20260709 13:10:52.557052 31319 consensus_queue.cc:1048] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.557801 31283 consensus_queue.cc:1048] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.559059 31204 leader_election.cc:304] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541; no voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794
I20260709 13:10:52.559937 31343 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.563197 31304 raft_consensus.cc:515] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.565423 31304 leader_election.cc:290] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:52.568207 31237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:52.568708 31287 raft_consensus.cc:493] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.568931 31237 raft_consensus.cc:3060] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.569293 31287 raft_consensus.cc:515] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.571561 31287 leader_election.cc:290] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:52.573491 31090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" is_pre_election: true
I20260709 13:10:52.575043 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75e2042be8e4dd48c0d83db4569c794" is_pre_election: true
I20260709 13:10:52.580106 31088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:52.580906 31088 raft_consensus.cc:3060] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.568176 31343 raft_consensus.cc:2749] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.582518 31204 leader_election.cc:304] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541; no voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794
I20260709 13:10:52.591837 31237 raft_consensus.cc:2468] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
I20260709 13:10:52.592684 31088 raft_consensus.cc:2468] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
I20260709 13:10:52.582573 31319 consensus_queue.cc:1048] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:10:52.600560 31304 consensus_queue.cc:1048] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.604918 31128 leader_election.cc:304] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:52.605897 31319 raft_consensus.cc:2804] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.606411 31319 raft_consensus.cc:697] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:52.607347 31319 consensus_queue.cc:237] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:52.610384 31287 raft_consensus.cc:2749] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:52.636581 30987 catalog_manager.cc:5697] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.748636 31319 consensus_queue.cc:1048] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.762707 31283 consensus_queue.cc:1048] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.817950 19967 ts_manager.cc:295] Set tserver state for a89ddf02312a44619eae5b9d6f7ecdbe to MAINTENANCE_MODE
I20260709 13:10:52.818773 19967 tablet_server.cc:179] TabletServer@127.19.127.193:0 shutting down...
W20260709 13:10:52.839831 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:52.846315 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:52.847031 19967 tablet_replica.cc:333] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.847705 19967 raft_consensus.cc:2243] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.848209 19967 raft_consensus.cc:2272] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.850425 19967 tablet_replica.cc:333] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.851208 19967 raft_consensus.cc:2243] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.851454 31287 consensus_queue.cc:1048] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.851797 19967 raft_consensus.cc:2272] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.854986 19967 tablet_replica.cc:333] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.855665 19967 raft_consensus.cc:2243] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:52.856462 19967 pending_rounds.cc:70] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Trying to abort 1 pending ops.
I20260709 13:10:52.856775 19967 pending_rounds.cc:77] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305636464447660032 op_type: NO_OP noop_request { }
I20260709 13:10:52.857163 19967 raft_consensus.cc:2889] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:52.857508 19967 raft_consensus.cc:2272] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.860036 19967 tablet_replica.cc:333] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.860805 19967 raft_consensus.cc:2243] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.861310 19967 raft_consensus.cc:2272] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.864918 19967 tablet_replica.cc:333] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.865741 19967 raft_consensus.cc:2243] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.866261 19967 raft_consensus.cc:2272] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.869031 19967 tablet_replica.cc:333] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.869716 19967 raft_consensus.cc:2243] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.870182 19967 raft_consensus.cc:2272] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.872634 19967 tablet_replica.cc:333] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.873415 19967 raft_consensus.cc:2243] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.873909 19967 raft_consensus.cc:2272] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.875965 19967 tablet_replica.cc:333] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.876539 19967 raft_consensus.cc:2243] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.876974 19967 raft_consensus.cc:2272] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:52.878350 31202 consensus_peers.cc:597] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:52.879053 19967 tablet_replica.cc:333] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.879716 19967 raft_consensus.cc:2243] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.880223 19967 raft_consensus.cc:2272] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.882350 19967 tablet_replica.cc:333] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: stopping tablet replica
I20260709 13:10:52.882836 19967 raft_consensus.cc:2243] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.883252 19967 raft_consensus.cc:2272] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.903759 19967 tablet_server.cc:196] TabletServer@127.19.127.193:0 shutdown complete.
W20260709 13:10:52.915083 31202 consensus_peers.cc:597] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:53.002463 31308 consensus_queue.cc:1048] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:53.012506 31202 consensus_peers.cc:597] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:53.049477 31283 consensus_queue.cc:1048] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:53.055627 31128 consensus_peers.cc:597] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:53.064533 31128 consensus_peers.cc:597] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:53.114351 31128 consensus_peers.cc:597] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:53.185612 31128 consensus_peers.cc:597] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:53.186164 31128 consensus_peers.cc:597] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:53.200232 31128 consensus_peers.cc:597] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:53.840912 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:10:53.942087 31202 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111) [suppressed 141 similar messages]
I20260709 13:10:54.189007 31318 raft_consensus.cc:493] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.189450 31318 raft_consensus.cc:515] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:54.191057 31318 leader_election.cc:290] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:54.192224 31236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541" is_pre_election: true
I20260709 13:10:54.192883 31236 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 1.
I20260709 13:10:54.194120 31129 leader_election.cc:304] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
W20260709 13:10:54.194783 31128 leader_election.cc:336] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111)
I20260709 13:10:54.194914 31318 raft_consensus.cc:2804] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.195389 31318 raft_consensus.cc:493] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.195722 31318 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.201117 31318 raft_consensus.cc:515] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:54.202875 31318 leader_election.cc:290] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 2 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:54.203958 31236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:54.204725 31236 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:54.207065 31128 leader_election.cc:336] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111)
I20260709 13:10:54.210343 31236 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 2.
I20260709 13:10:54.211493 31129 leader_election.cc:304] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: a89ddf02312a44619eae5b9d6f7ecdbe
I20260709 13:10:54.212193 31318 raft_consensus.cc:2804] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:54.213052 31318 raft_consensus.cc:697] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:54.213896 31318 consensus_queue.cc:237] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:54.220901 30987 catalog_manager.cc:5697] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 1 to 2, leader changed from a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193) to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.788784 31236 raft_consensus.cc:1275] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:54.790268 31367 consensus_queue.cc:1048] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:54.808537 31128 consensus_peers.cc:597] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:54.841980 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:10:55.362866 31202 consensus_peers.cc:597] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:55.410178 31128 consensus_peers.cc:597] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:55.458818 31202 consensus_peers.cc:597] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:55.516187 31202 consensus_peers.cc:597] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:55.518329 31128 consensus_peers.cc:597] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:55.624748 31128 consensus_peers.cc:597] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:55.634166 31128 consensus_peers.cc:597] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:55.746090 31128 consensus_peers.cc:597] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:55.798661 31128 consensus_peers.cc:597] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:55.842968 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:10:56.843977 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:10:57.361629 31128 consensus_peers.cc:597] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:57.844993 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:10:57.874833 31202 consensus_peers.cc:597] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:57.924408 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:57.931691 31379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:57.931944 31380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:57.933969 31382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:57.934163 19967 server_base.cc:1061] running on GCE node
I20260709 13:10:57.936259 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:57.936671 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:57.936870 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602657936853 us; error 0 us; skew 500 ppm
I20260709 13:10:57.937554 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260709 13:10:57.938254 31128 consensus_peers.cc:597] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:57.940171 19967 webserver.cc:533] Webserver started at http://127.19.127.193:40669/ using document root <none> and password file <none>
I20260709 13:10:57.940757 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:57.940974 19967 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:10:57.944715 19967 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:10:57.948545 31387 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:57.949501 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:57.949810 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
format_stamp: "Formatted at 2026-07-09 13:10:50 on dist-test-slave-s3n6"
I20260709 13:10:57.950067 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:57.968819 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:57.970034 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:57.971745 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:57.975528 31394 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:10:58.031888 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:10:58.032194 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.058s	user 0.002s	sys 0.000s
I20260709 13:10:58.032423 19967 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:10:58.037427 31394 tablet_bootstrap.cc:492] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:58.049880 31394 tablet_bootstrap.cc:492] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:10:58.050768 31394 tablet_bootstrap.cc:492] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.051409 31394 ts_tablet_manager.cc:1403] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:10:58.053830 31394 raft_consensus.cc:359] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.054615 31394 raft_consensus.cc:740] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.055351 31394 consensus_queue.cc:260] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.058820 31394 ts_tablet_manager.cc:1434] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:58.060256 31394 tablet_bootstrap.cc:492] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
W20260709 13:10:58.096828 31202 consensus_peers.cc:597] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:58.097455 31202 consensus_peers.cc:597] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:58.101564 31394 tablet_bootstrap.cc:492] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.102996 31394 tablet_bootstrap.cc:492] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.103919 31394 ts_tablet_manager.cc:1403] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.044s	user 0.029s	sys 0.004s
I20260709 13:10:58.107378 31394 raft_consensus.cc:359] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.108289 31394 raft_consensus.cc:740] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.109370 31394 consensus_queue.cc:260] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.112392 31394 ts_tablet_manager.cc:1434] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:58.114216 31394 tablet_bootstrap.cc:492] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:58.139956 31394 tablet_bootstrap.cc:492] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.141011 31394 tablet_bootstrap.cc:492] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.141815 31394 ts_tablet_manager.cc:1403] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.004s
I20260709 13:10:58.145082 31394 raft_consensus.cc:359] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.145900 31394 raft_consensus.cc:740] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.146682 31394 consensus_queue.cc:260] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.149000 31394 ts_tablet_manager.cc:1434] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:58.149880 31394 tablet_bootstrap.cc:492] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:58.150555 19967 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:10:58.150967 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.119s	user 0.059s	sys 0.003s
I20260709 13:10:58.165083 31394 tablet_bootstrap.cc:492] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.166191 31394 tablet_bootstrap.cc:492] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.167078 31394 ts_tablet_manager.cc:1403] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.002s
I20260709 13:10:58.231007 31394 raft_consensus.cc:359] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.273375 31394 raft_consensus.cc:740] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.276078 31394 consensus_queue.cc:260] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.283828 31394 ts_tablet_manager.cc:1434] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.055s	user 0.012s	sys 0.000s
I20260709 13:10:58.286834 31394 tablet_bootstrap.cc:492] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
W20260709 13:10:58.306756 31128 consensus_peers.cc:597] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:58.307955 31128 consensus_peers.cc:597] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:58.308861 31128 consensus_peers.cc:597] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:58.309688 31128 consensus_peers.cc:597] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:58.323024 31128 consensus_peers.cc:597] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:58.333279 31394 tablet_bootstrap.cc:492] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.334836 31394 tablet_bootstrap.cc:492] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.336035 31394 ts_tablet_manager.cc:1403] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.011s
I20260709 13:10:58.341521 31394 raft_consensus.cc:359] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.342453 31394 raft_consensus.cc:740] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.343856 31394 consensus_queue.cc:260] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.346586 31394 ts_tablet_manager.cc:1434] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:10:58.347990 31394 tablet_bootstrap.cc:492] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:58.369973 31394 tablet_bootstrap.cc:492] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.370903 31394 tablet_bootstrap.cc:492] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.371560 31394 ts_tablet_manager.cc:1403] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.008s
I20260709 13:10:58.373917 31394 raft_consensus.cc:359] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.374477 31394 raft_consensus.cc:740] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.375234 31394 consensus_queue.cc:260] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.377118 31394 ts_tablet_manager.cc:1434] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:58.378165 31394 tablet_bootstrap.cc:492] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:58.382402 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:46785
I20260709 13:10:58.382537 31460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:46785 every 8 connection(s)
W20260709 13:10:58.389575 31462 tablet.cc:2406] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:10:58.389747 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:58.402084 31394 tablet_bootstrap.cc:492] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.403077 31394 tablet_bootstrap.cc:492] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.403803 31394 ts_tablet_manager.cc:1403] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.008s
I20260709 13:10:58.406677 31394 raft_consensus.cc:359] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.407267 31394 raft_consensus.cc:740] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.408094 31394 consensus_queue.cc:260] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.410544 31394 ts_tablet_manager.cc:1434] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:58.411935 31394 tablet_bootstrap.cc:492] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:58.425329 31461 heartbeater.cc:344] Connected to a master server at 127.19.127.254:36679
I20260709 13:10:58.425843 31461 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:58.426931 31461 heartbeater.cc:507] Master 127.19.127.254:36679 requested a full tablet report, sending...
I20260709 13:10:58.427754 31394 tablet_bootstrap.cc:492] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.428798 31394 tablet_bootstrap.cc:492] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.429553 31394 ts_tablet_manager.cc:1403] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.005s
I20260709 13:10:58.432288 31164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "690a0dd835254d60b27ba503878f564c"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
mode: GRACEFUL
new_leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:41868
I20260709 13:10:58.432554 31394 raft_consensus.cc:359] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.433190 31164 raft_consensus.cc:606] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Received request to transfer leadership to TS 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:10:58.433389 31394 raft_consensus.cc:740] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.434333 31394 consensus_queue.cc:260] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.436545 31394 ts_tablet_manager.cc:1434] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:58.436978 30987 ts_manager.cc:194] Re-registered known tserver with Master: a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785)
I20260709 13:10:58.437278 31164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
mode: GRACEFUL
new_leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:41868
I20260709 13:10:58.437572 31394 tablet_bootstrap.cc:492] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:58.438027 31164 raft_consensus.cc:606] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Received request to transfer leadership to TS 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:10:58.440902 31164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
mode: GRACEFUL
new_leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:41868
I20260709 13:10:58.441567 31164 raft_consensus.cc:606] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Received request to transfer leadership to TS 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:10:58.442914 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:10:58.443526 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:58.444330 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:58.455776 30987 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60808
I20260709 13:10:58.456739 31394 tablet_bootstrap.cc:492] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.457969 31394 tablet_bootstrap.cc:492] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.458648 31394 ts_tablet_manager.cc:1403] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:10:58.460791 31461 heartbeater.cc:499] Master 127.19.127.254:36679 was elected leader, sending a full tablet report...
I20260709 13:10:58.461587 31394 raft_consensus.cc:359] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.462288 31394 raft_consensus.cc:740] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.463073 31394 consensus_queue.cc:260] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.465237 31394 ts_tablet_manager.cc:1434] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:10:58.466089 31394 tablet_bootstrap.cc:492] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap starting.
I20260709 13:10:58.473655 31419 raft_consensus.cc:993] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794: : Instructing follower 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:10:58.474577 31377 raft_consensus.cc:1081] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Signalling peer 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:10:58.477299 31236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:43140
I20260709 13:10:58.477896 31236 raft_consensus.cc:493] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:58.478289 31236 raft_consensus.cc:3060] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.483736 31394 tablet_bootstrap.cc:492] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.485005 31394 tablet_bootstrap.cc:492] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: Bootstrap complete.
I20260709 13:10:58.485842 31394 ts_tablet_manager.cc:1403] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:10:58.488868 31394 raft_consensus.cc:359] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.489449 31394 raft_consensus.cc:740] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89ddf02312a44619eae5b9d6f7ecdbe, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.489156 31236 raft_consensus.cc:515] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.490295 31394 consensus_queue.cc:260] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.492482 31394 ts_tablet_manager.cc:1434] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:58.493541 31236 leader_election.cc:290] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:58.494280 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:10:58.495178 31164 raft_consensus.cc:3055] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.495569 31164 raft_consensus.cc:740] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:58.496704 31164 consensus_queue.cc:260] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.498564 31164 raft_consensus.cc:3060] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.506798 31435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:58.507378 31164 raft_consensus.cc:2468] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:10:58.507669 31435 raft_consensus.cc:3060] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.508630 31204 leader_election.cc:304] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:58.509544 31308 raft_consensus.cc:2804] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:58.510067 31308 raft_consensus.cc:697] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:10:58.511328 31308 consensus_queue.cc:237] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.516773 31435 raft_consensus.cc:2468] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:10:58.523155 30987 catalog_manager.cc:5697] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 1 to 2, leader changed from f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194) to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:58.699867 31435 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.705742 31435 pending_rounds.cc:85] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Aborting all ops after (but not including) 0
I20260709 13:10:58.705982 31435 pending_rounds.cc:107] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:10:58.706173 31435 raft_consensus.cc:2889] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:58.738492 31378 raft_consensus.cc:993] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794: : Instructing follower 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:10:58.739005 31419 raft_consensus.cc:1081] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Signalling peer 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:10:58.740597 31236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:43140
I20260709 13:10:58.741132 31236 raft_consensus.cc:493] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:58.741448 31236 raft_consensus.cc:3060] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.746753 31236 raft_consensus.cc:515] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.748596 31236 leader_election.cc:290] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:58.749475 31435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:58.749816 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:10:58.750130 31435 raft_consensus.cc:3060] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.750489 31164 raft_consensus.cc:3055] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.750800 31164 raft_consensus.cc:740] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:58.751793 31164 consensus_queue.cc:260] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.753026 31164 raft_consensus.cc:3060] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.756032 31435 raft_consensus.cc:2468] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:10:58.757433 31202 leader_election.cc:304] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:10:58.758245 31308 raft_consensus.cc:2804] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:58.758641 31308 raft_consensus.cc:697] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:10:58.759495 31164 raft_consensus.cc:2468] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:10:58.759661 31308 consensus_queue.cc:237] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.766572 30987 catalog_manager.cc:5697] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 1 to 2, leader changed from f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194) to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
W20260709 13:10:58.846273 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:58.907532 31378 raft_consensus.cc:993] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794: : Instructing follower 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:10:58.908095 31419 raft_consensus.cc:1081] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Signalling peer 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:10:58.909649 31236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "690a0dd835254d60b27ba503878f564c"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:43140
I20260709 13:10:58.910149 31236 raft_consensus.cc:493] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:58.910461 31236 raft_consensus.cc:3060] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.918206 31236 raft_consensus.cc:515] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.920133 31236 leader_election.cc:290] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Requested vote from peers f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527), a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785)
I20260709 13:10:58.921176 31435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:58.921731 31435 raft_consensus.cc:3060] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.921726 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:10:58.922394 31164 raft_consensus.cc:3055] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.922685 31164 raft_consensus.cc:740] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:58.923715 31164 consensus_queue.cc:260] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.925200 31164 raft_consensus.cc:3060] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.933825 31164 raft_consensus.cc:2468] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:10:58.933808 31435 raft_consensus.cc:2468] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:10:58.935444 31202 leader_election.cc:304] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:10:58.936316 31308 raft_consensus.cc:2804] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:58.936782 31308 raft_consensus.cc:697] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:10:58.937709 31308 consensus_queue.cc:237] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:58.944712 30987 catalog_manager.cc:5697] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 1 to 2, leader changed from f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194) to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:58.964707 31435 raft_consensus.cc:1275] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.966055 31308 consensus_queue.cc:1048] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.981686 31164 raft_consensus.cc:1275] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.984752 31487 consensus_queue.cc:1048] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:59.173496 31186 debug-util.cc:398] Leaking SignalData structure 0x7b08001923e0 after lost signal to thread 19970
W20260709 13:10:59.176602 31186 debug-util.cc:398] Leaking SignalData structure 0x7b080015d580 after lost signal to thread 31263
I20260709 13:10:59.314536 31164 raft_consensus.cc:1275] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:59.316237 31487 consensus_queue.cc:1048] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:59.328121 31435 raft_consensus.cc:1275] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:59.330423 31486 consensus_queue.cc:1048] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:59.373135 31435 raft_consensus.cc:1275] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:59.375221 31486 consensus_queue.cc:1048] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:59.386510 31164 raft_consensus.cc:1275] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:59.388140 31488 consensus_queue.cc:1048] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:59.453449 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:59.465272 31164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
mode: GRACEFUL
new_leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:41868
I20260709 13:10:59.465873 31164 raft_consensus.cc:606] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Received request to transfer leadership to TS 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:10:59.480051 31236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
mode: GRACEFUL
new_leader_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:43162
I20260709 13:10:59.480615 31236 raft_consensus.cc:606] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 LEADER]: Received request to transfer leadership to TS f75e2042be8e4dd48c0d83db4569c794
I20260709 13:10:59.483563 31236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
mode: GRACEFUL
new_leader_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:43162
I20260709 13:10:59.484088 31236 raft_consensus.cc:606] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Received request to transfer leadership to TS f75e2042be8e4dd48c0d83db4569c794
I20260709 13:10:59.485188 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:10:59.485915 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:59.486690 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:59.543038 31487 raft_consensus.cc:993] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541: : Instructing follower f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:10:59.543633 31488 raft_consensus.cc:1081] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 LEADER]: Signalling peer f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:10:59.545154 31164 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:41852
I20260709 13:10:59.545657 31164 raft_consensus.cc:493] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:59.545984 31164 raft_consensus.cc:3060] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:59.550906 31164 raft_consensus.cc:515] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:59.552699 31164 leader_election.cc:290] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 2 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:59.553627 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:59.554001 31236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:59.554373 31434 raft_consensus.cc:3060] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:59.554754 31236 raft_consensus.cc:3055] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:59.555135 31236 raft_consensus.cc:740] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:10:59.556301 31236 consensus_queue.cc:260] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:59.557950 31236 raft_consensus.cc:3060] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:59.562184 31434 raft_consensus.cc:2468] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 2.
I20260709 13:10:59.563426 31128 leader_election.cc:304] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:59.564193 31419 raft_consensus.cc:2804] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:59.564642 31419 raft_consensus.cc:697] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:59.565501 31419 consensus_queue.cc:237] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:59.567145 31236 raft_consensus.cc:2468] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 2.
I20260709 13:10:59.573726 30987 catalog_manager.cc:5697] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 1 to 2, leader changed from 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195) to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:59.806103 31487 raft_consensus.cc:993] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541: : Instructing follower f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:10:59.806631 31488 raft_consensus.cc:1081] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Signalling peer f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:10:59.808493 31164 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:41852
I20260709 13:10:59.808979 31164 raft_consensus.cc:493] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:59.809264 31164 raft_consensus.cc:3060] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:59.815815 31164 raft_consensus.cc:515] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:59.818087 31164 leader_election.cc:290] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 3 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:10:59.818424 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:59.819402 31236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb50ff0d20914f53895e3e0aa1619d73" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:10:59.820156 31434 raft_consensus.cc:3060] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:59.820286 31236 raft_consensus.cc:3055] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:59.820753 31236 raft_consensus.cc:740] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:10:59.821499 31236 consensus_queue.cc:260] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:59.822501 31236 raft_consensus.cc:3060] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:59.828070 31236 raft_consensus.cc:2468] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 3.
I20260709 13:10:59.829134 31434 raft_consensus.cc:2468] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 3.
I20260709 13:10:59.829198 31129 leader_election.cc:304] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:10:59.830256 31419 raft_consensus.cc:2804] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:59.830740 31419 raft_consensus.cc:697] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:59.831598 31419 consensus_queue.cc:237] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:59.839855 30987 catalog_manager.cc:5697] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 2 to 3, leader changed from 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195) to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 3 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:59.847450 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:10:59.885195 31491 raft_consensus.cc:993] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794: : Instructing follower 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:10:59.885766 31419 raft_consensus.cc:1081] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Signalling peer 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:10:59.887729 31236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:43140
I20260709 13:10:59.888442 31236 raft_consensus.cc:493] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:59.888819 31236 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:59.897122 31236 raft_consensus.cc:515] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:59.899349 31236 leader_election.cc:290] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:10:59.900357 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:10:59.900848 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:10:59.901084 31434 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:59.901458 31164 raft_consensus.cc:3055] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:59.901783 31164 raft_consensus.cc:740] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:10:59.902673 31164 consensus_queue.cc:260] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:59.904139 31164 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:59.908980 31434 raft_consensus.cc:2468] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:10:59.910490 31202 leader_election.cc:304] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:10:59.911155 31164 raft_consensus.cc:2468] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:10:59.911500 31488 raft_consensus.cc:2804] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:59.912175 31488 raft_consensus.cc:697] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:10:59.913309 31488 consensus_queue.cc:237] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:10:59.922132 30987 catalog_manager.cc:5697] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 1 to 2, leader changed from f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194) to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:59.970714 31236 raft_consensus.cc:1275] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:59.972368 31419 consensus_queue.cc:1048] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:59.993101 31434 raft_consensus.cc:1275] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:59.996410 31491 consensus_queue.cc:1048] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:00.287456 31236 raft_consensus.cc:1275] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:00.288952 31491 consensus_queue.cc:1048] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:00.299953 31434 raft_consensus.cc:1275] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:00.303620 31506 consensus_queue.cc:1048] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:11:00.409766 31434 raft_consensus.cc:1275] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:00.411173 31488 consensus_queue.cc:1048] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:00.428056 31164 raft_consensus.cc:1275] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:00.431298 31511 consensus_queue.cc:1048] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:00.497148 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:00.508934 31236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
mode: GRACEFUL
new_leader_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:43162
I20260709 13:11:00.509548 31236 raft_consensus.cc:606] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Received request to transfer leadership to TS f75e2042be8e4dd48c0d83db4569c794
I20260709 13:11:00.512382 31164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
mode: GRACEFUL
new_leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:41868
I20260709 13:11:00.513051 31164 raft_consensus.cc:606] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Received request to transfer leadership to TS 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:11:00.516139 31164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
mode: GRACEFUL
new_leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:41868
I20260709 13:11:00.516770 31164 raft_consensus.cc:606] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Received request to transfer leadership to TS 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:11:00.518152 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:11:00.518729 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:00.519433 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:00.703526 31419 raft_consensus.cc:993] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794: : Instructing follower 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:00.704106 31491 raft_consensus.cc:1081] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Signalling peer 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:00.705871 31236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:43140
I20260709 13:11:00.706369 31236 raft_consensus.cc:493] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:00.706665 31236 raft_consensus.cc:3060] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:00.711826 31236 raft_consensus.cc:515] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:00.713577 31236 leader_election.cc:290] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:11:00.714740 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:11:00.715860 31434 raft_consensus.cc:3060] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:00.717474 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1885678eb9a4afdb5508e135f0eb6ff" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:11:00.722608 31164 raft_consensus.cc:3055] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:00.722997 31164 raft_consensus.cc:740] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:11:00.723946 31434 raft_consensus.cc:2468] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:11:00.724045 31164 consensus_queue.cc:260] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:00.725244 31202 leader_election.cc:304] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:11:00.725682 31164 raft_consensus.cc:3060] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:00.726011 31487 raft_consensus.cc:2804] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:00.726545 31487 raft_consensus.cc:697] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:11:00.727458 31487 consensus_queue.cc:237] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:00.733593 31164 raft_consensus.cc:2468] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:11:00.735520 30987 catalog_manager.cc:5697] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 1 to 2, leader changed from f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194) to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:00.761461 31506 raft_consensus.cc:993] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794: : Instructing follower 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:00.762019 31491 raft_consensus.cc:1081] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Signalling peer 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:00.763896 31236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:43140
I20260709 13:11:00.764493 31236 raft_consensus.cc:493] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:00.764853 31236 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:00.772708 31236 raft_consensus.cc:515] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:00.774787 31236 leader_election.cc:290] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 3 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:11:00.775874 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:11:00.775872 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:11:00.776765 31434 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:00.776839 31164 raft_consensus.cc:3055] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:00.777312 31164 raft_consensus.cc:740] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:11:00.778239 31164 consensus_queue.cc:260] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:00.779693 31164 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:00.782765 31434 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 3.
I20260709 13:11:00.784029 31202 leader_election.cc:304] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:11:00.784857 31487 raft_consensus.cc:2804] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:00.785318 31487 raft_consensus.cc:697] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:11:00.786298 31487 consensus_queue.cc:237] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:00.787428 31164 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 3.
I20260709 13:11:00.799786 30987 catalog_manager.cc:5697] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 2 to 3, leader changed from f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194) to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:00.848742 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:01.012234 31518 raft_consensus.cc:993] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541: : Instructing follower f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:11:01.012933 31487 raft_consensus.cc:1081] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Signalling peer f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:11:01.014837 31164 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:41852
I20260709 13:11:01.015439 31164 raft_consensus.cc:493] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:01.015765 31164 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.021564 31164 raft_consensus.cc:515] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.023711 31164 leader_election.cc:290] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 3 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:11:01.024721 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:11:01.024926 31236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:11:01.025406 31434 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.025641 31236 raft_consensus.cc:3055] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:01.026062 31236 raft_consensus.cc:740] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:11:01.026925 31236 consensus_queue.cc:260] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.028337 31236 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.032922 31434 raft_consensus.cc:2468] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 3.
I20260709 13:11:01.034502 31128 leader_election.cc:304] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:11:01.035393 31491 raft_consensus.cc:2804] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:01.035923 31491 raft_consensus.cc:697] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:11:01.036121 31236 raft_consensus.cc:2468] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 3.
I20260709 13:11:01.037055 31491 consensus_queue.cc:237] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.044472 30987 catalog_manager.cc:5697] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 2 to 3, leader changed from 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195) to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 3 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:01.248185 31434 raft_consensus.cc:1275] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:01.248343 31164 raft_consensus.cc:1275] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:01.249815 31518 consensus_queue.cc:1048] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:01.250669 31487 consensus_queue.cc:1048] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:01.257843 31164 raft_consensus.cc:1275] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:01.259784 31487 consensus_queue.cc:1048] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:01.291980 31435 raft_consensus.cc:1275] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:01.300653 31526 consensus_queue.cc:1048] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:01.424269 31434 raft_consensus.cc:1275] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:01.425632 31491 consensus_queue.cc:1048] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:01.437252 31236 raft_consensus.cc:1275] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:01.439070 31491 consensus_queue.cc:1048] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:01.526283 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:01.534690 31164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
mode: GRACEFUL
new_leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:41868
I20260709 13:11:01.535305 31164 raft_consensus.cc:606] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Received request to transfer leadership to TS 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:11:01.538388 31236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "690a0dd835254d60b27ba503878f564c"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
mode: GRACEFUL
new_leader_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:43162
I20260709 13:11:01.538942 31236 raft_consensus.cc:606] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Received request to transfer leadership to TS f75e2042be8e4dd48c0d83db4569c794
I20260709 13:11:01.542163 31236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
mode: GRACEFUL
new_leader_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:43162
I20260709 13:11:01.542917 31236 raft_consensus.cc:606] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Received request to transfer leadership to TS f75e2042be8e4dd48c0d83db4569c794
I20260709 13:11:01.544266 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:11:01.544871 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:01.545531 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:01.811997 31525 raft_consensus.cc:993] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541: : Instructing follower f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:11:01.812644 31511 raft_consensus.cc:1081] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Signalling peer f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:11:01.814419 31164 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "690a0dd835254d60b27ba503878f564c"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:41852
I20260709 13:11:01.814982 31164 raft_consensus.cc:493] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:01.815356 31164 raft_consensus.cc:3060] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.820688 31164 raft_consensus.cc:515] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.822470 31164 leader_election.cc:290] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 3 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:11:01.823731 31236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:11:01.823947 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "690a0dd835254d60b27ba503878f564c" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:11:01.824352 31236 raft_consensus.cc:3055] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:01.824705 31434 raft_consensus.cc:3060] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.824747 31236 raft_consensus.cc:740] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:11:01.825812 31236 consensus_queue.cc:260] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.826829 31236 raft_consensus.cc:3060] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.831817 31236 raft_consensus.cc:2468] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 3.
I20260709 13:11:01.833252 31129 leader_election.cc:304] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:11:01.833630 31434 raft_consensus.cc:2468] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 3.
I20260709 13:11:01.834017 31530 raft_consensus.cc:2804] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:01.834623 31530 raft_consensus.cc:697] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:11:01.835577 31530 consensus_queue.cc:237] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.845402 30986 catalog_manager.cc:5697] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 2 to 3, leader changed from 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195) to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 3 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:01.849843 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:01.883476 31506 raft_consensus.cc:993] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794: : Instructing follower 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:01.884176 31530 raft_consensus.cc:1081] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Signalling peer 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:01.885988 31236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:43140
I20260709 13:11:01.886526 31236 raft_consensus.cc:493] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:01.886873 31236 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:01.893607 31236 raft_consensus.cc:515] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.895179 31236 leader_election.cc:290] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 4 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:11:01.896299 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:11:01.896916 31164 raft_consensus.cc:3055] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:01.896103 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577ed9a02a104b7aa8e47067a9d33129" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:11:01.897258 31164 raft_consensus.cc:740] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:11:01.897456 31434 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:01.898072 31164 consensus_queue.cc:260] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.899551 31164 raft_consensus.cc:3060] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:01.905563 31434 raft_consensus.cc:2468] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 4.
I20260709 13:11:01.907212 31164 raft_consensus.cc:2468] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 4.
I20260709 13:11:01.908803 31204 leader_election.cc:304] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:11:01.909595 31511 raft_consensus.cc:2804] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:01.910140 31511 raft_consensus.cc:697] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 4 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:11:01.911015 31511 consensus_queue.cc:237] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.923276 30986 catalog_manager.cc:5697] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 3 to 4, leader changed from f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194) to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 4 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:01.947996 31525 raft_consensus.cc:993] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541: : Instructing follower f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:11:01.948572 31511 raft_consensus.cc:1081] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Signalling peer f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:11:01.950224 31164 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:41852
I20260709 13:11:01.950824 31164 raft_consensus.cc:493] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:01.951251 31164 raft_consensus.cc:3060] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.959126 31164 raft_consensus.cc:515] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.961589 31164 leader_election.cc:290] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 3 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:11:01.961910 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:11:01.962177 31236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "728f8355951f41aab19eebe146d1cf0d" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:11:01.962658 31434 raft_consensus.cc:3060] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.962811 31236 raft_consensus.cc:3055] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:01.963318 31236 raft_consensus.cc:740] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:11:01.964130 31236 consensus_queue.cc:260] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.965409 31236 raft_consensus.cc:3060] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.970247 31434 raft_consensus.cc:2468] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 3.
I20260709 13:11:01.971675 31128 leader_election.cc:304] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:11:01.972482 31530 raft_consensus.cc:2804] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:01.972608 31236 raft_consensus.cc:2468] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 3.
I20260709 13:11:01.973071 31530 raft_consensus.cc:697] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:11:01.974014 31530 consensus_queue.cc:237] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:01.982322 30986 catalog_manager.cc:5697] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 2 to 3, leader changed from 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195) to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 3 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.272274 31236 raft_consensus.cc:1275] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:02.273592 31530 consensus_queue.cc:1048] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.285447 31434 raft_consensus.cc:1275] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:02.288475 31506 consensus_queue.cc:1048] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:02.433127 31236 raft_consensus.cc:1275] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:02.434736 31506 consensus_queue.cc:1048] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.448292 31434 raft_consensus.cc:1275] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:02.449826 31506 consensus_queue.cc:1048] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.502417 31164 raft_consensus.cc:1275] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 4 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:02.503767 31511 consensus_queue.cc:1048] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:02.518074 31434 raft_consensus.cc:1275] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 4 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:02.520164 31511 consensus_queue.cc:1048] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:02.553574 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:02.565455 31164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4be225566b9c4cdd87b16f24336912b9"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
mode: GRACEFUL
new_leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:41868
I20260709 13:11:02.565990 31164 raft_consensus.cc:606] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Received request to transfer leadership to TS 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:11:02.568813 31164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
mode: GRACEFUL
new_leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:41868
I20260709 13:11:02.569336 31164 raft_consensus.cc:606] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Received request to transfer leadership to TS 91ce733e5cc24d65b53fb2e607d8e541
I20260709 13:11:02.572021 31236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
mode: GRACEFUL
new_leader_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:43162
I20260709 13:11:02.572551 31236 raft_consensus.cc:606] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 LEADER]: Received request to transfer leadership to TS f75e2042be8e4dd48c0d83db4569c794
I20260709 13:11:02.573709 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:11:02.574401 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:02.575120 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:02.757807 31546 raft_consensus.cc:993] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541: : Instructing follower f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:11:02.758281 31511 raft_consensus.cc:1081] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 LEADER]: Signalling peer f75e2042be8e4dd48c0d83db4569c794 to start an election
I20260709 13:11:02.759765 31164 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231"
dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
 from {username='slave'} at 127.0.0.1:41852
I20260709 13:11:02.760234 31164 raft_consensus.cc:493] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:02.760514 31164 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:02.765034 31164 raft_consensus.cc:515] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:02.766479 31164 leader_election.cc:290] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 4 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195:38317)
I20260709 13:11:02.767361 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:11:02.767551 31236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6935f128074cf6bcc6a2294094f231" candidate_uuid: "f75e2042be8e4dd48c0d83db4569c794" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
I20260709 13:11:02.767982 31434 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:02.768100 31236 raft_consensus.cc:3055] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:02.768447 31236 raft_consensus.cc:740] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:11:02.769119 31236 consensus_queue.cc:260] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:02.770062 31236 raft_consensus.cc:3060] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:02.772541 31434 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 4.
I20260709 13:11:02.773603 31128 leader_election.cc:304] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89ddf02312a44619eae5b9d6f7ecdbe, f75e2042be8e4dd48c0d83db4569c794; no voters: 
I20260709 13:11:02.774297 31506 raft_consensus.cc:2804] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:02.774823 31506 raft_consensus.cc:697] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 4 LEADER]: Becoming Leader. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:11:02.775063 31236 raft_consensus.cc:2468] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75e2042be8e4dd48c0d83db4569c794 in term 4.
I20260709 13:11:02.775650 31506 consensus_queue.cc:237] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:02.782621 30986 catalog_manager.cc:5697] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 reported cstate change: term changed from 3 to 4, leader changed from 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195) to f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194). New cstate: current_term: 4 leader_uuid: "f75e2042be8e4dd48c0d83db4569c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.808656 31530 raft_consensus.cc:993] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794: : Instructing follower 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:02.809158 31506 raft_consensus.cc:1081] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Signalling peer 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:02.810797 31236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4be225566b9c4cdd87b16f24336912b9"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:43140
I20260709 13:11:02.811354 31236 raft_consensus.cc:493] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:02.811640 31236 raft_consensus.cc:3060] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.816159 31236 raft_consensus.cc:515] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:02.817634 31236 leader_election.cc:290] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:11:02.818579 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4be225566b9c4cdd87b16f24336912b9" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:11:02.818650 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4be225566b9c4cdd87b16f24336912b9" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:11:02.819332 31434 raft_consensus.cc:3060] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.819474 31164 raft_consensus.cc:3055] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:02.819859 31164 raft_consensus.cc:740] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:11:02.820580 31164 consensus_queue.cc:260] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:02.821543 31164 raft_consensus.cc:3060] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.824138 31434 raft_consensus.cc:2468] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:11:02.825217 31202 leader_election.cc:304] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:11:02.826217 31511 raft_consensus.cc:2804] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:02.826745 31511 raft_consensus.cc:697] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:11:02.826741 31164 raft_consensus.cc:2468] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 2.
I20260709 13:11:02.827822 31511 consensus_queue.cc:237] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:02.835500 30986 catalog_manager.cc:5697] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 1 to 2, leader changed from f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194) to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 2 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:02.850912 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:03.050884 31530 raft_consensus.cc:993] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794: : Instructing follower 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:03.051530 31506 raft_consensus.cc:1081] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Signalling peer 91ce733e5cc24d65b53fb2e607d8e541 to start an election
I20260709 13:11:03.053483 31236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7"
dest_uuid: "91ce733e5cc24d65b53fb2e607d8e541"
 from {username='slave'} at 127.0.0.1:43140
I20260709 13:11:03.053957 31236 raft_consensus.cc:493] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:03.054263 31236 raft_consensus.cc:3060] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.059358 31236 raft_consensus.cc:515] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:03.061167 31236 leader_election.cc:290] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 3 election: Requested vote from peers a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785), f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527)
I20260709 13:11:03.061920 31434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe"
I20260709 13:11:03.062291 31164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc6ecb9c4c40b19c759263267cf5e7" candidate_uuid: "91ce733e5cc24d65b53fb2e607d8e541" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f75e2042be8e4dd48c0d83db4569c794"
I20260709 13:11:03.062505 31434 raft_consensus.cc:3060] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.062850 31164 raft_consensus.cc:3055] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:03.063117 31164 raft_consensus.cc:740] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f75e2042be8e4dd48c0d83db4569c794, State: Running, Role: LEADER
I20260709 13:11:03.063748 31164 consensus_queue.cc:260] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:03.064640 31164 raft_consensus.cc:3060] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.067514 31434 raft_consensus.cc:2468] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 3.
I20260709 13:11:03.068652 31202 leader_election.cc:304] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ce733e5cc24d65b53fb2e607d8e541, a89ddf02312a44619eae5b9d6f7ecdbe; no voters: 
I20260709 13:11:03.069453 31511 raft_consensus.cc:2804] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:03.069446 31164 raft_consensus.cc:2468] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ce733e5cc24d65b53fb2e607d8e541 in term 3.
I20260709 13:11:03.070092 31511 raft_consensus.cc:697] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 LEADER]: Becoming Leader. State: Replica: 91ce733e5cc24d65b53fb2e607d8e541, State: Running, Role: LEADER
I20260709 13:11:03.071020 31511 consensus_queue.cc:237] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } }
I20260709 13:11:03.077812 30986 catalog_manager.cc:5697] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 reported cstate change: term changed from 2 to 3, leader changed from f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194) to 91ce733e5cc24d65b53fb2e607d8e541 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "91ce733e5cc24d65b53fb2e607d8e541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:03.257139 31236 raft_consensus.cc:1275] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 4 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:03.258401 31506 consensus_queue.cc:1048] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ce733e5cc24d65b53fb2e607d8e541" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:03.270638 31434 raft_consensus.cc:1275] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 4 FOLLOWER]: Refusing update from remote peer f75e2042be8e4dd48c0d83db4569c794: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:03.272212 31506 consensus_queue.cc:1048] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:03.360929 31164 raft_consensus.cc:1275] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:03.362260 31555 consensus_queue.cc:1048] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:03.372375 31434 raft_consensus.cc:1275] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:03.373787 31555 consensus_queue.cc:1048] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:03.583122 31434 raft_consensus.cc:1275] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:03.584678 31555 consensus_queue.cc:1048] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89ddf02312a44619eae5b9d6f7ecdbe" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:03.585429 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:03.594589 31164 raft_consensus.cc:1275] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Refusing update from remote peer 91ce733e5cc24d65b53fb2e607d8e541: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:03.596383 31546 consensus_queue.cc:1048] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75e2042be8e4dd48c0d83db4569c794" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:03.627403 19967 tablet_server.cc:179] TabletServer@127.19.127.193:46785 shutting down...
I20260709 13:11:03.648976 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:03.649652 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.650247 19967 raft_consensus.cc:2243] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.650681 19967 raft_consensus.cc:2272] T fb50ff0d20914f53895e3e0aa1619d73 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.652726 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.653321 19967 raft_consensus.cc:2243] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.653818 19967 raft_consensus.cc:2272] T 690a0dd835254d60b27ba503878f564c P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.655788 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.656359 19967 raft_consensus.cc:2243] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.656853 19967 raft_consensus.cc:2272] T b1885678eb9a4afdb5508e135f0eb6ff P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.659356 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.660066 19967 raft_consensus.cc:2243] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.660552 19967 raft_consensus.cc:2272] T 75e9c2083400466995f59a7c4758e58e P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.662534 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.663092 19967 raft_consensus.cc:2243] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.663563 19967 raft_consensus.cc:2272] T 4be225566b9c4cdd87b16f24336912b9 P a89ddf02312a44619eae5b9d6f7ecdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.666559 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.667212 19967 raft_consensus.cc:2243] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.667760 19967 raft_consensus.cc:2272] T 8e6935f128074cf6bcc6a2294094f231 P a89ddf02312a44619eae5b9d6f7ecdbe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.670935 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.671583 19967 raft_consensus.cc:2243] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.672035 19967 raft_consensus.cc:2272] T 728f8355951f41aab19eebe146d1cf0d P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.673801 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.674312 19967 raft_consensus.cc:2243] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.674712 19967 raft_consensus.cc:2272] T bca443435e19492d9a937605286180b3 P a89ddf02312a44619eae5b9d6f7ecdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.676905 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.677455 19967 raft_consensus.cc:2243] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.677896 19967 raft_consensus.cc:2272] T 577ed9a02a104b7aa8e47067a9d33129 P a89ddf02312a44619eae5b9d6f7ecdbe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.679973 19967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:03.680442 19967 raft_consensus.cc:2243] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.680883 19967 raft_consensus.cc:2272] T 66dc6ecb9c4c40b19c759263267cf5e7 P a89ddf02312a44619eae5b9d6f7ecdbe [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:03.699720 31128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111) [suppressed 90 similar messages]
W20260709 13:11:03.702641 31128 consensus_peers.cc:597] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.712805 19967 tablet_server.cc:196] TabletServer@127.19.127.193:46785 shutdown complete.
I20260709 13:11:03.736052 19967 tablet_server.cc:179] TabletServer@127.19.127.194:0 shutting down...
I20260709 13:11:03.758172 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:03.758803 19967 tablet_replica.cc:333] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.759536 19967 raft_consensus.cc:2243] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.759974 19967 raft_consensus.cc:2272] T 577ed9a02a104b7aa8e47067a9d33129 P f75e2042be8e4dd48c0d83db4569c794 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.762847 19967 tablet_replica.cc:333] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.763654 19967 raft_consensus.cc:2243] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.764196 19967 raft_consensus.cc:2272] T 75e9c2083400466995f59a7c4758e58e P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:03.766886 31202 consensus_peers.cc:597] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.767526 19967 tablet_replica.cc:333] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.768244 19967 raft_consensus.cc:2243] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.768651 19967 raft_consensus.cc:2272] T 4be225566b9c4cdd87b16f24336912b9 P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.772248 19967 tablet_replica.cc:333] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.772921 19967 raft_consensus.cc:2243] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.773430 19967 raft_consensus.cc:2272] T 66dc6ecb9c4c40b19c759263267cf5e7 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:03.775749 31128 consensus_peers.cc:597] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.776629 19967 tablet_replica.cc:333] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.777367 19967 raft_consensus.cc:2243] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.778226 19967 raft_consensus.cc:2272] T 8e6935f128074cf6bcc6a2294094f231 P f75e2042be8e4dd48c0d83db4569c794 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.781638 19967 tablet_replica.cc:333] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.782320 19967 raft_consensus.cc:2243] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.782923 19967 raft_consensus.cc:2272] T bca443435e19492d9a937605286180b3 P f75e2042be8e4dd48c0d83db4569c794 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:03.784919 31128 consensus_peers.cc:597] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.786161 19967 tablet_replica.cc:333] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.786829 19967 raft_consensus.cc:2243] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.787828 19967 raft_consensus.cc:2272] T 728f8355951f41aab19eebe146d1cf0d P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.790243 19967 tablet_replica.cc:333] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.790807 19967 raft_consensus.cc:2243] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.791671 19967 raft_consensus.cc:2272] T fb50ff0d20914f53895e3e0aa1619d73 P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.793959 19967 tablet_replica.cc:333] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.794523 19967 raft_consensus.cc:2243] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.794981 19967 raft_consensus.cc:2272] T b1885678eb9a4afdb5508e135f0eb6ff P f75e2042be8e4dd48c0d83db4569c794 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.797291 19967 tablet_replica.cc:333] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794: stopping tablet replica
I20260709 13:11:03.797842 19967 raft_consensus.cc:2243] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.798676 19967 raft_consensus.cc:2272] T 690a0dd835254d60b27ba503878f564c P f75e2042be8e4dd48c0d83db4569c794 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:03.812219 31202 consensus_peers.cc:597] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer a89ddf02312a44619eae5b9d6f7ecdbe (127.19.127.193:46785): Couldn't send request to peer a89ddf02312a44619eae5b9d6f7ecdbe. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:46785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.843020 19967 tablet_server.cc:196] TabletServer@127.19.127.194:0 shutdown complete.
W20260709 13:11:03.851997 31036 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:03.871565 19967 tablet_server.cc:179] TabletServer@127.19.127.195:0 shutting down...
W20260709 13:11:03.873075 31204 consensus_peers.cc:597] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527): Couldn't send request to peer f75e2042be8e4dd48c0d83db4569c794. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:44527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.905123 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:11:03.905057 31204 consensus_peers.cc:597] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 -> Peer f75e2042be8e4dd48c0d83db4569c794 (127.19.127.194:44527): Couldn't send request to peer f75e2042be8e4dd48c0d83db4569c794. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:44527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.906020 19967 tablet_replica.cc:333] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.906951 19967 raft_consensus.cc:2243] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.907790 19967 raft_consensus.cc:2272] T 577ed9a02a104b7aa8e47067a9d33129 P 91ce733e5cc24d65b53fb2e607d8e541 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.909883 19967 tablet_replica.cc:333] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.910398 19967 raft_consensus.cc:2243] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.911158 19967 raft_consensus.cc:2272] T 66dc6ecb9c4c40b19c759263267cf5e7 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.913820 19967 tablet_replica.cc:333] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.914415 19967 raft_consensus.cc:2243] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.915212 19967 raft_consensus.cc:2272] T 75e9c2083400466995f59a7c4758e58e P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.917634 19967 tablet_replica.cc:333] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.918247 19967 raft_consensus.cc:2243] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.919065 19967 raft_consensus.cc:2272] T 4be225566b9c4cdd87b16f24336912b9 P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.921674 19967 tablet_replica.cc:333] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.922187 19967 raft_consensus.cc:2243] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.922582 19967 raft_consensus.cc:2272] T fb50ff0d20914f53895e3e0aa1619d73 P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.924691 19967 tablet_replica.cc:333] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.925185 19967 raft_consensus.cc:2243] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.925926 19967 raft_consensus.cc:2272] T b1885678eb9a4afdb5508e135f0eb6ff P 91ce733e5cc24d65b53fb2e607d8e541 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.928220 19967 tablet_replica.cc:333] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.928740 19967 raft_consensus.cc:2243] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.929101 19967 raft_consensus.cc:2272] T 690a0dd835254d60b27ba503878f564c P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.931210 19967 tablet_replica.cc:333] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.931804 19967 raft_consensus.cc:2243] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.932189 19967 raft_consensus.cc:2272] T 8e6935f128074cf6bcc6a2294094f231 P 91ce733e5cc24d65b53fb2e607d8e541 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.934787 19967 tablet_replica.cc:333] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.935424 19967 raft_consensus.cc:2243] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.935832 19967 raft_consensus.cc:2272] T 728f8355951f41aab19eebe146d1cf0d P 91ce733e5cc24d65b53fb2e607d8e541 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.937994 19967 tablet_replica.cc:333] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541: stopping tablet replica
I20260709 13:11:03.938431 19967 raft_consensus.cc:2243] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:03.939200 19967 raft_consensus.cc:2272] T bca443435e19492d9a937605286180b3 P 91ce733e5cc24d65b53fb2e607d8e541 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.973435 19967 tablet_server.cc:196] TabletServer@127.19.127.195:0 shutdown complete.
I20260709 13:11:04.001200 19967 master.cc:562] Master@127.19.127.254:36679 shutting down...
I20260709 13:11:04.028934 19967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:04.029481 19967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.029776 19967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1643a4acf6ac4697b36e1e59f4998df8: stopping tablet replica
I20260709 13:11:04.051903 19967 master.cc:584] Master@127.19.127.254:36679 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13387 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:11:04.095817 19967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.127.254:40833
I20260709 13:11:04.096964 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:04.103354 31574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:04.103401 31573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:04.103583 31576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:04.105710 19967 server_base.cc:1061] running on GCE node
I20260709 13:11:04.106755 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:04.107028 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:04.107209 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602664107178 us; error 0 us; skew 500 ppm
I20260709 13:11:04.107863 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:04.110343 19967 webserver.cc:533] Webserver started at http://127.19.127.254:37861/ using document root <none> and password file <none>
I20260709 13:11:04.110885 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:04.111090 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:04.111441 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:04.112666 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/master-0-root/instance:
uuid: "76600a68a4d641b5bf94feb8251d89f9"
format_stamp: "Formatted at 2026-07-09 13:11:04 on dist-test-slave-s3n6"
I20260709 13:11:04.117486 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:11:04.121336 31581 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.122274 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:11:04.122622 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/master-0-root
uuid: "76600a68a4d641b5bf94feb8251d89f9"
format_stamp: "Formatted at 2026-07-09 13:11:04 on dist-test-slave-s3n6"
I20260709 13:11:04.122924 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:04.140359 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:04.141618 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:04.180063 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.254:40833
I20260709 13:11:04.180174 31632 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.254:40833 every 8 connection(s)
I20260709 13:11:04.183995 31633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:04.195559 31633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9: Bootstrap starting.
I20260709 13:11:04.200726 31633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.205363 31633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9: No bootstrap required, opened a new log
I20260709 13:11:04.207762 31633 raft_consensus.cc:359] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76600a68a4d641b5bf94feb8251d89f9" member_type: VOTER }
I20260709 13:11:04.208164 31633 raft_consensus.cc:385] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.208427 31633 raft_consensus.cc:740] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76600a68a4d641b5bf94feb8251d89f9, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.209146 31633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76600a68a4d641b5bf94feb8251d89f9" member_type: VOTER }
I20260709 13:11:04.209610 31633 raft_consensus.cc:399] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:11:04.209846 31633 raft_consensus.cc:493] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:11:04.210132 31633 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.215348 31633 raft_consensus.cc:515] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76600a68a4d641b5bf94feb8251d89f9" member_type: VOTER }
I20260709 13:11:04.215945 31633 leader_election.cc:304] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76600a68a4d641b5bf94feb8251d89f9; no voters: 
I20260709 13:11:04.217214 31633 leader_election.cc:290] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:11:04.217590 31636 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:04.219123 31636 raft_consensus.cc:697] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 1 LEADER]: Becoming Leader. State: Replica: 76600a68a4d641b5bf94feb8251d89f9, State: Running, Role: LEADER
I20260709 13:11:04.219856 31636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76600a68a4d641b5bf94feb8251d89f9" member_type: VOTER }
I20260709 13:11:04.220228 31633 sys_catalog.cc:565] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:11:04.224418 31637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76600a68a4d641b5bf94feb8251d89f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76600a68a4d641b5bf94feb8251d89f9" member_type: VOTER } }
I20260709 13:11:04.224597 31638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76600a68a4d641b5bf94feb8251d89f9. Latest consensus state: current_term: 1 leader_uuid: "76600a68a4d641b5bf94feb8251d89f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76600a68a4d641b5bf94feb8251d89f9" member_type: VOTER } }
I20260709 13:11:04.225236 31637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:04.225345 31638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:04.234508 31643 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:11:04.239993 31643 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:11:04.241405 19967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:11:04.249239 31643 catalog_manager.cc:1383] Generated new cluster ID: 066468da22dd41c8bdc4f10ce6ae82c4
I20260709 13:11:04.249564 31643 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:11:04.271054 31643 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:11:04.272461 31643 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:11:04.291543 31643 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9: Generated new TSK 0
I20260709 13:11:04.292214 31643 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:11:04.308630 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:04.315500 31654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:04.316577 31655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:04.318197 31657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:04.318408 19967 server_base.cc:1061] running on GCE node
I20260709 13:11:04.319846 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:04.320075 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:04.320264 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602664320244 us; error 0 us; skew 500 ppm
I20260709 13:11:04.320852 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:04.323648 19967 webserver.cc:533] Webserver started at http://127.19.127.193:44185/ using document root <none> and password file <none>
I20260709 13:11:04.324196 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:04.324426 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:04.324745 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:04.325941 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-0-root/instance:
uuid: "f16cfd719c844ef09f66734f0c255830"
format_stamp: "Formatted at 2026-07-09 13:11:04 on dist-test-slave-s3n6"
I20260709 13:11:04.330899 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:11:04.335054 31662 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.336063 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:11:04.336346 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "f16cfd719c844ef09f66734f0c255830"
format_stamp: "Formatted at 2026-07-09 13:11:04 on dist-test-slave-s3n6"
I20260709 13:11:04.336632 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:04.348155 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:04.349313 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:04.350771 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:04.353328 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:04.353516 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.353818 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:04.353996 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.397251 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:33763
I20260709 13:11:04.397439 31724 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:33763 every 8 connection(s)
I20260709 13:11:04.402477 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:04.410976 31729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:04.411535 31730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:04.414968 19967 server_base.cc:1061] running on GCE node
W20260709 13:11:04.414991 31732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:04.415726 31725 heartbeater.cc:344] Connected to a master server at 127.19.127.254:40833
I20260709 13:11:04.416221 31725 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:04.416476 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:04.416791 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:04.416999 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602664416978 us; error 0 us; skew 500 ppm
I20260709 13:11:04.417168 31725 heartbeater.cc:507] Master 127.19.127.254:40833 requested a full tablet report, sending...
I20260709 13:11:04.417788 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:04.419859 31598 ts_manager.cc:194] Registered new tserver with Master: f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763)
I20260709 13:11:04.421983 31598 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40384
I20260709 13:11:04.424791 19967 webserver.cc:533] Webserver started at http://127.19.127.194:43003/ using document root <none> and password file <none>
I20260709 13:11:04.425426 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:04.425630 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:04.425913 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:04.427050 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-1-root/instance:
uuid: "744f86684ddb45b99a4d4d8ba669611d"
format_stamp: "Formatted at 2026-07-09 13:11:04 on dist-test-slave-s3n6"
I20260709 13:11:04.431882 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:11:04.435662 31737 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.436618 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:11:04.436996 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-1-root
uuid: "744f86684ddb45b99a4d4d8ba669611d"
format_stamp: "Formatted at 2026-07-09 13:11:04 on dist-test-slave-s3n6"
I20260709 13:11:04.437322 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:04.450232 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:04.451532 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:04.453037 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:04.455399 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:04.455672 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.455919 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:04.456089 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.497298 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.194:34349
I20260709 13:11:04.497394 31799 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.194:34349 every 8 connection(s)
I20260709 13:11:04.502269 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:04.509794 31803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:04.510599 31804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:04.514859 31800 heartbeater.cc:344] Connected to a master server at 127.19.127.254:40833
I20260709 13:11:04.515470 31800 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:04.516582 31800 heartbeater.cc:507] Master 127.19.127.254:40833 requested a full tablet report, sending...
W20260709 13:11:04.517117 31806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:04.517848 19967 server_base.cc:1061] running on GCE node
I20260709 13:11:04.518929 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:04.519162 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:04.519495 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602664519473 us; error 0 us; skew 500 ppm
I20260709 13:11:04.519218 31598 ts_manager.cc:194] Registered new tserver with Master: 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349)
I20260709 13:11:04.520254 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:04.521255 31598 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40390
I20260709 13:11:04.523433 19967 webserver.cc:533] Webserver started at http://127.19.127.195:39927/ using document root <none> and password file <none>
I20260709 13:11:04.524192 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:04.524422 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:04.524770 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:04.525952 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-2-root/instance:
uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
format_stamp: "Formatted at 2026-07-09 13:11:04 on dist-test-slave-s3n6"
I20260709 13:11:04.530858 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:11:04.534512 31811 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.535549 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:11:04.535867 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-2-root
uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
format_stamp: "Formatted at 2026-07-09 13:11:04 on dist-test-slave-s3n6"
I20260709 13:11:04.536219 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602314247669-19967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:04.547755 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:04.548975 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:04.550464 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:04.552913 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:04.553170 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.553418 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:04.553593 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:04.597630 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.195:45299
I20260709 13:11:04.597739 31873 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.195:45299 every 8 connection(s)
I20260709 13:11:04.612284 31874 heartbeater.cc:344] Connected to a master server at 127.19.127.254:40833
I20260709 13:11:04.612725 31874 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:04.613531 31874 heartbeater.cc:507] Master 127.19.127.254:40833 requested a full tablet report, sending...
I20260709 13:11:04.615605 31598 ts_manager.cc:194] Registered new tserver with Master: ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195:45299)
I20260709 13:11:04.615885 19967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014929706s
I20260709 13:11:04.617568 31598 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40402
I20260709 13:11:04.644373 31598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40410:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:04.646528 31598 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:04.705441 31839 tablet_service.cc:1511] Processing CreateTablet for tablet 9b2b0208babf4561a94ba880f5050dd8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=caee3182504d4f3bb2db29cc101336c8]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:04.706233 31838 tablet_service.cc:1511] Processing CreateTablet for tablet d6a58880ac054e49a6b58c7b37a62441 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=caee3182504d4f3bb2db29cc101336c8]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:04.706924 31839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2b0208babf4561a94ba880f5050dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:04.707687 31838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a58880ac054e49a6b58c7b37a62441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:04.708463 31765 tablet_service.cc:1511] Processing CreateTablet for tablet 9b2b0208babf4561a94ba880f5050dd8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=caee3182504d4f3bb2db29cc101336c8]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:04.709932 31765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2b0208babf4561a94ba880f5050dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:04.710301 31764 tablet_service.cc:1511] Processing CreateTablet for tablet d6a58880ac054e49a6b58c7b37a62441 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=caee3182504d4f3bb2db29cc101336c8]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:04.711723 31764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a58880ac054e49a6b58c7b37a62441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:04.725939 31690 tablet_service.cc:1511] Processing CreateTablet for tablet 9b2b0208babf4561a94ba880f5050dd8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=caee3182504d4f3bb2db29cc101336c8]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:04.727519 31690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2b0208babf4561a94ba880f5050dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:04.729399 31689 tablet_service.cc:1511] Processing CreateTablet for tablet d6a58880ac054e49a6b58c7b37a62441 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=caee3182504d4f3bb2db29cc101336c8]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:04.730805 31689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a58880ac054e49a6b58c7b37a62441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:04.734400 31889 tablet_bootstrap.cc:492] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9: Bootstrap starting.
I20260709 13:11:04.737650 31890 tablet_bootstrap.cc:492] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d: Bootstrap starting.
I20260709 13:11:04.744189 31889 tablet_bootstrap.cc:654] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.745136 31890 tablet_bootstrap.cc:654] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.756410 31889 tablet_bootstrap.cc:492] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9: No bootstrap required, opened a new log
I20260709 13:11:04.756724 31890 tablet_bootstrap.cc:492] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d: No bootstrap required, opened a new log
I20260709 13:11:04.757050 31889 ts_tablet_manager.cc:1403] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.009s
I20260709 13:11:04.757328 31890 ts_tablet_manager.cc:1403] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.006s
I20260709 13:11:04.760573 31893 tablet_bootstrap.cc:492] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830: Bootstrap starting.
I20260709 13:11:04.760432 31889 raft_consensus.cc:359] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:04.763052 31889 raft_consensus.cc:385] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.763487 31889 raft_consensus.cc:740] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe75d7e7037446f9207b997bd3bf3b9, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.764469 31889 consensus_queue.cc:260] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:04.761967 31890 raft_consensus.cc:359] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:04.766747 31890 raft_consensus.cc:385] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.767163 31890 raft_consensus.cc:740] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744f86684ddb45b99a4d4d8ba669611d, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.768198 31890 consensus_queue.cc:260] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:04.768769 31874 heartbeater.cc:499] Master 127.19.127.254:40833 was elected leader, sending a full tablet report...
I20260709 13:11:04.769306 31889 ts_tablet_manager.cc:1434] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:11:04.769877 31893 tablet_bootstrap.cc:654] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.770350 31889 tablet_bootstrap.cc:492] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9: Bootstrap starting.
I20260709 13:11:04.777855 31890 ts_tablet_manager.cc:1434] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.007s
I20260709 13:11:04.777971 31800 heartbeater.cc:499] Master 127.19.127.254:40833 was elected leader, sending a full tablet report...
I20260709 13:11:04.778177 31889 tablet_bootstrap.cc:654] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.779089 31890 tablet_bootstrap.cc:492] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d: Bootstrap starting.
I20260709 13:11:04.786222 31890 tablet_bootstrap.cc:654] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.800132 31893 tablet_bootstrap.cc:492] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830: No bootstrap required, opened a new log
I20260709 13:11:04.800895 31893 ts_tablet_manager.cc:1403] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.009s
I20260709 13:11:04.802265 31889 tablet_bootstrap.cc:492] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9: No bootstrap required, opened a new log
I20260709 13:11:04.802667 31890 tablet_bootstrap.cc:492] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d: No bootstrap required, opened a new log
I20260709 13:11:04.802891 31889 ts_tablet_manager.cc:1403] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.011s
I20260709 13:11:04.803413 31890 ts_tablet_manager.cc:1403] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.010s
I20260709 13:11:04.804737 31893 raft_consensus.cc:359] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:04.805682 31893 raft_consensus.cc:385] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.806073 31893 raft_consensus.cc:740] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.806246 31889 raft_consensus.cc:359] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:04.807113 31889 raft_consensus.cc:385] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.806900 31890 raft_consensus.cc:359] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:04.807482 31889 raft_consensus.cc:740] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe75d7e7037446f9207b997bd3bf3b9, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.807165 31893 consensus_queue.cc:260] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:04.807858 31890 raft_consensus.cc:385] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.808247 31890 raft_consensus.cc:740] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744f86684ddb45b99a4d4d8ba669611d, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.808487 31889 consensus_queue.cc:260] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:04.809242 31890 consensus_queue.cc:260] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:04.813050 31725 heartbeater.cc:499] Master 127.19.127.254:40833 was elected leader, sending a full tablet report...
I20260709 13:11:04.813709 31890 ts_tablet_manager.cc:1434] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 13:11:04.814630 31893 ts_tablet_manager.cc:1434] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:11:04.815814 31893 tablet_bootstrap.cc:492] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830: Bootstrap starting.
I20260709 13:11:04.816387 31889 ts_tablet_manager.cc:1434] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:11:04.823912 31893 tablet_bootstrap.cc:654] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.830210 31893 tablet_bootstrap.cc:492] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830: No bootstrap required, opened a new log
I20260709 13:11:04.830677 31893 ts_tablet_manager.cc:1403] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:11:04.833135 31893 raft_consensus.cc:359] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:04.833765 31893 raft_consensus.cc:385] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.834111 31893 raft_consensus.cc:740] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.834808 31893 consensus_queue.cc:260] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:04.836669 31893 ts_tablet_manager.cc:1434] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.003s
I20260709 13:11:04.906488 31895 raft_consensus.cc:493] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.907110 31895 raft_consensus.cc:515] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:04.909241 31895 leader_election.cc:290] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195:45299), f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763)
I20260709 13:11:04.919363 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2b0208babf4561a94ba880f5050dd8" candidate_uuid: "744f86684ddb45b99a4d4d8ba669611d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" is_pre_election: true
I20260709 13:11:04.920219 31849 raft_consensus.cc:2468] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744f86684ddb45b99a4d4d8ba669611d in term 0.
I20260709 13:11:04.920462 31700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2b0208babf4561a94ba880f5050dd8" candidate_uuid: "744f86684ddb45b99a4d4d8ba669611d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16cfd719c844ef09f66734f0c255830" is_pre_election: true
I20260709 13:11:04.921257 31700 raft_consensus.cc:2468] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744f86684ddb45b99a4d4d8ba669611d in term 0.
I20260709 13:11:04.921504 31740 leader_election.cc:304] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744f86684ddb45b99a4d4d8ba669611d, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:04.922394 31895 raft_consensus.cc:2804] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:04.922744 31895 raft_consensus.cc:493] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.923022 31895 raft_consensus.cc:3060] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.929183 31895 raft_consensus.cc:515] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:04.930948 31895 leader_election.cc:290] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [CANDIDATE]: Term 1 election: Requested vote from peers ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195:45299), f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763)
I20260709 13:11:04.931968 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2b0208babf4561a94ba880f5050dd8" candidate_uuid: "744f86684ddb45b99a4d4d8ba669611d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
I20260709 13:11:04.932106 31700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2b0208babf4561a94ba880f5050dd8" candidate_uuid: "744f86684ddb45b99a4d4d8ba669611d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16cfd719c844ef09f66734f0c255830"
I20260709 13:11:04.932581 31849 raft_consensus.cc:3060] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.932735 31700 raft_consensus.cc:3060] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.938843 31849 raft_consensus.cc:2468] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744f86684ddb45b99a4d4d8ba669611d in term 1.
I20260709 13:11:04.939042 31700 raft_consensus.cc:2468] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744f86684ddb45b99a4d4d8ba669611d in term 1.
I20260709 13:11:04.940099 31740 leader_election.cc:304] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744f86684ddb45b99a4d4d8ba669611d, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:04.940838 31895 raft_consensus.cc:2804] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:04.941808 31895 raft_consensus.cc:697] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 1 LEADER]: Becoming Leader. State: Replica: 744f86684ddb45b99a4d4d8ba669611d, State: Running, Role: LEADER
I20260709 13:11:04.942714 31895 consensus_queue.cc:237] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:04.950924 31596 catalog_manager.cc:5697] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d reported cstate change: term changed from 0 to 1, leader changed from <none> to 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194). New cstate: current_term: 1 leader_uuid: "744f86684ddb45b99a4d4d8ba669611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:05.106166 31899 raft_consensus.cc:493] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.106796 31899 raft_consensus.cc:515] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.109201 31899 leader_election.cc:290] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349), ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195:45299)
I20260709 13:11:05.119408 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a58880ac054e49a6b58c7b37a62441" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744f86684ddb45b99a4d4d8ba669611d" is_pre_election: true
I20260709 13:11:05.119822 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a58880ac054e49a6b58c7b37a62441" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" is_pre_election: true
I20260709 13:11:05.120213 31775 raft_consensus.cc:2468] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 0.
I20260709 13:11:05.120496 31849 raft_consensus.cc:2468] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 0.
I20260709 13:11:05.121593 31663 leader_election.cc:304] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744f86684ddb45b99a4d4d8ba669611d, f16cfd719c844ef09f66734f0c255830; no voters: 
I20260709 13:11:05.122430 31899 raft_consensus.cc:2804] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:05.122725 31899 raft_consensus.cc:493] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.122958 31899 raft_consensus.cc:3060] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.128701 31899 raft_consensus.cc:515] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.130558 31899 leader_election.cc:290] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 1 election: Requested vote from peers 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349), ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195:45299)
I20260709 13:11:05.131572 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a58880ac054e49a6b58c7b37a62441" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744f86684ddb45b99a4d4d8ba669611d"
I20260709 13:11:05.131676 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a58880ac054e49a6b58c7b37a62441" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
I20260709 13:11:05.132220 31775 raft_consensus.cc:3060] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.132349 31849 raft_consensus.cc:3060] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.139014 31849 raft_consensus.cc:2468] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 1.
I20260709 13:11:05.139206 31775 raft_consensus.cc:2468] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 1.
I20260709 13:11:05.140220 31665 leader_election.cc:304] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16cfd719c844ef09f66734f0c255830, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:05.141052 31899 raft_consensus.cc:2804] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:05.141965 31899 raft_consensus.cc:697] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 1 LEADER]: Becoming Leader. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Running, Role: LEADER
I20260709 13:11:05.142809 31899 consensus_queue.cc:237] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.149420 31596 catalog_manager.cc:5697] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 reported cstate change: term changed from 0 to 1, leader changed from <none> to f16cfd719c844ef09f66734f0c255830 (127.19.127.193). New cstate: current_term: 1 leader_uuid: "f16cfd719c844ef09f66734f0c255830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:05.192638 31596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40424:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:05.194689 31596 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:05.219269 31690 tablet_service.cc:1511] Processing CreateTablet for tablet 30ec4afb32a2428d948291815ce55045 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3942a942472541ed8449c479555de4e8]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:05.220556 31690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30ec4afb32a2428d948291815ce55045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.220594 31764 tablet_service.cc:1511] Processing CreateTablet for tablet 30ec4afb32a2428d948291815ce55045 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3942a942472541ed8449c479555de4e8]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:05.221954 31764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30ec4afb32a2428d948291815ce55045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.224709 31689 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7add65d16f438283e81d877ec6c4f3 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3942a942472541ed8449c479555de4e8]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:05.226061 31689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7add65d16f438283e81d877ec6c4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.228340 31838 tablet_service.cc:1511] Processing CreateTablet for tablet 30ec4afb32a2428d948291815ce55045 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3942a942472541ed8449c479555de4e8]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:05.229915 31838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30ec4afb32a2428d948291815ce55045. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:11:05.235795 31646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:05.235845 31893 tablet_bootstrap.cc:492] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830: Bootstrap starting.
I20260709 13:11:05.239778 31890 tablet_bootstrap.cc:492] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d: Bootstrap starting.
I20260709 13:11:05.242211 31839 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7add65d16f438283e81d877ec6c4f3 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3942a942472541ed8449c479555de4e8]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:05.243650 31839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7add65d16f438283e81d877ec6c4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.247197 31893 tablet_bootstrap.cc:654] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.249421 31889 tablet_bootstrap.cc:492] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9: Bootstrap starting.
I20260709 13:11:05.250685 31764 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7add65d16f438283e81d877ec6c4f3 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3942a942472541ed8449c479555de4e8]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:05.252063 31764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7add65d16f438283e81d877ec6c4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.255050 31890 tablet_bootstrap.cc:654] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.256301 31889 tablet_bootstrap.cc:654] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.286074 31893 tablet_bootstrap.cc:492] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830: No bootstrap required, opened a new log
I20260709 13:11:05.286463 31890 tablet_bootstrap.cc:492] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d: No bootstrap required, opened a new log
I20260709 13:11:05.286834 31893 ts_tablet_manager.cc:1403] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.006s
I20260709 13:11:05.287055 31890 ts_tablet_manager.cc:1403] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d: Time spent bootstrapping tablet: real 0.048s	user 0.007s	sys 0.010s
I20260709 13:11:05.290566 31890 raft_consensus.cc:359] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.291343 31890 raft_consensus.cc:385] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.291636 31890 raft_consensus.cc:740] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744f86684ddb45b99a4d4d8ba669611d, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.291890 31893 raft_consensus.cc:359] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.292743 31893 raft_consensus.cc:385] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.292613 31890 consensus_queue.cc:260] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.293176 31893 raft_consensus.cc:740] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.294004 31893 consensus_queue.cc:260] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.294445 31889 tablet_bootstrap.cc:492] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9: No bootstrap required, opened a new log
I20260709 13:11:05.295137 31889 ts_tablet_manager.cc:1403] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.000s
I20260709 13:11:05.296391 31893 ts_tablet_manager.cc:1434] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:11:05.297526 31893 tablet_bootstrap.cc:492] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830: Bootstrap starting.
I20260709 13:11:05.298453 31889 raft_consensus.cc:359] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.299249 31889 raft_consensus.cc:385] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.299553 31889 raft_consensus.cc:740] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe75d7e7037446f9207b997bd3bf3b9, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.300340 31889 consensus_queue.cc:260] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.303110 31889 ts_tablet_manager.cc:1434] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.006s
I20260709 13:11:05.304248 31890 ts_tablet_manager.cc:1434] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:11:05.304299 31889 tablet_bootstrap.cc:492] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9: Bootstrap starting.
I20260709 13:11:05.306051 31893 tablet_bootstrap.cc:654] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.307082 31890 tablet_bootstrap.cc:492] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d: Bootstrap starting.
I20260709 13:11:05.315593 31890 tablet_bootstrap.cc:654] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.322624 31889 tablet_bootstrap.cc:654] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.326313 31893 tablet_bootstrap.cc:492] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830: No bootstrap required, opened a new log
I20260709 13:11:05.326828 31893 ts_tablet_manager.cc:1403] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.013s
I20260709 13:11:05.327590 31890 tablet_bootstrap.cc:492] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d: No bootstrap required, opened a new log
I20260709 13:11:05.328146 31890 ts_tablet_manager.cc:1403] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 13:11:05.328908 31889 tablet_bootstrap.cc:492] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9: No bootstrap required, opened a new log
I20260709 13:11:05.329464 31889 ts_tablet_manager.cc:1403] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.005s
I20260709 13:11:05.329409 31893 raft_consensus.cc:359] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.330222 31893 raft_consensus.cc:385] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.330509 31893 raft_consensus.cc:740] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.331223 31893 consensus_queue.cc:260] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.331399 31890 raft_consensus.cc:359] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.332191 31890 raft_consensus.cc:385] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.332496 31890 raft_consensus.cc:740] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744f86684ddb45b99a4d4d8ba669611d, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.332747 31889 raft_consensus.cc:359] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.333439 31893 ts_tablet_manager.cc:1434] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:11:05.334120 31889 raft_consensus.cc:385] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.333280 31890 consensus_queue.cc:260] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.334436 31889 raft_consensus.cc:740] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe75d7e7037446f9207b997bd3bf3b9, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.338896 31889 consensus_queue.cc:260] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.340965 31889 ts_tablet_manager.cc:1434] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:11:05.338003 31890 ts_tablet_manager.cc:1434] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:11:05.372999 31894 raft_consensus.cc:493] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.373540 31894 raft_consensus.cc:515] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.375752 31894 leader_election.cc:290] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763), 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349)
I20260709 13:11:05.385434 31700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16cfd719c844ef09f66734f0c255830" is_pre_election: true
I20260709 13:11:05.385975 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744f86684ddb45b99a4d4d8ba669611d" is_pre_election: true
I20260709 13:11:05.386262 31700 raft_consensus.cc:2468] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 0.
I20260709 13:11:05.386834 31775 raft_consensus.cc:2468] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 0.
I20260709 13:11:05.387485 31812 leader_election.cc:304] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16cfd719c844ef09f66734f0c255830, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:05.388365 31894 raft_consensus.cc:2804] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:05.388736 31894 raft_consensus.cc:493] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.389041 31894 raft_consensus.cc:3060] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.394397 31894 raft_consensus.cc:515] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.396209 31894 leader_election.cc:290] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 1 election: Requested vote from peers f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763), 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349)
I20260709 13:11:05.397035 31700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16cfd719c844ef09f66734f0c255830"
I20260709 13:11:05.397219 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744f86684ddb45b99a4d4d8ba669611d"
I20260709 13:11:05.397619 31700 raft_consensus.cc:3060] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.397820 31775 raft_consensus.cc:3060] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.403834 31700 raft_consensus.cc:2468] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 1.
I20260709 13:11:05.404992 31812 leader_election.cc:304] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16cfd719c844ef09f66734f0c255830, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:05.405134 31775 raft_consensus.cc:2468] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 1.
I20260709 13:11:05.405754 31894 raft_consensus.cc:2804] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:05.406885 31894 raft_consensus.cc:697] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Becoming Leader. State: Replica: ffe75d7e7037446f9207b997bd3bf3b9, State: Running, Role: LEADER
I20260709 13:11:05.407811 31894 consensus_queue.cc:237] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.414851 31598 catalog_manager.cc:5697] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:05.455145 31894 raft_consensus.cc:493] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.455727 31894 raft_consensus.cc:515] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.457532 31894 leader_election.cc:290] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763), 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349)
I20260709 13:11:05.458595 31700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30ec4afb32a2428d948291815ce55045" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16cfd719c844ef09f66734f0c255830" is_pre_election: true
I20260709 13:11:05.458943 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30ec4afb32a2428d948291815ce55045" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744f86684ddb45b99a4d4d8ba669611d" is_pre_election: true
I20260709 13:11:05.459265 31700 raft_consensus.cc:2468] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 0.
I20260709 13:11:05.459678 31775 raft_consensus.cc:2468] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 0.
I20260709 13:11:05.460431 31812 leader_election.cc:304] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16cfd719c844ef09f66734f0c255830, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:05.461306 31894 raft_consensus.cc:2804] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:05.461723 31894 raft_consensus.cc:493] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.462040 31894 raft_consensus.cc:3060] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.467258 31894 raft_consensus.cc:515] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.469136 31894 leader_election.cc:290] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 1 election: Requested vote from peers f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763), 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349)
I20260709 13:11:05.470010 31700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30ec4afb32a2428d948291815ce55045" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16cfd719c844ef09f66734f0c255830"
I20260709 13:11:05.470522 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30ec4afb32a2428d948291815ce55045" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744f86684ddb45b99a4d4d8ba669611d"
I20260709 13:11:05.470724 31700 raft_consensus.cc:3060] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.471132 31775 raft_consensus.cc:3060] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.478204 31775 raft_consensus.cc:2468] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 1.
I20260709 13:11:05.478204 31700 raft_consensus.cc:2468] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 1.
I20260709 13:11:05.479755 31812 leader_election.cc:304] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16cfd719c844ef09f66734f0c255830, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:05.480500 31894 raft_consensus.cc:2804] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:05.480991 31894 raft_consensus.cc:697] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Becoming Leader. State: Replica: ffe75d7e7037446f9207b997bd3bf3b9, State: Running, Role: LEADER
I20260709 13:11:05.481823 31894 consensus_queue.cc:237] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.488713 31598 catalog_manager.cc:5697] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:05.499480 31895 consensus_queue.cc:1048] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.512831 31928 consensus_queue.cc:1048] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:05.534991 31775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b2b0208babf4561a94ba880f5050dd8"
dest_uuid: "744f86684ddb45b99a4d4d8ba669611d"
mode: GRACEFUL
new_leader_uuid: "f16cfd719c844ef09f66734f0c255830"
 from {username='slave'} at 127.0.0.1:56760
I20260709 13:11:05.535620 31775 raft_consensus.cc:606] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 1 LEADER]: Received request to transfer leadership to TS f16cfd719c844ef09f66734f0c255830
I20260709 13:11:05.548666 31700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6a58880ac054e49a6b58c7b37a62441"
dest_uuid: "f16cfd719c844ef09f66734f0c255830"
mode: GRACEFUL
new_leader_uuid: "744f86684ddb45b99a4d4d8ba669611d"
 from {username='slave'} at 127.0.0.1:45670
I20260709 13:11:05.549244 31700 raft_consensus.cc:606] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 1 LEADER]: Received request to transfer leadership to TS 744f86684ddb45b99a4d4d8ba669611d
I20260709 13:11:05.562078 31849 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30ec4afb32a2428d948291815ce55045"
dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
mode: GRACEFUL
new_leader_uuid: "f16cfd719c844ef09f66734f0c255830"
 from {username='slave'} at 127.0.0.1:38214
I20260709 13:11:05.562629 31849 raft_consensus.cc:606] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Received request to transfer leadership to TS f16cfd719c844ef09f66734f0c255830
I20260709 13:11:05.567873 31849 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3"
dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
mode: GRACEFUL
new_leader_uuid: "f16cfd719c844ef09f66734f0c255830"
 from {username='slave'} at 127.0.0.1:38214
I20260709 13:11:05.568401 31849 raft_consensus.cc:606] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Received request to transfer leadership to TS f16cfd719c844ef09f66734f0c255830
I20260709 13:11:05.625767 31899 consensus_queue.cc:1048] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.634840 31899 raft_consensus.cc:993] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830: : Instructing follower 744f86684ddb45b99a4d4d8ba669611d to start an election
I20260709 13:11:05.635421 31934 consensus_queue.cc:1048] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:05.636039 31906 raft_consensus.cc:1081] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 1 LEADER]: Signalling peer 744f86684ddb45b99a4d4d8ba669611d to start an election
I20260709 13:11:05.637921 31774 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6a58880ac054e49a6b58c7b37a62441"
dest_uuid: "744f86684ddb45b99a4d4d8ba669611d"
 from {username='slave'} at 127.0.0.1:56742
I20260709 13:11:05.638592 31774 raft_consensus.cc:493] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:05.638971 31774 raft_consensus.cc:3060] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.649142 31774 raft_consensus.cc:515] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.650038 31775 raft_consensus.cc:1240] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Rejecting Update request from peer f16cfd719c844ef09f66734f0c255830 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:05.651051 31774 leader_election.cc:290] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [CANDIDATE]: Term 2 election: Requested vote from peers f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763), ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195:45299)
I20260709 13:11:05.653057 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a58880ac054e49a6b58c7b37a62441" candidate_uuid: "744f86684ddb45b99a4d4d8ba669611d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
I20260709 13:11:05.653057 31700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a58880ac054e49a6b58c7b37a62441" candidate_uuid: "744f86684ddb45b99a4d4d8ba669611d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f16cfd719c844ef09f66734f0c255830"
I20260709 13:11:05.653935 31849 raft_consensus.cc:3060] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.653970 31700 raft_consensus.cc:3055] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:05.654139 31899 consensus_queue.cc:1059] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:05.654628 31700 raft_consensus.cc:740] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Running, Role: LEADER
I20260709 13:11:05.655580 31700 consensus_queue.cc:260] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.656992 31700 raft_consensus.cc:3060] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.661478 31849 raft_consensus.cc:2468] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744f86684ddb45b99a4d4d8ba669611d in term 2.
I20260709 13:11:05.662734 31740 leader_election.cc:304] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744f86684ddb45b99a4d4d8ba669611d, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:05.663514 31928 raft_consensus.cc:2804] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:05.663951 31928 raft_consensus.cc:697] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 2 LEADER]: Becoming Leader. State: Replica: 744f86684ddb45b99a4d4d8ba669611d, State: Running, Role: LEADER
I20260709 13:11:05.664355 31700 raft_consensus.cc:2468] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744f86684ddb45b99a4d4d8ba669611d in term 2.
W20260709 13:11:05.664963 31906 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:11:05.664770 31928 consensus_queue.cc:237] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.673281 31598 catalog_manager.cc:5697] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d reported cstate change: term changed from 1 to 2, leader changed from f16cfd719c844ef09f66734f0c255830 (127.19.127.193) to 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194). New cstate: current_term: 2 leader_uuid: "744f86684ddb45b99a4d4d8ba669611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:05.862289 31894 consensus_queue.cc:1048] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.869593 31922 raft_consensus.cc:993] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9: : Instructing follower f16cfd719c844ef09f66734f0c255830 to start an election
I20260709 13:11:05.870414 31894 raft_consensus.cc:1081] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Signalling peer f16cfd719c844ef09f66734f0c255830 to start an election
I20260709 13:11:05.876701 31700 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3"
dest_uuid: "f16cfd719c844ef09f66734f0c255830"
 from {username='slave'} at 127.0.0.1:45654
I20260709 13:11:05.876816 31894 consensus_queue.cc:1048] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:05.877329 31700 raft_consensus.cc:493] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:05.877766 31700 raft_consensus.cc:3060] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.887053 31700 raft_consensus.cc:515] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.889842 31700 leader_election.cc:290] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 2 election: Requested vote from peers 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349), ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195:45299)
I20260709 13:11:05.891124 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
I20260709 13:11:05.891639 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "744f86684ddb45b99a4d4d8ba669611d"
I20260709 13:11:05.891878 31849 raft_consensus.cc:3055] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:05.892283 31775 raft_consensus.cc:3060] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.892253 31849 raft_consensus.cc:740] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffe75d7e7037446f9207b997bd3bf3b9, State: Running, Role: LEADER
I20260709 13:11:05.893443 31849 consensus_queue.cc:260] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.894668 31849 raft_consensus.cc:3060] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.897924 31775 raft_consensus.cc:2468] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 2.
I20260709 13:11:05.899210 31663 leader_election.cc:304] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744f86684ddb45b99a4d4d8ba669611d, f16cfd719c844ef09f66734f0c255830; no voters: 
I20260709 13:11:05.900094 31906 raft_consensus.cc:2804] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:05.900561 31906 raft_consensus.cc:697] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: Becoming Leader. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Running, Role: LEADER
I20260709 13:11:05.900632 31849 raft_consensus.cc:2468] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 2.
I20260709 13:11:05.901438 31906 consensus_queue.cc:237] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.908161 31598 catalog_manager.cc:5697] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 reported cstate change: term changed from 1 to 2, leader changed from ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195) to f16cfd719c844ef09f66734f0c255830 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "f16cfd719c844ef09f66734f0c255830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:05.911234 31937 consensus_queue.cc:1048] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.924852 31937 consensus_queue.cc:1048] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.932030 31894 raft_consensus.cc:993] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9: : Instructing follower f16cfd719c844ef09f66734f0c255830 to start an election
I20260709 13:11:05.932483 31937 raft_consensus.cc:1081] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Signalling peer f16cfd719c844ef09f66734f0c255830 to start an election
I20260709 13:11:05.933993 31700 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30ec4afb32a2428d948291815ce55045"
dest_uuid: "f16cfd719c844ef09f66734f0c255830"
 from {username='slave'} at 127.0.0.1:45654
I20260709 13:11:05.934490 31700 raft_consensus.cc:493] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:05.934866 31700 raft_consensus.cc:3060] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.940315 31700 raft_consensus.cc:515] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.941998 31700 leader_election.cc:290] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 2 election: Requested vote from peers 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349), ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195:45299)
I20260709 13:11:05.943109 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30ec4afb32a2428d948291815ce55045" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
I20260709 13:11:05.942952 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30ec4afb32a2428d948291815ce55045" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "744f86684ddb45b99a4d4d8ba669611d"
I20260709 13:11:05.943926 31849 raft_consensus.cc:3055] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:05.944032 31775 raft_consensus.cc:3060] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.944331 31849 raft_consensus.cc:740] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffe75d7e7037446f9207b997bd3bf3b9, State: Running, Role: LEADER
I20260709 13:11:05.945238 31849 consensus_queue.cc:260] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.946616 31849 raft_consensus.cc:3060] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.949877 31775 raft_consensus.cc:2468] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 2.
I20260709 13:11:05.951033 31663 leader_election.cc:304] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744f86684ddb45b99a4d4d8ba669611d, f16cfd719c844ef09f66734f0c255830; no voters: 
I20260709 13:11:05.951886 31906 raft_consensus.cc:2804] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:05.952121 31849 raft_consensus.cc:2468] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 2.
I20260709 13:11:05.952301 31906 raft_consensus.cc:697] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: Becoming Leader. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Running, Role: LEADER
I20260709 13:11:05.953233 31906 consensus_queue.cc:237] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:05.959376 31596 catalog_manager.cc:5697] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 reported cstate change: term changed from 1 to 2, leader changed from ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195) to f16cfd719c844ef09f66734f0c255830 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "f16cfd719c844ef09f66734f0c255830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.090775 31849 raft_consensus.cc:1275] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Refusing update from remote peer 744f86684ddb45b99a4d4d8ba669611d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:06.092440 31928 consensus_queue.cc:1048] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:06.102519 31700 raft_consensus.cc:1275] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Refusing update from remote peer 744f86684ddb45b99a4d4d8ba669611d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:06.104380 31928 consensus_queue.cc:1048] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:06.115056 31941 raft_consensus.cc:993] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d: : Instructing follower f16cfd719c844ef09f66734f0c255830 to start an election
I20260709 13:11:06.115635 31928 raft_consensus.cc:1081] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 1 LEADER]: Signalling peer f16cfd719c844ef09f66734f0c255830 to start an election
I20260709 13:11:06.117637 31699 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b2b0208babf4561a94ba880f5050dd8"
dest_uuid: "f16cfd719c844ef09f66734f0c255830"
 from {username='slave'} at 127.0.0.1:45650
I20260709 13:11:06.118185 31699 raft_consensus.cc:493] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:06.118546 31699 raft_consensus.cc:3060] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:06.125149 31699 raft_consensus.cc:515] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:06.127022 31699 leader_election.cc:290] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 2 election: Requested vote from peers ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195:45299), 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349)
I20260709 13:11:06.127959 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2b0208babf4561a94ba880f5050dd8" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
I20260709 13:11:06.128118 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2b0208babf4561a94ba880f5050dd8" candidate_uuid: "f16cfd719c844ef09f66734f0c255830" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "744f86684ddb45b99a4d4d8ba669611d"
I20260709 13:11:06.128727 31849 raft_consensus.cc:3060] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:06.128834 31775 raft_consensus.cc:3055] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:06.129262 31775 raft_consensus.cc:740] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 744f86684ddb45b99a4d4d8ba669611d, State: Running, Role: LEADER
I20260709 13:11:06.129941 31775 consensus_queue.cc:260] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:06.130905 31775 raft_consensus.cc:3060] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:06.134454 31849 raft_consensus.cc:2468] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 2.
I20260709 13:11:06.135885 31665 leader_election.cc:304] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16cfd719c844ef09f66734f0c255830, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:06.136224 31775 raft_consensus.cc:2468] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16cfd719c844ef09f66734f0c255830 in term 2.
I20260709 13:11:06.136669 31906 raft_consensus.cc:2804] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:06.137279 31906 raft_consensus.cc:697] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: Becoming Leader. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Running, Role: LEADER
I20260709 13:11:06.138178 31906 consensus_queue.cc:237] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } }
I20260709 13:11:06.144090 31596 catalog_manager.cc:5697] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 reported cstate change: term changed from 1 to 2, leader changed from 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194) to f16cfd719c844ef09f66734f0c255830 (127.19.127.193). New cstate: current_term: 2 leader_uuid: "f16cfd719c844ef09f66734f0c255830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.319455 19967 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:11:06.321034 19967 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:11:06.321383 19967 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:11:06.333926 31699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3"
dest_uuid: "f16cfd719c844ef09f66734f0c255830"
mode: GRACEFUL
new_leader_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
 from {username='slave'} at 127.0.0.1:45678
I20260709 13:11:06.334625 31699 raft_consensus.cc:606] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: Received request to transfer leadership to TS ffe75d7e7037446f9207b997bd3bf3b9
I20260709 13:11:06.335916 19967 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:11:06.363865 31849 raft_consensus.cc:1275] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Refusing update from remote peer f16cfd719c844ef09f66734f0c255830: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:06.365311 31906 consensus_queue.cc:1048] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:06.370671 31899 raft_consensus.cc:993] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830: : Instructing follower ffe75d7e7037446f9207b997bd3bf3b9 to start an election
I20260709 13:11:06.371482 31906 raft_consensus.cc:1081] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: Signalling peer ffe75d7e7037446f9207b997bd3bf3b9 to start an election
I20260709 13:11:06.373276 31848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3"
dest_uuid: "ffe75d7e7037446f9207b997bd3bf3b9"
 from {username='slave'} at 127.0.0.1:38206
I20260709 13:11:06.375838 31848 raft_consensus.cc:493] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:06.376040 31775 raft_consensus.cc:1275] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Refusing update from remote peer f16cfd719c844ef09f66734f0c255830: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:06.376271 31848 raft_consensus.cc:3060] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.378137 31899 consensus_queue.cc:1048] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:06.384416 31848 raft_consensus.cc:515] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:06.392478 31699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f16cfd719c844ef09f66734f0c255830"
I20260709 13:11:06.393203 31699 raft_consensus.cc:3055] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:06.393568 31699 raft_consensus.cc:740] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f16cfd719c844ef09f66734f0c255830, State: Running, Role: LEADER
I20260709 13:11:06.394141 31848 leader_election.cc:290] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 3 election: Requested vote from peers f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763), 744f86684ddb45b99a4d4d8ba669611d (127.19.127.194:34349)
I20260709 13:11:06.394696 31775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7add65d16f438283e81d877ec6c4f3" candidate_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "744f86684ddb45b99a4d4d8ba669611d"
I20260709 13:11:06.395437 31775 raft_consensus.cc:3060] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.395591 31699 consensus_queue.cc:260] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:06.396651 31699 raft_consensus.cc:3060] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.401450 31775 raft_consensus.cc:2468] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 3.
I20260709 13:11:06.401820 31699 raft_consensus.cc:2468] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe75d7e7037446f9207b997bd3bf3b9 in term 3.
I20260709 13:11:06.403008 31812 leader_election.cc:304] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744f86684ddb45b99a4d4d8ba669611d, ffe75d7e7037446f9207b997bd3bf3b9; no voters: 
I20260709 13:11:06.403859 31937 raft_consensus.cc:2804] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:06.404445 31937 raft_consensus.cc:697] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 3 LEADER]: Becoming Leader. State: Replica: ffe75d7e7037446f9207b997bd3bf3b9, State: Running, Role: LEADER
I20260709 13:11:06.405201 31937 consensus_queue.cc:237] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } }
I20260709 13:11:06.412320 31598 catalog_manager.cc:5697] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 reported cstate change: term changed from 2 to 3, leader changed from f16cfd719c844ef09f66734f0c255830 (127.19.127.193) to ffe75d7e7037446f9207b997bd3bf3b9 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16cfd719c844ef09f66734f0c255830" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 33763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744f86684ddb45b99a4d4d8ba669611d" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:06.442713 19967 tablet_server.cc:179] TabletServer@127.19.127.193:0 shutting down...
I20260709 13:11:06.465008 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:06.465766 19967 tablet_replica.cc:333] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830: stopping tablet replica
I20260709 13:11:06.466560 19967 raft_consensus.cc:2243] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.467041 19967 raft_consensus.cc:2272] T 6d7add65d16f438283e81d877ec6c4f3 P f16cfd719c844ef09f66734f0c255830 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.467278 31848 raft_consensus.cc:1275] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Refusing update from remote peer f16cfd719c844ef09f66734f0c255830: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:06.468734 31899 consensus_queue.cc:1048] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffe75d7e7037446f9207b997bd3bf3b9" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 45299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:06.469772 19967 tablet_replica.cc:333] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830: stopping tablet replica
I20260709 13:11:06.470628 19967 raft_consensus.cc:2243] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:06.471267 19967 pending_rounds.cc:70] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830: Trying to abort 1 pending ops.
I20260709 13:11:06.471561 19967 pending_rounds.cc:77] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305636519751438336 op_type: NO_OP noop_request { }
I20260709 13:11:06.471966 19967 raft_consensus.cc:2889] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:06.472316 19967 raft_consensus.cc:2272] T 30ec4afb32a2428d948291815ce55045 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.474998 19967 tablet_replica.cc:333] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830: stopping tablet replica
I20260709 13:11:06.475610 19967 raft_consensus.cc:2243] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:06.476225 19967 pending_rounds.cc:70] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830: Trying to abort 1 pending ops.
I20260709 13:11:06.476390 19967 pending_rounds.cc:77] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305636520509767680 op_type: NO_OP noop_request { }
I20260709 13:11:06.476704 19967 raft_consensus.cc:2889] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:06.477013 19967 raft_consensus.cc:2272] T 9b2b0208babf4561a94ba880f5050dd8 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.479633 19967 tablet_replica.cc:333] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830: stopping tablet replica
I20260709 13:11:06.480245 19967 raft_consensus.cc:2243] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.480674 19967 raft_consensus.cc:2272] T d6a58880ac054e49a6b58c7b37a62441 P f16cfd719c844ef09f66734f0c255830 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:06.488920 31738 consensus_peers.cc:597] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d -> Peer f16cfd719c844ef09f66734f0c255830 (127.19.127.193:33763): Couldn't send request to peer f16cfd719c844ef09f66734f0c255830. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:06.506335 19967 tablet_server.cc:196] TabletServer@127.19.127.193:0 shutdown complete.
I20260709 13:11:06.519820 19967 tablet_server.cc:179] TabletServer@127.19.127.194:0 shutting down...
I20260709 13:11:06.541563 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:06.542348 19967 tablet_replica.cc:333] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d: stopping tablet replica
I20260709 13:11:06.542945 19967 raft_consensus.cc:2243] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.543501 19967 raft_consensus.cc:2272] T 6d7add65d16f438283e81d877ec6c4f3 P 744f86684ddb45b99a4d4d8ba669611d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.546051 19967 tablet_replica.cc:333] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d: stopping tablet replica
I20260709 13:11:06.546681 19967 raft_consensus.cc:2243] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.547134 19967 raft_consensus.cc:2272] T 30ec4afb32a2428d948291815ce55045 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.549419 19967 tablet_replica.cc:333] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d: stopping tablet replica
I20260709 13:11:06.549971 19967 raft_consensus.cc:2243] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:06.550822 19967 raft_consensus.cc:2272] T d6a58880ac054e49a6b58c7b37a62441 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.553202 19967 tablet_replica.cc:333] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d: stopping tablet replica
I20260709 13:11:06.553705 19967 raft_consensus.cc:2243] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.554105 19967 raft_consensus.cc:2272] T 9b2b0208babf4561a94ba880f5050dd8 P 744f86684ddb45b99a4d4d8ba669611d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.577737 19967 tablet_server.cc:196] TabletServer@127.19.127.194:0 shutdown complete.
I20260709 13:11:06.590730 19967 tablet_server.cc:179] TabletServer@127.19.127.195:0 shutting down...
I20260709 13:11:06.610107 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:06.610849 19967 tablet_replica.cc:333] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9: stopping tablet replica
I20260709 13:11:06.611483 19967 raft_consensus.cc:2243] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:06.612172 19967 pending_rounds.cc:70] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9: Trying to abort 1 pending ops.
I20260709 13:11:06.612367 19967 pending_rounds.cc:77] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305636521604927488 op_type: NO_OP noop_request { }
I20260709 13:11:06.612646 19967 raft_consensus.cc:2889] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:06.612900 19967 raft_consensus.cc:2272] T 6d7add65d16f438283e81d877ec6c4f3 P ffe75d7e7037446f9207b997bd3bf3b9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.615118 19967 tablet_replica.cc:333] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9: stopping tablet replica
I20260709 13:11:06.615731 19967 raft_consensus.cc:2243] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.616138 19967 pending_rounds.cc:70] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9: Trying to abort 1 pending ops.
I20260709 13:11:06.616305 19967 pending_rounds.cc:77] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305636519751438336 op_type: NO_OP noop_request { }
I20260709 13:11:06.616585 19967 raft_consensus.cc:2889] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:06.616878 19967 raft_consensus.cc:2272] T 30ec4afb32a2428d948291815ce55045 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.619134 19967 tablet_replica.cc:333] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9: stopping tablet replica
I20260709 13:11:06.619697 19967 raft_consensus.cc:2243] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.620123 19967 raft_consensus.cc:2272] T 9b2b0208babf4561a94ba880f5050dd8 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.622082 19967 tablet_replica.cc:333] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9: stopping tablet replica
I20260709 13:11:06.622565 19967 raft_consensus.cc:2243] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.622989 19967 raft_consensus.cc:2272] T d6a58880ac054e49a6b58c7b37a62441 P ffe75d7e7037446f9207b997bd3bf3b9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.645958 19967 tablet_server.cc:196] TabletServer@127.19.127.195:0 shutdown complete.
I20260709 13:11:06.658917 19967 master.cc:562] Master@127.19.127.254:40833 shutting down...
I20260709 13:11:06.675249 19967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:06.675880 19967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.676214 19967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 76600a68a4d641b5bf94feb8251d89f9: stopping tablet replica
I20260709 13:11:06.687256 19967 master.cc:584] Master@127.19.127.254:40833 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2620 ms)
[----------] 6 tests from LeaderRebalancerTest (352294 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:11:06.716356 19967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.127.254:33803
I20260709 13:11:06.717523 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:06.722549 31946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:06.722708 31947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:06.724443 31949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:06.725873 19967 server_base.cc:1061] running on GCE node
I20260709 13:11:06.726791 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:06.726985 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:06.727157 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602666727143 us; error 0 us; skew 500 ppm
I20260709 13:11:06.727764 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:06.730136 19967 webserver.cc:533] Webserver started at http://127.19.127.254:36753/ using document root <none> and password file <none>
I20260709 13:11:06.730669 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:06.730861 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:06.731150 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:06.732234 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/master-0-root/instance:
uuid: "663e888d5ca24232b5e5009a4481a246"
format_stamp: "Formatted at 2026-07-09 13:11:06 on dist-test-slave-s3n6"
I20260709 13:11:06.736907 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:11:06.740226 31954 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:06.740998 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:11:06.741294 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/master-0-root
uuid: "663e888d5ca24232b5e5009a4481a246"
format_stamp: "Formatted at 2026-07-09 13:11:06 on dist-test-slave-s3n6"
I20260709 13:11:06.741586 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:06.757308 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:06.758363 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:06.798328 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.254:33803
I20260709 13:11:06.798477 32005 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.254:33803 every 8 connection(s)
I20260709 13:11:06.802263 32006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:06.813360 32006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246: Bootstrap starting.
I20260709 13:11:06.817901 32006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.823112 32006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246: No bootstrap required, opened a new log
I20260709 13:11:06.825299 32006 raft_consensus.cc:359] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663e888d5ca24232b5e5009a4481a246" member_type: VOTER }
I20260709 13:11:06.825718 32006 raft_consensus.cc:385] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.825960 32006 raft_consensus.cc:740] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663e888d5ca24232b5e5009a4481a246, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.826519 32006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663e888d5ca24232b5e5009a4481a246" member_type: VOTER }
I20260709 13:11:06.826977 32006 raft_consensus.cc:399] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:11:06.827239 32006 raft_consensus.cc:493] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:11:06.827574 32006 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.832679 32006 raft_consensus.cc:515] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663e888d5ca24232b5e5009a4481a246" member_type: VOTER }
I20260709 13:11:06.833247 32006 leader_election.cc:304] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 663e888d5ca24232b5e5009a4481a246; no voters: 
I20260709 13:11:06.834467 32006 leader_election.cc:290] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:11:06.834833 32009 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:06.836285 32009 raft_consensus.cc:697] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 1 LEADER]: Becoming Leader. State: Replica: 663e888d5ca24232b5e5009a4481a246, State: Running, Role: LEADER
I20260709 13:11:06.836992 32009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663e888d5ca24232b5e5009a4481a246" member_type: VOTER }
I20260709 13:11:06.837404 32006 sys_catalog.cc:565] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:11:06.839871 32011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 663e888d5ca24232b5e5009a4481a246. Latest consensus state: current_term: 1 leader_uuid: "663e888d5ca24232b5e5009a4481a246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663e888d5ca24232b5e5009a4481a246" member_type: VOTER } }
I20260709 13:11:06.839864 32010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "663e888d5ca24232b5e5009a4481a246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663e888d5ca24232b5e5009a4481a246" member_type: VOTER } }
I20260709 13:11:06.840690 32011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:06.840746 32010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:06.846402 32014 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:11:06.854020 32014 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:11:06.855010 19967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:11:06.863143 32014 catalog_manager.cc:1383] Generated new cluster ID: 60b28f7aee6e45d9995c8113cc8db050
I20260709 13:11:06.863526 32014 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:11:06.888553 32014 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:11:06.889991 32014 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:11:06.904891 32014 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246: Generated new TSK 0
I20260709 13:11:06.905547 32014 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:11:06.922560 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:06.929179 32027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:06.930395 32028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:06.932538 32030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:06.935164 19967 server_base.cc:1061] running on GCE node
I20260709 13:11:06.936285 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:06.936574 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:06.936774 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602666936753 us; error 0 us; skew 500 ppm
I20260709 13:11:06.937413 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:06.940196 19967 webserver.cc:533] Webserver started at http://127.19.127.193:42361/ using document root <none> and password file <none>
I20260709 13:11:06.940761 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:06.941013 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:06.941319 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:06.942524 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-0-root/instance:
uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
format_stamp: "Formatted at 2026-07-09 13:11:06 on dist-test-slave-s3n6"
I20260709 13:11:06.947957 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:11:06.951956 32035 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:06.952872 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:11:06.953233 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
format_stamp: "Formatted at 2026-07-09 13:11:06 on dist-test-slave-s3n6"
I20260709 13:11:06.953580 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:06.968001 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:06.969244 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:06.970875 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:06.973524 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:06.973783 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:06.974081 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:06.974278 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:07.015570 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:43055
I20260709 13:11:07.015667 32097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:43055 every 8 connection(s)
I20260709 13:11:07.020984 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:07.027992 32102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:07.028285 32103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:07.032224 19967 server_base.cc:1061] running on GCE node
W20260709 13:11:07.032903 32105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:07.034070 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:07.034377 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:07.034610 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602667034586 us; error 0 us; skew 500 ppm
I20260709 13:11:07.035257 32098 heartbeater.cc:344] Connected to a master server at 127.19.127.254:33803
I20260709 13:11:07.035347 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:07.035890 32098 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:07.036743 32098 heartbeater.cc:507] Master 127.19.127.254:33803 requested a full tablet report, sending...
I20260709 13:11:07.038444 19967 webserver.cc:533] Webserver started at http://127.19.127.194:39971/ using document root <none> and password file <none>
I20260709 13:11:07.039219 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:07.039247 31971 ts_manager.cc:194] Registered new tserver with Master: ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055)
I20260709 13:11:07.039574 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:07.039947 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:07.041394 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-1-root/instance:
uuid: "3bc3d543183d4ff5a6635ffb2e110227"
format_stamp: "Formatted at 2026-07-09 13:11:07 on dist-test-slave-s3n6"
I20260709 13:11:07.041867 31971 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42656
I20260709 13:11:07.046512 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:11:07.050292 32110 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:07.051791 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:11:07.052205 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-1-root
uuid: "3bc3d543183d4ff5a6635ffb2e110227"
format_stamp: "Formatted at 2026-07-09 13:11:07 on dist-test-slave-s3n6"
I20260709 13:11:07.052511 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:07.108518 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:07.109615 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:07.111902 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:07.114223 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:07.114457 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:07.114728 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:07.114912 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:07.155946 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.194:46657
I20260709 13:11:07.156044 32172 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.194:46657 every 8 connection(s)
I20260709 13:11:07.161054 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:07.168195 32176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:07.170864 32177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:07.171378 19967 server_base.cc:1061] running on GCE node
W20260709 13:11:07.172267 32179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:07.173408 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:07.173705 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:07.173923 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602667173907 us; error 0 us; skew 500 ppm
I20260709 13:11:07.174484 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:07.174921 32173 heartbeater.cc:344] Connected to a master server at 127.19.127.254:33803
I20260709 13:11:07.175431 32173 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:07.176434 32173 heartbeater.cc:507] Master 127.19.127.254:33803 requested a full tablet report, sending...
I20260709 13:11:07.177433 19967 webserver.cc:533] Webserver started at http://127.19.127.195:46491/ using document root <none> and password file <none>
I20260709 13:11:07.178045 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:07.178231 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:07.178539 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:07.178864 31971 ts_manager.cc:194] Registered new tserver with Master: 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:07.179738 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-2-root/instance:
uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
format_stamp: "Formatted at 2026-07-09 13:11:07 on dist-test-slave-s3n6"
I20260709 13:11:07.180464 31971 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42672
I20260709 13:11:07.184978 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:11:07.188807 32184 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:07.189733 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:11:07.190083 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-2-root
uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
format_stamp: "Formatted at 2026-07-09 13:11:07 on dist-test-slave-s3n6"
I20260709 13:11:07.190387 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602314247669-19967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:07.202220 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:07.203400 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:07.204825 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:07.207139 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:07.207424 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.001s
I20260709 13:11:07.207691 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:07.207873 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:07.247949 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.195:43043
I20260709 13:11:07.248081 32246 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.195:43043 every 8 connection(s)
I20260709 13:11:07.262656 32247 heartbeater.cc:344] Connected to a master server at 127.19.127.254:33803
I20260709 13:11:07.263162 32247 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:07.263963 32247 heartbeater.cc:507] Master 127.19.127.254:33803 requested a full tablet report, sending...
I20260709 13:11:07.266208 31971 ts_manager.cc:194] Registered new tserver with Master: 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:07.266950 19967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01540039s
I20260709 13:11:07.268014 31971 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42680
I20260709 13:11:07.294641 31971 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42690:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:07.297334 31971 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:07.373052 32138 tablet_service.cc:1511] Processing CreateTablet for tablet c2b786f061514123aca8f3d1750e39d9 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:07.373672 32136 tablet_service.cc:1511] Processing CreateTablet for tablet 3dda35f1227549ff933e504aadd37233 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:07.374389 32138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2b786f061514123aca8f3d1750e39d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.375370 32136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dda35f1227549ff933e504aadd37233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.376642 32135 tablet_service.cc:1511] Processing CreateTablet for tablet cc00c7f1fec24b569dd708c0b47e1ae7 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:07.378065 32135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc00c7f1fec24b569dd708c0b47e1ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.376710 32137 tablet_service.cc:1511] Processing CreateTablet for tablet b28632f2919847d2b35e28a7baff3fa0 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:07.384897 32137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b28632f2919847d2b35e28a7baff3fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.388118 32134 tablet_service.cc:1511] Processing CreateTablet for tablet fdef32d47736414d8b9a2f7d897ecee8 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:07.388223 32133 tablet_service.cc:1511] Processing CreateTablet for tablet 93f3937eb75d421abe285280d24d430d (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:07.389762 32134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdef32d47736414d8b9a2f7d897ecee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.391971 32133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f3937eb75d421abe285280d24d430d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.397456 32132 tablet_service.cc:1511] Processing CreateTablet for tablet f604db35eda34a43997c049f9d49baa3 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:07.398954 32132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f604db35eda34a43997c049f9d49baa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.400465 32131 tablet_service.cc:1511] Processing CreateTablet for tablet 9e05e529aa954954acd28cf6021b0d35 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:07.400556 32212 tablet_service.cc:1511] Processing CreateTablet for tablet c2b786f061514123aca8f3d1750e39d9 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:07.402132 32212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2b786f061514123aca8f3d1750e39d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.404568 32211 tablet_service.cc:1511] Processing CreateTablet for tablet b28632f2919847d2b35e28a7baff3fa0 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:07.407660 32131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e05e529aa954954acd28cf6021b0d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.410457 32209 tablet_service.cc:1511] Processing CreateTablet for tablet cc00c7f1fec24b569dd708c0b47e1ae7 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:07.411429 32208 tablet_service.cc:1511] Processing CreateTablet for tablet fdef32d47736414d8b9a2f7d897ecee8 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:07.412127 32211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b28632f2919847d2b35e28a7baff3fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.413854 32207 tablet_service.cc:1511] Processing CreateTablet for tablet 93f3937eb75d421abe285280d24d430d (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:07.416252 32206 tablet_service.cc:1511] Processing CreateTablet for tablet f604db35eda34a43997c049f9d49baa3 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:07.417665 32208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdef32d47736414d8b9a2f7d897ecee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.410468 32210 tablet_service.cc:1511] Processing CreateTablet for tablet 3dda35f1227549ff933e504aadd37233 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:07.419973 32210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dda35f1227549ff933e504aadd37233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.423946 32209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc00c7f1fec24b569dd708c0b47e1ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.427945 32206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f604db35eda34a43997c049f9d49baa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.429484 32207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f3937eb75d421abe285280d24d430d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.430363 32063 tablet_service.cc:1511] Processing CreateTablet for tablet c2b786f061514123aca8f3d1750e39d9 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:07.431840 32063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2b786f061514123aca8f3d1750e39d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.436273 32062 tablet_service.cc:1511] Processing CreateTablet for tablet b28632f2919847d2b35e28a7baff3fa0 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:07.438236 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b28632f2919847d2b35e28a7baff3fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.440707 32205 tablet_service.cc:1511] Processing CreateTablet for tablet 9e05e529aa954954acd28cf6021b0d35 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:07.442677 32205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e05e529aa954954acd28cf6021b0d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.444837 32061 tablet_service.cc:1511] Processing CreateTablet for tablet 3dda35f1227549ff933e504aadd37233 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:07.446727 32061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dda35f1227549ff933e504aadd37233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.448853 32204 tablet_service.cc:1511] Processing CreateTablet for tablet f27ad9f1bb0349b4bb39afe329cf59d8 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:07.451134 32204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27ad9f1bb0349b4bb39afe329cf59d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.462857 32060 tablet_service.cc:1511] Processing CreateTablet for tablet cc00c7f1fec24b569dd708c0b47e1ae7 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:07.470191 32060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc00c7f1fec24b569dd708c0b47e1ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.485221 32138 tablet_service.cc:1511] Processing CreateTablet for tablet f27ad9f1bb0349b4bb39afe329cf59d8 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:07.486876 32138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27ad9f1bb0349b4bb39afe329cf59d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.526851 32056 tablet_service.cc:1511] Processing CreateTablet for tablet 9e05e529aa954954acd28cf6021b0d35 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:07.528543 32056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e05e529aa954954acd28cf6021b0d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.532640 32055 tablet_service.cc:1511] Processing CreateTablet for tablet f27ad9f1bb0349b4bb39afe329cf59d8 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:07.534300 32055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27ad9f1bb0349b4bb39afe329cf59d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.524745 32059 tablet_service.cc:1511] Processing CreateTablet for tablet fdef32d47736414d8b9a2f7d897ecee8 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:07.541065 32057 tablet_service.cc:1511] Processing CreateTablet for tablet f604db35eda34a43997c049f9d49baa3 (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:07.544484 32058 tablet_service.cc:1511] Processing CreateTablet for tablet 93f3937eb75d421abe285280d24d430d (DEFAULT_TABLE table=test-workload [id=f7902a4cff074014aeece3a41cbe9fe6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:07.571540 32262 tablet_bootstrap.cc:492] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:07.580380 32059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdef32d47736414d8b9a2f7d897ecee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.590260 32057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f604db35eda34a43997c049f9d49baa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.592228 32058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f3937eb75d421abe285280d24d430d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:07.602376 32263 tablet_bootstrap.cc:492] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:07.619755 32264 tablet_bootstrap.cc:492] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:07.625286 32262 tablet_bootstrap.cc:654] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.632318 32262 tablet_bootstrap.cc:492] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:07.632797 32262 ts_tablet_manager.cc:1403] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.063s	user 0.004s	sys 0.007s
I20260709 13:11:07.633386 32263 tablet_bootstrap.cc:654] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.634936 32262 raft_consensus.cc:359] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.635746 32262 raft_consensus.cc:385] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.636062 32262 raft_consensus.cc:740] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.636859 32262 consensus_queue.cc:260] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.639431 32173 heartbeater.cc:499] Master 127.19.127.254:33803 was elected leader, sending a full tablet report...
I20260709 13:11:07.639631 32263 tablet_bootstrap.cc:492] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:07.639902 32262 ts_tablet_manager.cc:1434] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:07.640393 32263 ts_tablet_manager.cc:1403] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.002s
I20260709 13:11:07.641031 32262 tablet_bootstrap.cc:492] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:07.643764 32263 raft_consensus.cc:359] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.644655 32263 raft_consensus.cc:385] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.645021 32263 raft_consensus.cc:740] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.645999 32263 consensus_queue.cc:260] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.647643 32262 tablet_bootstrap.cc:654] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.650983 32264 tablet_bootstrap.cc:654] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.655920 32263 ts_tablet_manager.cc:1434] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.012s
I20260709 13:11:07.656883 32247 heartbeater.cc:499] Master 127.19.127.254:33803 was elected leader, sending a full tablet report...
I20260709 13:11:07.671446 32263 tablet_bootstrap.cc:492] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:07.671964 32264 tablet_bootstrap.cc:492] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:07.672122 32262 tablet_bootstrap.cc:492] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:07.672573 32264 ts_tablet_manager.cc:1403] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.053s	user 0.018s	sys 0.006s
I20260709 13:11:07.672747 32262 ts_tablet_manager.cc:1403] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.019s
I20260709 13:11:07.675952 32264 raft_consensus.cc:359] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.676231 32262 raft_consensus.cc:359] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:07.676842 32264 raft_consensus.cc:385] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.677138 32262 raft_consensus.cc:385] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.677243 32264 raft_consensus.cc:740] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.677603 32262 raft_consensus.cc:740] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.678256 32264 consensus_queue.cc:260] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.678687 32263 tablet_bootstrap.cc:654] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.678740 32262 consensus_queue.cc:260] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:07.683766 32262 ts_tablet_manager.cc:1434] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 13:11:07.685138 32262 tablet_bootstrap.cc:492] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:07.693799 32262 tablet_bootstrap.cc:654] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.694463 32263 tablet_bootstrap.cc:492] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:07.695083 32263 ts_tablet_manager.cc:1403] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.014s
I20260709 13:11:07.698767 32263 raft_consensus.cc:359] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.699793 32263 raft_consensus.cc:385] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.700182 32263 raft_consensus.cc:740] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.701359 32263 consensus_queue.cc:260] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.707724 32263 ts_tablet_manager.cc:1434] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:11:07.709206 32263 tablet_bootstrap.cc:492] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:07.717880 32263 tablet_bootstrap.cc:654] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.728341 32098 heartbeater.cc:499] Master 127.19.127.254:33803 was elected leader, sending a full tablet report...
I20260709 13:11:07.728709 32264 ts_tablet_manager.cc:1434] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.056s	user 0.022s	sys 0.032s
I20260709 13:11:07.729837 32264 tablet_bootstrap.cc:492] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:07.731149 32262 tablet_bootstrap.cc:492] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:07.731791 32262 ts_tablet_manager.cc:1403] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.047s	user 0.026s	sys 0.019s
I20260709 13:11:07.734782 32262 raft_consensus.cc:359] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.735589 32262 raft_consensus.cc:385] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.735879 32264 tablet_bootstrap.cc:654] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.735906 32262 raft_consensus.cc:740] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.737097 32262 consensus_queue.cc:260] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.737702 32263 tablet_bootstrap.cc:492] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:07.738435 32263 ts_tablet_manager.cc:1403] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.012s
I20260709 13:11:07.742425 32263 raft_consensus.cc:359] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.743848 32263 raft_consensus.cc:385] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.744113 32262 ts_tablet_manager.cc:1434] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:11:07.744382 32263 raft_consensus.cc:740] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.745366 32262 tablet_bootstrap.cc:492] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:07.745762 32263 consensus_queue.cc:260] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.749286 32263 ts_tablet_manager.cc:1434] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:11:07.750837 32263 tablet_bootstrap.cc:492] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:07.751912 32262 tablet_bootstrap.cc:654] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.757877 32263 tablet_bootstrap.cc:654] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.759835 32264 tablet_bootstrap.cc:492] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:07.760413 32264 ts_tablet_manager.cc:1403] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.008s
I20260709 13:11:07.764330 32264 raft_consensus.cc:359] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.765362 32264 raft_consensus.cc:385] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.765751 32264 raft_consensus.cc:740] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.767057 32264 consensus_queue.cc:260] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.769428 32264 ts_tablet_manager.cc:1434] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:11:07.770081 32263 tablet_bootstrap.cc:492] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:07.770520 32264 tablet_bootstrap.cc:492] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:07.770710 32263 ts_tablet_manager.cc:1403] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 13:11:07.774871 32263 raft_consensus.cc:359] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.775712 32263 raft_consensus.cc:385] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.776012 32263 raft_consensus.cc:740] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.776691 32262 tablet_bootstrap.cc:492] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:07.777420 32262 ts_tablet_manager.cc:1403] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.032s	user 0.025s	sys 0.000s
I20260709 13:11:07.777535 32264 tablet_bootstrap.cc:654] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.776800 32263 consensus_queue.cc:260] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.780400 32262 raft_consensus.cc:359] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.781180 32262 raft_consensus.cc:385] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.781492 32262 raft_consensus.cc:740] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.782246 32262 consensus_queue.cc:260] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.782776 32263 ts_tablet_manager.cc:1434] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:11:07.786577 32263 tablet_bootstrap.cc:492] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:07.787588 32267 raft_consensus.cc:493] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.788112 32267 raft_consensus.cc:515] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.792670 32267 leader_election.cc:290] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:07.793285 32263 tablet_bootstrap.cc:654] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.784688 32262 ts_tablet_manager.cc:1434] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:07.800688 32264 tablet_bootstrap.cc:492] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:07.801236 32264 ts_tablet_manager.cc:1403] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.004s
I20260709 13:11:07.802224 32262 tablet_bootstrap.cc:492] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:07.804973 32264 raft_consensus.cc:359] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.805729 32264 raft_consensus.cc:385] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.806026 32264 raft_consensus.cc:740] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.811074 32264 consensus_queue.cc:260] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.813731 32262 tablet_bootstrap.cc:654] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.816437 32264 ts_tablet_manager.cc:1434] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.003s
I20260709 13:11:07.817639 32264 tablet_bootstrap.cc:492] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:07.822961 32263 tablet_bootstrap.cc:492] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:07.823915 32263 ts_tablet_manager.cc:1403] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.015s
I20260709 13:11:07.827697 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3937eb75d421abe285280d24d430d" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:07.827908 32263 raft_consensus.cc:359] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.828907 32263 raft_consensus.cc:385] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.829279 32263 raft_consensus.cc:740] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.829859 32262 tablet_bootstrap.cc:492] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
W20260709 13:11:07.829954 32114 leader_election.cc:343] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.830691 32262 ts_tablet_manager.cc:1403] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.013s
I20260709 13:11:07.830145 32263 consensus_queue.cc:260] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.831247 32271 raft_consensus.cc:493] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.831894 32271 raft_consensus.cc:515] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.833488 32263 ts_tablet_manager.cc:1434] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:11:07.834018 32262 raft_consensus.cc:359] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.834700 32263 tablet_bootstrap.cc:492] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:07.834833 32262 raft_consensus.cc:385] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.835335 32262 raft_consensus.cc:740] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.836987 32271 leader_election.cc:290] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:07.836671 32262 consensus_queue.cc:260] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.840126 32262 ts_tablet_manager.cc:1434] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:11:07.843698 32268 raft_consensus.cc:493] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:07.848171 32019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:07.848453 32264 tablet_bootstrap.cc:654] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.852342 32262 tablet_bootstrap.cc:492] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:07.854568 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3937eb75d421abe285280d24d430d" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:07.855418 32222 raft_consensus.cc:2468] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:07.851516 32268 raft_consensus.cc:515] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.860173 32268 leader_election.cc:290] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:07.864718 32113 leader_election.cc:304] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:07.865262 32263 tablet_bootstrap.cc:654] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.866178 32267 raft_consensus.cc:2804] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:07.866649 32267 raft_consensus.cc:493] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.867022 32267 raft_consensus.cc:3060] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.869500 32268 raft_consensus.cc:493] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.870074 32268 raft_consensus.cc:515] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.872653 32268 leader_election.cc:290] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:07.885717 32268 raft_consensus.cc:493] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.886250 32268 raft_consensus.cc:515] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.888589 32268 leader_election.cc:290] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:07.891664 32262 tablet_bootstrap.cc:654] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.888216 32267 raft_consensus.cc:515] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.897583 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3937eb75d421abe285280d24d430d" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:07.898979 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28632f2919847d2b35e28a7baff3fa0" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:07.899837 32148 raft_consensus.cc:2468] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 0.
W20260709 13:11:07.899202 32114 leader_election.cc:343] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.901278 32038 leader_election.cc:304] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:07.902597 32267 leader_election.cc:290] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:07.902843 32271 raft_consensus.cc:2804] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:07.903254 32271 raft_consensus.cc:493] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.903643 32271 raft_consensus.cc:3060] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.905543 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3937eb75d421abe285280d24d430d" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:07.906420 32222 raft_consensus.cc:3060] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.910849 32221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28632f2919847d2b35e28a7baff3fa0" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:07.911644 32267 raft_consensus.cc:493] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.912011 32221 raft_consensus.cc:2468] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 0.
I20260709 13:11:07.912245 32267 raft_consensus.cc:515] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:07.915385 32221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:07.915490 32267 leader_election.cc:290] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043), ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055)
I20260709 13:11:07.916109 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
W20260709 13:11:07.916711 32113 leader_election.cc:343] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:07.917807 32114 leader_election.cc:343] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.918255 32114 leader_election.cc:304] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:07.919121 32267 raft_consensus.cc:2749] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:07.925772 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:07.925508 32271 raft_consensus.cc:515] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.928740 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28632f2919847d2b35e28a7baff3fa0" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:07.930327 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28632f2919847d2b35e28a7baff3fa0" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:07.930416 32073 raft_consensus.cc:2393] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b650608729b4ad6a28fd1c7e3d30d1c in current term 1: Already voted for candidate ee90f5b2a1e849739b82a66fdce52a8b in this term.
I20260709 13:11:07.931973 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:07.932648 32264 tablet_bootstrap.cc:492] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:07.933218 32264 ts_tablet_manager.cc:1403] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.116s	user 0.016s	sys 0.005s
I20260709 13:11:07.933022 32072 raft_consensus.cc:2468] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:07.934608 32221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28632f2919847d2b35e28a7baff3fa0" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:07.935488 32221 raft_consensus.cc:3060] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.935595 32148 raft_consensus.cc:3060] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.939500 32147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:07.928184 32271 leader_election.cc:290] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:07.940335 32147 raft_consensus.cc:2468] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:07.940701 32271 raft_consensus.cc:493] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.930796 32222 raft_consensus.cc:2468] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
W20260709 13:11:07.929271 32188 leader_election.cc:343] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.941679 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28632f2919847d2b35e28a7baff3fa0" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:07.942306 32188 leader_election.cc:304] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:07.943341 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:07.944204 32113 leader_election.cc:304] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:07.943528 32268 raft_consensus.cc:2804] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:07.945005 32268 raft_consensus.cc:493] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.945116 32267 raft_consensus.cc:2804] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:07.945384 32268 raft_consensus.cc:3060] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.947152 32187 leader_election.cc:304] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:07.941170 32271 raft_consensus.cc:515] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.950923 32264 raft_consensus.cc:359] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.958210 32263 tablet_bootstrap.cc:492] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:07.962996 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:07.963469 32263 ts_tablet_manager.cc:1403] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.129s	user 0.028s	sys 0.004s
I20260709 13:11:07.959769 32262 tablet_bootstrap.cc:492] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:07.958123 32268 raft_consensus.cc:515] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.960994 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:07.967713 32262 ts_tablet_manager.cc:1403] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.116s	user 0.018s	sys 0.011s
W20260709 13:11:07.968999 32038 leader_election.cc:343] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.969967 32222 raft_consensus.cc:2393] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee90f5b2a1e849739b82a66fdce52a8b in current term 1: Already voted for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in this term.
I20260709 13:11:07.970160 32268 leader_election.cc:290] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:07.963927 32264 raft_consensus.cc:385] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.966688 32148 raft_consensus.cc:2468] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 1.
I20260709 13:11:07.966392 32263 raft_consensus.cc:359] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.962392 32221 raft_consensus.cc:2468] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 1.
I20260709 13:11:07.972486 32268 raft_consensus.cc:2804] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:07.973040 32263 raft_consensus.cc:385] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.973081 32268 raft_consensus.cc:493] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.973466 32038 leader_election.cc:304] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:07.973598 32263 raft_consensus.cc:740] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.973798 32268 raft_consensus.cc:3060] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.974730 32263 consensus_queue.cc:260] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.971508 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:07.975832 32072 raft_consensus.cc:3060] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.975998 32038 leader_election.cc:304] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee90f5b2a1e849739b82a66fdce52a8b; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:07.971271 32264 raft_consensus.cc:740] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.967713 32271 leader_election.cc:290] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:07.970911 32267 raft_consensus.cc:697] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:07.982167 32267 consensus_queue.cc:237] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.983246 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:07.982486 32187 leader_election.cc:304] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c; no voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:07.984791 32293 raft_consensus.cc:2749] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:07.986244 32187 leader_election.cc:343] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.985987 32262 raft_consensus.cc:359] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.987380 32262 raft_consensus.cc:385] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.987437 32291 raft_consensus.cc:2804] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:07.989599 32072 raft_consensus.cc:2468] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:07.990724 32262 raft_consensus.cc:740] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.992547 32263 ts_tablet_manager.cc:1434] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:11:07.992378 32262 consensus_queue.cc:260] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.986838 32264 consensus_queue.cc:260] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:07.994904 32264 ts_tablet_manager.cc:1434] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.002s
I20260709 13:11:07.991051 32271 raft_consensus.cc:2749] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:07.995802 32187 leader_election.cc:343] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.994904 32262 ts_tablet_manager.cc:1434] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 13:11:07.996225 32293 raft_consensus.cc:493] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.999821 32293 raft_consensus.cc:515] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.002197 32293 leader_election.cc:290] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:08.002075 32268 raft_consensus.cc:515] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.004494 32263 tablet_bootstrap.cc:492] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:08.005525 32188 leader_election.cc:304] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:08.006124 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:08.006824 32293 raft_consensus.cc:2804] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:08.005017 32264 tablet_bootstrap.cc:492] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:08.007686 32291 raft_consensus.cc:697] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:08.008520 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:08.004442 32268 leader_election.cc:290] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
W20260709 13:11:08.007704 32188 leader_election.cc:343] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:08.011664 32262 tablet_bootstrap.cc:492] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:08.020731 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
W20260709 13:11:08.023726 32188 leader_election.cc:343] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:08.021822 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:08.024887 32145 raft_consensus.cc:3060] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.018532 32263 tablet_bootstrap.cc:654] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.016453 32291 consensus_queue.cc:237] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.027056 31971 catalog_manager.cc:5697] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.019444 32293 raft_consensus.cc:697] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:08.031033 32264 tablet_bootstrap.cc:654] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.035566 32293 consensus_queue.cc:237] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
W20260709 13:11:08.035952 32187 leader_election.cc:343] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:08.036736 32187 leader_election.cc:304] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c; no voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:08.036170 32267 raft_consensus.cc:493] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.037912 32297 raft_consensus.cc:2749] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:08.038285 32267 raft_consensus.cc:515] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.042004 32292 raft_consensus.cc:493] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.042570 32292 raft_consensus.cc:515] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.047076 32267 leader_election.cc:290] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.046839 32145 raft_consensus.cc:2468] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:08.049261 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:08.051959 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dda35f1227549ff933e504aadd37233" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:08.053303 32073 raft_consensus.cc:2468] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:08.053551 32072 raft_consensus.cc:2393] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bc3d543183d4ff5a6635ffb2e110227 in current term 1: Already voted for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in this term.
I20260709 13:11:08.055075 32114 leader_election.cc:304] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:08.056751 32267 raft_consensus.cc:2804] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:08.057533 32267 raft_consensus.cc:493] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.057940 32267 raft_consensus.cc:3060] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.060070 32262 tablet_bootstrap.cc:654] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.066311 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:08.067662 32113 leader_election.cc:304] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:08.070686 31970 catalog_manager.cc:5697] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 1 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.063198 32187 leader_election.cc:304] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:08.081893 32292 leader_election.cc:290] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.081519 31969 catalog_manager.cc:5697] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 1 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:08.082535 32292 raft_consensus.cc:3060] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.088047 32293 raft_consensus.cc:2804] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:08.089249 32293 raft_consensus.cc:697] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:08.090116 32293 consensus_queue.cc:237] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.092222 32292 raft_consensus.cc:2749] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:08.095703 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dda35f1227549ff933e504aadd37233" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:08.096813 32222 raft_consensus.cc:2468] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:08.099020 32267 raft_consensus.cc:515] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.100391 32264 tablet_bootstrap.cc:492] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:08.100950 32264 ts_tablet_manager.cc:1403] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.096s	user 0.011s	sys 0.008s
I20260709 13:11:08.101539 32267 leader_election.cc:290] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.102748 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dda35f1227549ff933e504aadd37233" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:08.103544 32263 tablet_bootstrap.cc:492] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:08.103681 32072 raft_consensus.cc:3060] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.104130 32263 ts_tablet_manager.cc:1403] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.100s	user 0.015s	sys 0.016s
I20260709 13:11:08.104272 32264 raft_consensus.cc:359] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.105082 32264 raft_consensus.cc:385] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.105391 32264 raft_consensus.cc:740] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.106293 32264 consensus_queue.cc:260] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.109433 32263 raft_consensus.cc:359] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.110057 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dda35f1227549ff933e504aadd37233" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:08.110348 32263 raft_consensus.cc:385] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.110669 32263 raft_consensus.cc:740] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.110704 32222 raft_consensus.cc:3060] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.111553 32263 consensus_queue.cc:260] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.113369 32072 raft_consensus.cc:2468] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:08.113844 32263 ts_tablet_manager.cc:1434] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.005s
I20260709 13:11:08.114454 32114 leader_election.cc:304] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:08.114977 32263 tablet_bootstrap.cc:492] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:08.115126 32267 raft_consensus.cc:2804] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:08.115770 32267 raft_consensus.cc:697] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:08.116571 32267 consensus_queue.cc:237] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.118616 32222 raft_consensus.cc:2468] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:08.121062 32264 ts_tablet_manager.cc:1434] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:11:08.122251 32264 tablet_bootstrap.cc:492] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:08.122641 32262 tablet_bootstrap.cc:492] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:08.123257 32262 ts_tablet_manager.cc:1403] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.113s	user 0.017s	sys 0.020s
I20260709 13:11:08.124140 32263 tablet_bootstrap.cc:654] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.126612 32262 raft_consensus.cc:359] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.128219 32262 raft_consensus.cc:385] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.128659 32262 raft_consensus.cc:740] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.129963 32262 consensus_queue.cc:260] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.138099 32264 tablet_bootstrap.cc:654] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.140429 32291 raft_consensus.cc:493] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.141047 32291 raft_consensus.cc:515] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.144722 32291 leader_election.cc:290] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.145555 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e05e529aa954954acd28cf6021b0d35" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:08.149629 32145 raft_consensus.cc:2468] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 0.
I20260709 13:11:08.151610 32038 leader_election.cc:304] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:08.151685 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e05e529aa954954acd28cf6021b0d35" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:08.152812 32222 raft_consensus.cc:2468] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 0.
I20260709 13:11:08.153126 32296 raft_consensus.cc:2804] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:08.153975 32296 raft_consensus.cc:493] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.154356 32296 raft_consensus.cc:3060] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.156580 32262 ts_tablet_manager.cc:1434] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 13:11:08.158192 32262 tablet_bootstrap.cc:492] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:08.163555 32263 tablet_bootstrap.cc:492] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:08.164160 32263 ts_tablet_manager.cc:1403] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.004s
I20260709 13:11:08.165688 32264 tablet_bootstrap.cc:492] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:08.166258 32264 ts_tablet_manager.cc:1403] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.006s
I20260709 13:11:08.167176 32263 raft_consensus.cc:359] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.167972 32263 raft_consensus.cc:385] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.168248 32263 raft_consensus.cc:740] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.168967 32263 consensus_queue.cc:260] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.167665 32262 tablet_bootstrap.cc:654] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.171618 32263 ts_tablet_manager.cc:1434] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:08.172703 32263 tablet_bootstrap.cc:492] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:08.173316 32296 raft_consensus.cc:515] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.177683 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e05e529aa954954acd28cf6021b0d35" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:08.178257 32296 leader_election.cc:290] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.169873 32264 raft_consensus.cc:359] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.178735 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e05e529aa954954acd28cf6021b0d35" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:08.178951 32263 tablet_bootstrap.cc:654] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.179401 32264 raft_consensus.cc:385] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.179512 32222 raft_consensus.cc:3060] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.180014 32264 raft_consensus.cc:740] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.178368 32145 raft_consensus.cc:3060] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.181234 32264 consensus_queue.cc:260] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.185338 32264 ts_tablet_manager.cc:1434] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.003s
I20260709 13:11:08.186434 32264 tablet_bootstrap.cc:492] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:08.187773 31971 catalog_manager.cc:5697] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.192764 32264 tablet_bootstrap.cc:654] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.199168 32222 raft_consensus.cc:2468] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 1.
I20260709 13:11:08.200534 32038 leader_election.cc:304] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:08.201411 32296 raft_consensus.cc:2804] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:08.201897 32296 raft_consensus.cc:697] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:08.202193 32145 raft_consensus.cc:2468] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 1.
I20260709 13:11:08.202692 32296 consensus_queue.cc:237] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.206157 32262 tablet_bootstrap.cc:492] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:08.206696 32262 ts_tablet_manager.cc:1403] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.019s
I20260709 13:11:08.209023 32262 raft_consensus.cc:359] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.209697 32262 raft_consensus.cc:385] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.209980 32262 raft_consensus.cc:740] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.210587 32262 consensus_queue.cc:260] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.212709 32262 ts_tablet_manager.cc:1434] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:11:08.213400 31969 catalog_manager.cc:5697] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 1 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:08.215574 32263 tablet_bootstrap.cc:492] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:08.216255 32263 ts_tablet_manager.cc:1403] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.007s
I20260709 13:11:08.217211 32262 tablet_bootstrap.cc:492] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:08.223546 32262 tablet_bootstrap.cc:654] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.227192 32264 tablet_bootstrap.cc:492] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:08.227762 32264 ts_tablet_manager.cc:1403] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.011s
I20260709 13:11:08.227831 32263 raft_consensus.cc:359] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.231030 32264 raft_consensus.cc:359] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.231948 32264 raft_consensus.cc:385] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.230492 31971 catalog_manager.cc:5697] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 1 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.232257 32264 raft_consensus.cc:740] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.233156 32264 consensus_queue.cc:260] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.235773 32264 ts_tablet_manager.cc:1434] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:11:08.237255 32264 tablet_bootstrap.cc:492] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:08.239274 32262 tablet_bootstrap.cc:492] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:08.228587 32263 raft_consensus.cc:385] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.239935 32262 ts_tablet_manager.cc:1403] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.004s
I20260709 13:11:08.240134 32263 raft_consensus.cc:740] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.241145 32263 consensus_queue.cc:260] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.243045 32262 raft_consensus.cc:359] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.243527 32264 tablet_bootstrap.cc:654] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.244077 32262 raft_consensus.cc:385] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.244458 32262 raft_consensus.cc:740] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.245313 32262 consensus_queue.cc:260] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.250825 32262 ts_tablet_manager.cc:1434] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:11:08.251916 32263 ts_tablet_manager.cc:1434] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.001s
I20260709 13:11:08.261492 32264 tablet_bootstrap.cc:492] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:08.262059 32264 ts_tablet_manager.cc:1403] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.018s
I20260709 13:11:08.264120 32264 raft_consensus.cc:359] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.264658 32264 raft_consensus.cc:385] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.264894 32264 raft_consensus.cc:740] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.265450 32264 consensus_queue.cc:260] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.267246 32264 ts_tablet_manager.cc:1434] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.003s
I20260709 13:11:08.268466 32264 tablet_bootstrap.cc:492] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:08.272603 32264 tablet_bootstrap.cc:654] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:08.276502 32264 tablet_bootstrap.cc:492] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:08.276942 32264 ts_tablet_manager.cc:1403] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:11:08.279019 32264 raft_consensus.cc:359] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.279595 32264 raft_consensus.cc:385] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:08.279815 32264 raft_consensus.cc:740] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:08.280339 32264 consensus_queue.cc:260] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.281739 32264 ts_tablet_manager.cc:1434] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:08.311523 32267 raft_consensus.cc:493] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.312142 32267 raft_consensus.cc:515] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.314100 32267 leader_election.cc:290] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.315037 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef32d47736414d8b9a2f7d897ecee8" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:08.315769 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef32d47736414d8b9a2f7d897ecee8" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:08.315807 32072 raft_consensus.cc:2468] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:08.316601 32222 raft_consensus.cc:2468] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:08.317392 32114 leader_election.cc:304] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:08.318302 32267 raft_consensus.cc:2804] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:08.318691 32267 raft_consensus.cc:493] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.319046 32267 raft_consensus.cc:3060] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.326114 32267 raft_consensus.cc:515] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.361723 32267 leader_election.cc:290] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.364279 32296 raft_consensus.cc:493] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.365924 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef32d47736414d8b9a2f7d897ecee8" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:08.365831 32296 raft_consensus.cc:515] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.367245 32072 raft_consensus.cc:3060] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.367197 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdef32d47736414d8b9a2f7d897ecee8" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:08.368410 32222 raft_consensus.cc:3060] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.369721 32296 leader_election.cc:290] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.371812 32221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3937eb75d421abe285280d24d430d" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:08.372159 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3937eb75d421abe285280d24d430d" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:08.373137 32221 raft_consensus.cc:2393] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee90f5b2a1e849739b82a66fdce52a8b in current term 1: Already voted for candidate 3bc3d543183d4ff5a6635ffb2e110227 in this term.
I20260709 13:11:08.375356 32038 leader_election.cc:304] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee90f5b2a1e849739b82a66fdce52a8b; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:08.376433 32296 raft_consensus.cc:3060] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.381692 32222 raft_consensus.cc:2468] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:08.382265 32072 raft_consensus.cc:2468] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:08.383651 32113 leader_election.cc:304] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:08.384907 32267 raft_consensus.cc:2804] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:08.385485 32267 raft_consensus.cc:697] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:08.386615 32296 raft_consensus.cc:2749] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:08.387081 32267 consensus_queue.cc:237] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.398343 31970 catalog_manager.cc:5697] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.427107 32296 raft_consensus.cc:493] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.427898 32296 raft_consensus.cc:515] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.429986 32296 leader_election.cc:290] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.430925 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:08.431325 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:08.432117 32145 raft_consensus.cc:2393] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee90f5b2a1e849739b82a66fdce52a8b in current term 1: Already voted for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in this term.
I20260709 13:11:08.433210 32038 leader_election.cc:304] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee90f5b2a1e849739b82a66fdce52a8b; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:08.434031 32296 raft_consensus.cc:3060] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.440907 32296 raft_consensus.cc:2749] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:08.495530 32267 raft_consensus.cc:493] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.496201 32267 raft_consensus.cc:515] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.498284 32267 leader_election.cc:290] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.499181 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:08.499459 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:08.500139 32072 raft_consensus.cc:2468] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:08.500376 32222 raft_consensus.cc:2468] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:08.501718 32113 leader_election.cc:304] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:08.502547 32267 raft_consensus.cc:2804] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:08.502944 32267 raft_consensus.cc:493] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.503324 32267 raft_consensus.cc:3060] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.504608 32293 raft_consensus.cc:493] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.505658 32293 raft_consensus.cc:515] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.509800 32303 consensus_queue.cc:1048] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.510389 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:08.511449 32148 raft_consensus.cc:2468] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:08.513245 32187 leader_election.cc:304] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:08.514374 32297 raft_consensus.cc:2804] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:08.514875 32297 raft_consensus.cc:493] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.515264 32297 raft_consensus.cc:3060] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.517714 32291 consensus_queue.cc:1048] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.522150 32267 raft_consensus.cc:515] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.522846 32293 leader_election.cc:290] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055)
I20260709 13:11:08.522878 32303 consensus_queue.cc:1048] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.525606 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:08.526136 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:08.526510 32072 raft_consensus.cc:2468] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:08.527153 32071 raft_consensus.cc:3060] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.528615 32297 raft_consensus.cc:515] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.532371 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:08.533103 32148 raft_consensus.cc:3060] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.533689 32267 leader_election.cc:290] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.532392 32297 leader_election.cc:290] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055)
I20260709 13:11:08.542847 32220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:08.543730 32220 raft_consensus.cc:3060] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.544063 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:08.544763 32073 raft_consensus.cc:3060] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.546950 32148 raft_consensus.cc:2468] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:08.548749 32187 leader_election.cc:304] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:08.549938 32297 raft_consensus.cc:2804] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:08.550493 32297 raft_consensus.cc:697] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:08.551805 32297 consensus_queue.cc:237] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.555220 32073 raft_consensus.cc:2468] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:08.555959 32292 consensus_queue.cc:1048] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.570091 32220 raft_consensus.cc:2468] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:08.571872 32113 leader_election.cc:304] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:08.572799 32267 raft_consensus.cc:2804] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:08.573333 32267 raft_consensus.cc:697] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:08.574380 32267 consensus_queue.cc:237] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:08.575639 32303 consensus_queue.cc:1048] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:08.579643 32071 raft_consensus.cc:2468] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:08.587999 32291 consensus_queue.cc:1048] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.613932 32293 consensus_queue.cc:1048] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.616353 31969 catalog_manager.cc:5697] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 1 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:08.625870 32114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:43055, user_credentials={real_user=slave}} blocked reactor thread for 50977.8us
I20260709 13:11:08.642664 32291 consensus_queue.cc:1048] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:11:08.668114 32338 consensus_queue.cc:1048] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.672161 32293 consensus_queue.cc:1048] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:08.692849 31971 catalog_manager.cc:5697] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.695716 32297 consensus_queue.cc:1048] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.711653 32296 consensus_queue.cc:1048] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.772812 32303 consensus_queue.cc:1048] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:08.784999 32330 consensus_queue.cc:1048] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.886402 32219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
mode: GRACEFUL
new_leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:40766
I20260709 13:11:08.887005 32219 raft_consensus.cc:606] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Received request to transfer leadership to TS ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:08.903560 32148 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dda35f1227549ff933e504aadd37233"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
mode: GRACEFUL
new_leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:50352
I20260709 13:11:08.904060 32148 raft_consensus.cc:606] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Received request to transfer leadership to TS ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:08.909708 32219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
mode: GRACEFUL
new_leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:40766
I20260709 13:11:08.910231 32219 raft_consensus.cc:606] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Received request to transfer leadership to TS ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:08.921140 32219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
mode: GRACEFUL
new_leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:40766
I20260709 13:11:08.921639 32219 raft_consensus.cc:606] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Received request to transfer leadership to TS 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:08.935827 32072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e05e529aa954954acd28cf6021b0d35"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
mode: GRACEFUL
new_leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:60832
I20260709 13:11:08.936486 32072 raft_consensus.cc:606] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Received request to transfer leadership to TS 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:08.936764 32297 consensus_queue.cc:1048] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:08.943634 32146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
mode: GRACEFUL
new_leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:50352
I20260709 13:11:08.944350 32146 raft_consensus.cc:606] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Received request to transfer leadership to TS 1b650608729b4ad6a28fd1c7e3d30d1c
I20260709 13:11:08.950929 32297 consensus_queue.cc:1048] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.959694 32338 raft_consensus.cc:993] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c: : Instructing follower ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:08.960273 32297 raft_consensus.cc:1081] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Signalling peer ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:08.961923 32072 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:60826
I20260709 13:11:08.962604 32072 raft_consensus.cc:493] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:08.962998 32072 raft_consensus.cc:3060] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:08.968668 32072 raft_consensus.cc:515] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.970613 32072 leader_election.cc:290] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:08.971611 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:08.971825 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2b786f061514123aca8f3d1750e39d9" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:08.972321 32146 raft_consensus.cc:3060] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:08.972551 32219 raft_consensus.cc:3055] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:08.972935 32219 raft_consensus.cc:740] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:08.973696 32219 consensus_queue.cc:260] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.974725 32219 raft_consensus.cc:3060] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:08.980854 32146 raft_consensus.cc:2468] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:08.981194 32219 raft_consensus.cc:2468] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:08.982331 32038 leader_election.cc:304] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:08.983179 32296 raft_consensus.cc:2804] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:08.983742 32296 raft_consensus.cc:697] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:08.984541 32296 consensus_queue.cc:237] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } }
I20260709 13:11:08.991794 31969 catalog_manager.cc:5697] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 1 to 2, leader changed from 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195) to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 2 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:08.993969 32330 consensus_queue.cc:1048] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:09.003842 32303 raft_consensus.cc:993] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227: : Instructing follower 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:09.004400 32292 raft_consensus.cc:1081] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Signalling peer 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:09.006659 32222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:40734
I20260709 13:11:09.007509 32222 raft_consensus.cc:493] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:09.007905 32222 raft_consensus.cc:3060] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.009694 32292 consensus_queue.cc:1048] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:09.017558 32222 raft_consensus.cc:515] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.019729 32219 raft_consensus.cc:1240] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Rejecting Update request from peer 3bc3d543183d4ff5a6635ffb2e110227 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:09.021725 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:09.021816 32292 consensus_queue.cc:1059] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:09.022166 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27ad9f1bb0349b4bb39afe329cf59d8" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:09.022563 32072 raft_consensus.cc:3060] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.022943 32146 raft_consensus.cc:3055] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:09.023427 32146 raft_consensus.cc:740] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:09.024309 32146 consensus_queue.cc:260] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.025449 32146 raft_consensus.cc:3060] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.030050 32072 raft_consensus.cc:2468] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 2.
I20260709 13:11:09.030663 32222 leader_election.cc:290] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 2 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:09.031551 32146 raft_consensus.cc:2468] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 2.
I20260709 13:11:09.031735 32188 leader_election.cc:304] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
W20260709 13:11:09.032243 32303 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:11:09.033849 32297 raft_consensus.cc:2804] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:09.034373 32297 raft_consensus.cc:697] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:09.035434 32297 consensus_queue.cc:237] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.043208 31969 catalog_manager.cc:5697] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 1 to 2, leader changed from 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194) to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 2 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:09.088357 32297 raft_consensus.cc:993] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c: : Instructing follower 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:09.089082 32336 raft_consensus.cc:1081] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Signalling peer 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:09.090770 32146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:50336
I20260709 13:11:09.091329 32146 raft_consensus.cc:493] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:09.091737 32146 raft_consensus.cc:3060] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.097030 32146 raft_consensus.cc:515] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.099339 32146 leader_election.cc:290] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:09.099826 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:09.100008 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:09.100612 32072 raft_consensus.cc:3060] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.100814 32222 raft_consensus.cc:3055] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:09.101323 32222 raft_consensus.cc:740] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:09.102233 32222 consensus_queue.cc:260] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.103566 32222 raft_consensus.cc:3060] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.107053 32072 raft_consensus.cc:2468] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 2.
I20260709 13:11:09.108433 32114 leader_election.cc:304] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:09.109407 32292 raft_consensus.cc:2804] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:09.109958 32292 raft_consensus.cc:697] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:09.110271 32222 raft_consensus.cc:2468] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 2.
I20260709 13:11:09.111060 32292 consensus_queue.cc:237] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.117626 31971 catalog_manager.cc:5697] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 1 to 2, leader changed from 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195) to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:09.130955 32332 raft_consensus.cc:993] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b: : Instructing follower 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:09.131536 32296 raft_consensus.cc:1081] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Signalling peer 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:09.133071 32146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e05e529aa954954acd28cf6021b0d35"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:50326
I20260709 13:11:09.133591 32146 raft_consensus.cc:493] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:09.133942 32146 raft_consensus.cc:3060] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.139154 32146 raft_consensus.cc:515] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.140893 32146 leader_election.cc:290] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:09.141772 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e05e529aa954954acd28cf6021b0d35" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:09.142170 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e05e529aa954954acd28cf6021b0d35" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:09.142518 32072 raft_consensus.cc:3055] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:09.142844 32222 raft_consensus.cc:3060] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.142874 32072 raft_consensus.cc:740] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:09.143916 32072 consensus_queue.cc:260] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.145265 32072 raft_consensus.cc:3060] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.150589 32222 raft_consensus.cc:2468] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 2.
I20260709 13:11:09.152070 32113 leader_election.cc:304] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:09.152273 32072 raft_consensus.cc:2468] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 2.
I20260709 13:11:09.152804 32292 raft_consensus.cc:2804] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:09.153312 32292 raft_consensus.cc:697] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:09.154248 32292 consensus_queue.cc:237] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.160449 31971 catalog_manager.cc:5697] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 1 to 2, leader changed from ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193) to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:09.211153 32297 raft_consensus.cc:993] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c: : Instructing follower ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:09.211755 32336 raft_consensus.cc:1081] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Signalling peer ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:09.213280 32072 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:60826
I20260709 13:11:09.213788 32072 raft_consensus.cc:493] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:09.214105 32072 raft_consensus.cc:3060] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.219177 32072 raft_consensus.cc:515] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.221130 32072 leader_election.cc:290] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:09.221951 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:09.222298 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:09.222623 32146 raft_consensus.cc:3060] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.222949 32222 raft_consensus.cc:3055] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:09.223263 32222 raft_consensus.cc:740] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:09.224040 32222 consensus_queue.cc:260] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.225008 32222 raft_consensus.cc:3060] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.228649 32146 raft_consensus.cc:2468] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:09.230021 32038 leader_election.cc:304] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:09.230741 32296 raft_consensus.cc:2804] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:09.231241 32296 raft_consensus.cc:697] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:09.231765 32222 raft_consensus.cc:2468] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:09.232201 32296 consensus_queue.cc:237] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.239187 31971 catalog_manager.cc:5697] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 1 to 2, leader changed from 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195) to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 2 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:09.247094 32292 raft_consensus.cc:993] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227: : Instructing follower ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:09.247759 32330 raft_consensus.cc:1081] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Signalling peer ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:09.249420 32072 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3dda35f1227549ff933e504aadd37233"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:60820
I20260709 13:11:09.250028 32072 raft_consensus.cc:493] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:09.250365 32072 raft_consensus.cc:3060] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.256197 32072 raft_consensus.cc:515] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.258011 32072 leader_election.cc:290] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:09.258963 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dda35f1227549ff933e504aadd37233" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:09.259400 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dda35f1227549ff933e504aadd37233" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:09.259649 32146 raft_consensus.cc:3055] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:09.260041 32222 raft_consensus.cc:3060] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.260066 32146 raft_consensus.cc:740] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:09.261054 32146 consensus_queue.cc:260] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.262041 32146 raft_consensus.cc:3060] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.265909 32222 raft_consensus.cc:2468] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:09.267203 32038 leader_election.cc:304] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:09.267335 32146 raft_consensus.cc:2468] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:09.268162 32296 raft_consensus.cc:2804] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:09.268676 32296 raft_consensus.cc:697] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:09.269588 32296 consensus_queue.cc:237] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:09.276629 31969 catalog_manager.cc:5697] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 1 to 2, leader changed from 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194) to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 2 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:09.457265 32146 raft_consensus.cc:1275] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.458624 32296 consensus_queue.cc:1048] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.466598 32222 raft_consensus.cc:1275] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.468219 32291 consensus_queue.cc:1048] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.522282 32072 raft_consensus.cc:1275] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.523754 32330 consensus_queue.cc:1048] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.534442 32222 raft_consensus.cc:1275] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.536121 32330 consensus_queue.cc:1048] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:09.591199 32146 raft_consensus.cc:1275] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.592964 32336 consensus_queue.cc:1048] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.596977 32072 raft_consensus.cc:1275] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.598553 32336 consensus_queue.cc:1048] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:09.608120 32222 raft_consensus.cc:1275] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.609968 32330 consensus_queue.cc:1048] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.623597 32072 raft_consensus.cc:1275] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.625067 32292 consensus_queue.cc:1048] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.651924 32222 raft_consensus.cc:1275] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.653374 32291 consensus_queue.cc:1048] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.661954 32146 raft_consensus.cc:1275] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.663515 32296 consensus_queue.cc:1048] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.705090 32146 raft_consensus.cc:1275] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.706485 32291 consensus_queue.cc:1048] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.715809 32222 raft_consensus.cc:1275] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:09.718144 32291 consensus_queue.cc:1048] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.973733 31970 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58186:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:11.976548 31970 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:12.031469 32060 tablet_service.cc:1511] Processing CreateTablet for tablet e291faba389e4a21b5bf01b3a0de914d (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:12.032462 32138 tablet_service.cc:1511] Processing CreateTablet for tablet e291faba389e4a21b5bf01b3a0de914d (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:12.033279 32060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e291faba389e4a21b5bf01b3a0de914d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.034233 32138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e291faba389e4a21b5bf01b3a0de914d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.036293 32208 tablet_service.cc:1511] Processing CreateTablet for tablet e291faba389e4a21b5bf01b3a0de914d (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:12.037196 32055 tablet_service.cc:1511] Processing CreateTablet for tablet 981b55af3f1041619f509a52299dab01 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:12.038152 32208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e291faba389e4a21b5bf01b3a0de914d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.038734 32055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981b55af3f1041619f509a52299dab01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.040760 32212 tablet_service.cc:1511] Processing CreateTablet for tablet 981b55af3f1041619f509a52299dab01 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:12.044505 32134 tablet_service.cc:1511] Processing CreateTablet for tablet 4e06665820de481db4d97fa058f082f3 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:12.040930 32137 tablet_service.cc:1511] Processing CreateTablet for tablet 981b55af3f1041619f509a52299dab01 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:12.046981 32058 tablet_service.cc:1511] Processing CreateTablet for tablet 4e06665820de481db4d97fa058f082f3 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:12.047606 32205 tablet_service.cc:1511] Processing CreateTablet for tablet 4e06665820de481db4d97fa058f082f3 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:12.048233 32134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e06665820de481db4d97fa058f082f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.048316 32058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e06665820de481db4d97fa058f082f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.045586 32212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981b55af3f1041619f509a52299dab01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.050257 32137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981b55af3f1041619f509a52299dab01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.050751 32205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e06665820de481db4d97fa058f082f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.072698 32056 tablet_service.cc:1511] Processing CreateTablet for tablet 26975e63d5a54a1399df116dc00be42a (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:12.074520 32056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26975e63d5a54a1399df116dc00be42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.075047 32131 tablet_service.cc:1511] Processing CreateTablet for tablet 26975e63d5a54a1399df116dc00be42a (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:12.076725 32131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26975e63d5a54a1399df116dc00be42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.082314 32210 tablet_service.cc:1511] Processing CreateTablet for tablet 26975e63d5a54a1399df116dc00be42a (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:12.084018 32210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26975e63d5a54a1399df116dc00be42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.088752 32132 tablet_service.cc:1511] Processing CreateTablet for tablet 2150aff863544382a1801920674d76be (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:12.090380 32132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2150aff863544382a1801920674d76be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.094036 32063 tablet_service.cc:1511] Processing CreateTablet for tablet 2150aff863544382a1801920674d76be (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:12.095574 32063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2150aff863544382a1801920674d76be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.108846 32371 tablet_bootstrap.cc:492] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:12.113046 32209 tablet_service.cc:1511] Processing CreateTablet for tablet 2150aff863544382a1801920674d76be (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:12.116262 32135 tablet_service.cc:1511] Processing CreateTablet for tablet 54a4bfa356274b21a59d280e9ac8b477 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:12.118184 32135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a4bfa356274b21a59d280e9ac8b477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.136616 32059 tablet_service.cc:1511] Processing CreateTablet for tablet c8ca9fffb23a4ec5820b34d42497b5dc (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:12.138041 32059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ca9fffb23a4ec5820b34d42497b5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.134897 32057 tablet_service.cc:1511] Processing CreateTablet for tablet ea3f5770f9654ebcb524ef1028166cbc (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:12.141847 32057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3f5770f9654ebcb524ef1028166cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.125262 32206 tablet_service.cc:1511] Processing CreateTablet for tablet 54a4bfa356274b21a59d280e9ac8b477 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:12.125974 32204 tablet_service.cc:1511] Processing CreateTablet for tablet ea3f5770f9654ebcb524ef1028166cbc (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:12.131476 32138 tablet_service.cc:1511] Processing CreateTablet for tablet c8ca9fffb23a4ec5820b34d42497b5dc (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:12.133013 32062 tablet_service.cc:1511] Processing CreateTablet for tablet 54a4bfa356274b21a59d280e9ac8b477 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:12.142743 32209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2150aff863544382a1801920674d76be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.144649 32138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ca9fffb23a4ec5820b34d42497b5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.146847 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a4bfa356274b21a59d280e9ac8b477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.148571 32133 tablet_service.cc:1511] Processing CreateTablet for tablet 0ebb8d66845146b1b2d45c442e803bd1 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:12.149992 32133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebb8d66845146b1b2d45c442e803bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.153115 32207 tablet_service.cc:1511] Processing CreateTablet for tablet 0ebb8d66845146b1b2d45c442e803bd1 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:12.173617 32371 tablet_bootstrap.cc:654] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.174790 32056 tablet_service.cc:1511] Processing CreateTablet for tablet 0ebb8d66845146b1b2d45c442e803bd1 (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:12.176262 32056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebb8d66845146b1b2d45c442e803bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.182039 32372 tablet_bootstrap.cc:492] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:12.188040 32137 tablet_service.cc:1511] Processing CreateTablet for tablet ea3f5770f9654ebcb524ef1028166cbc (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:12.189595 32137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3f5770f9654ebcb524ef1028166cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.191182 32207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebb8d66845146b1b2d45c442e803bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.196090 32206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a4bfa356274b21a59d280e9ac8b477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.200582 32204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3f5770f9654ebcb524ef1028166cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.261722 32372 tablet_bootstrap.cc:654] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.266500 32205 tablet_service.cc:1511] Processing CreateTablet for tablet c8ca9fffb23a4ec5820b34d42497b5dc (DEFAULT_TABLE table=soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:12.268096 32205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ca9fffb23a4ec5820b34d42497b5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:12.282897 32371 tablet_bootstrap.cc:492] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:12.283639 32371 ts_tablet_manager.cc:1403] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.175s	user 0.002s	sys 0.014s
I20260709 13:11:12.287111 32371 raft_consensus.cc:359] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.287889 32371 raft_consensus.cc:385] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.288195 32371 raft_consensus.cc:740] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.288897 32371 consensus_queue.cc:260] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.297962 32371 ts_tablet_manager.cc:1434] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.008s
I20260709 13:11:12.301357 32330 raft_consensus.cc:493] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.303505 32330 raft_consensus.cc:515] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.306056 32330 leader_election.cc:290] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.308389 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
W20260709 13:11:12.310578 32114 leader_election.cc:343] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:12.331779 32371 tablet_bootstrap.cc:492] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:12.347699 32376 tablet_bootstrap.cc:492] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
W20260709 13:11:12.347681 32113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.195:43043, user_credentials={real_user=slave}} blocked reactor thread for 47839.2us
I20260709 13:11:12.349664 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:12.352288 32372 tablet_bootstrap.cc:492] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:12.353770 32372 ts_tablet_manager.cc:1403] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.172s	user 0.023s	sys 0.039s
W20260709 13:11:12.354218 32113 leader_election.cc:343] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:12.354449 32371 tablet_bootstrap.cc:654] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.354949 32113 leader_election.cc:304] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:12.357937 32372 raft_consensus.cc:359] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.358956 32372 raft_consensus.cc:385] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.359364 32372 raft_consensus.cc:740] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.359421 32330 raft_consensus.cc:2749] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:12.360404 32376 tablet_bootstrap.cc:654] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.361542 32372 consensus_queue.cc:260] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.364508 32372 ts_tablet_manager.cc:1434] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:11:12.365624 32372 tablet_bootstrap.cc:492] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:12.366946 32371 tablet_bootstrap.cc:492] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:12.367755 32371 ts_tablet_manager.cc:1403] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.008s
I20260709 13:11:12.371165 32371 raft_consensus.cc:359] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.371963 32371 raft_consensus.cc:385] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.372251 32371 raft_consensus.cc:740] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.372534 32372 tablet_bootstrap.cc:654] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.373289 32376 tablet_bootstrap.cc:492] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:12.373014 32371 consensus_queue.cc:260] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.375715 32371 ts_tablet_manager.cc:1434] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:11:12.376758 32371 tablet_bootstrap.cc:492] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:12.383273 32371 tablet_bootstrap.cc:654] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.384557 32372 tablet_bootstrap.cc:492] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:12.385121 32372 ts_tablet_manager.cc:1403] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:11:12.388645 32372 raft_consensus.cc:359] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.389633 32372 raft_consensus.cc:385] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.389945 32372 raft_consensus.cc:740] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.390677 32372 consensus_queue.cc:260] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.392292 32376 ts_tablet_manager.cc:1403] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.000s
I20260709 13:11:12.393189 32372 ts_tablet_manager.cc:1434] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:11:12.394266 32372 tablet_bootstrap.cc:492] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:12.397169 32376 raft_consensus.cc:359] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.398242 32376 raft_consensus.cc:385] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.398670 32376 raft_consensus.cc:740] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.399914 32376 consensus_queue.cc:260] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.401163 32372 tablet_bootstrap.cc:654] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.403400 32376 ts_tablet_manager.cc:1434] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:11:12.405853 32376 tablet_bootstrap.cc:492] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:12.412429 32371 tablet_bootstrap.cc:492] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:12.413014 32371 ts_tablet_manager.cc:1403] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.037s	user 0.023s	sys 0.008s
I20260709 13:11:12.415946 32372 tablet_bootstrap.cc:492] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:12.415822 32371 raft_consensus.cc:359] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.416551 32376 tablet_bootstrap.cc:654] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.416617 32372 ts_tablet_manager.cc:1403] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.008s
I20260709 13:11:12.416626 32371 raft_consensus.cc:385] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.417358 32371 raft_consensus.cc:740] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.418149 32371 consensus_queue.cc:260] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.420560 32371 ts_tablet_manager.cc:1434] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:12.420241 32372 raft_consensus.cc:359] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.421391 32372 raft_consensus.cc:385] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.421627 32371 tablet_bootstrap.cc:492] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:12.421819 32372 raft_consensus.cc:740] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.423136 32372 consensus_queue.cc:260] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.426254 32372 ts_tablet_manager.cc:1434] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:11:12.427686 32372 tablet_bootstrap.cc:492] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:12.429286 32371 tablet_bootstrap.cc:654] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.437435 32372 tablet_bootstrap.cc:654] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.441672 32376 tablet_bootstrap.cc:492] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:12.442374 32376 ts_tablet_manager.cc:1403] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.037s	user 0.022s	sys 0.013s
I20260709 13:11:12.453048 32376 raft_consensus.cc:359] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.454221 32376 raft_consensus.cc:385] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.454598 32371 tablet_bootstrap.cc:492] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:12.454838 32376 raft_consensus.cc:740] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.455159 32371 ts_tablet_manager.cc:1403] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.011s
I20260709 13:11:12.456511 32376 consensus_queue.cc:260] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.457973 32371 raft_consensus.cc:359] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.458715 32371 raft_consensus.cc:385] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.459030 32371 raft_consensus.cc:740] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.459935 32371 consensus_queue.cc:260] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.462152 32376 ts_tablet_manager.cc:1434] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.000s
I20260709 13:11:12.462270 32371 ts_tablet_manager.cc:1434] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:11:12.463266 32372 tablet_bootstrap.cc:492] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:12.463388 32376 tablet_bootstrap.cc:492] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:12.463639 32371 tablet_bootstrap.cc:492] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:12.464037 32372 ts_tablet_manager.cc:1403] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.014s
I20260709 13:11:12.470435 32372 raft_consensus.cc:359] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.471211 32372 raft_consensus.cc:385] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.471261 32371 tablet_bootstrap.cc:654] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.471465 32376 tablet_bootstrap.cc:654] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.471761 32372 raft_consensus.cc:740] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.472839 32372 consensus_queue.cc:260] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.475555 32372 ts_tablet_manager.cc:1434] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:11:12.477191 32372 tablet_bootstrap.cc:492] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:12.483309 32371 tablet_bootstrap.cc:492] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:12.483949 32376 tablet_bootstrap.cc:492] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:12.483942 32371 ts_tablet_manager.cc:1403] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.007s
I20260709 13:11:12.484854 32376 ts_tablet_manager.cc:1403] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.015s
I20260709 13:11:12.501875 32376 raft_consensus.cc:359] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.502524 32376 raft_consensus.cc:385] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.502782 32376 raft_consensus.cc:740] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.485270 32372 tablet_bootstrap.cc:654] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.487679 32371 raft_consensus.cc:359] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.503618 32371 raft_consensus.cc:385] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.503605 32376 consensus_queue.cc:260] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.504205 32371 raft_consensus.cc:740] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.495653 32359 raft_consensus.cc:493] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.504505 32330 raft_consensus.cc:493] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.504930 32359 raft_consensus.cc:515] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.505108 32330 raft_consensus.cc:515] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.505252 32371 consensus_queue.cc:260] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.506392 32376 ts_tablet_manager.cc:1434] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:12.507472 32376 tablet_bootstrap.cc:492] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:12.508327 32371 ts_tablet_manager.cc:1434] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.024s	user 0.012s	sys 0.005s
I20260709 13:11:12.508544 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e06665820de481db4d97fa058f082f3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:12.509240 32330 leader_election.cc:290] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.509346 32148 raft_consensus.cc:2468] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:12.510115 32371 tablet_bootstrap.cc:492] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:12.508605 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981b55af3f1041619f509a52299dab01" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
W20260709 13:11:12.512177 32114 leader_election.cc:343] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:12.514822 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981b55af3f1041619f509a52299dab01" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:12.515128 32376 tablet_bootstrap.cc:654] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.515520 32222 raft_consensus.cc:2468] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:12.517048 32187 leader_election.cc:304] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:12.519536 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e06665820de481db4d97fa058f082f3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:12.520136 32371 tablet_bootstrap.cc:654] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.520563 32071 raft_consensus.cc:2468] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:12.522610 32372 tablet_bootstrap.cc:492] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:12.522504 32113 leader_election.cc:304] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:12.523504 32372 ts_tablet_manager.cc:1403] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.047s	user 0.030s	sys 0.004s
I20260709 13:11:12.524470 32330 raft_consensus.cc:2804] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:12.525137 32330 raft_consensus.cc:493] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.525663 32330 raft_consensus.cc:3060] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.527839 32372 raft_consensus.cc:359] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.528960 32372 raft_consensus.cc:385] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.529340 32372 raft_consensus.cc:740] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.530514 32359 leader_election.cc:290] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:12.540481 32372 consensus_queue.cc:260] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.544808 32359 raft_consensus.cc:493] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.545617 32359 raft_consensus.cc:515] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.547439 32372 ts_tablet_manager.cc:1434] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 13:11:12.548625 32372 tablet_bootstrap.cc:492] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:12.549001 32359 leader_election.cc:290] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:12.553516 32376 tablet_bootstrap.cc:492] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:12.554347 32376 ts_tablet_manager.cc:1403] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.016s
I20260709 13:11:12.557269 32330 raft_consensus.cc:515] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.558935 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2150aff863544382a1801920674d76be" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:12.538718 32390 raft_consensus.cc:2804] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:12.561007 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981b55af3f1041619f509a52299dab01" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:12.564942 32219 raft_consensus.cc:3060] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.565640 32376 raft_consensus.cc:359] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.565909 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2150aff863544382a1801920674d76be" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:12.566393 32376 raft_consensus.cc:385] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.566642 32376 raft_consensus.cc:740] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.567065 32393 raft_consensus.cc:493] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:12.567512 32187 leader_election.cc:343] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:12.567358 32376 consensus_queue.cc:260] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.567806 32393 raft_consensus.cc:515] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.568481 32072 raft_consensus.cc:2468] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:12.571208 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:12.570036 32376 ts_tablet_manager.cc:1434] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
W20260709 13:11:12.572943 32113 leader_election.cc:343] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:12.573454 32376 tablet_bootstrap.cc:492] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:12.573542 32393 leader_election.cc:290] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.576921 32188 leader_election.cc:304] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:12.578181 32394 raft_consensus.cc:2804] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:12.578572 32394 raft_consensus.cc:493] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.578881 32394 raft_consensus.cc:3060] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.580906 32376 tablet_bootstrap.cc:654] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.583606 32390 raft_consensus.cc:493] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.584046 32390 raft_consensus.cc:3060] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.584262 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981b55af3f1041619f509a52299dab01" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:12.561174 32330 leader_election.cc:290] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.562999 32372 tablet_bootstrap.cc:654] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.585760 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:12.588207 32219 raft_consensus.cc:2468] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
W20260709 13:11:12.588308 32114 leader_election.cc:343] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:12.589704 32113 leader_election.cc:304] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: ee90f5b2a1e849739b82a66fdce52a8b
W20260709 13:11:12.589814 32114 leader_election.cc:343] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:12.590495 32114 leader_election.cc:304] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:12.590909 32330 raft_consensus.cc:2804] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:12.563273 32371 tablet_bootstrap.cc:492] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:12.591583 32393 raft_consensus.cc:2749] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:12.591908 32371 ts_tablet_manager.cc:1403] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.082s	user 0.018s	sys 0.027s
I20260709 13:11:12.592306 32376 tablet_bootstrap.cc:492] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:12.592864 32376 ts_tablet_manager.cc:1403] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.003s
I20260709 13:11:12.592494 32390 raft_consensus.cc:515] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.595574 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e06665820de481db4d97fa058f082f3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:12.596302 32071 raft_consensus.cc:3060] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.596094 32376 raft_consensus.cc:359] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.596925 32376 raft_consensus.cc:385] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.597229 32376 raft_consensus.cc:740] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.596108 32330 raft_consensus.cc:697] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:12.602068 32330 consensus_queue.cc:237] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.603981 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e06665820de481db4d97fa058f082f3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:12.604671 32148 raft_consensus.cc:3060] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.596298 32394 raft_consensus.cc:515] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.604631 32376 consensus_queue.cc:260] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.607992 32376 ts_tablet_manager.cc:1434] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:11:12.605014 32371 raft_consensus.cc:359] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.608968 32371 raft_consensus.cc:385] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.609097 32376 tablet_bootstrap.cc:492] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:12.609299 32371 raft_consensus.cc:740] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.610177 32371 consensus_queue.cc:260] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.612565 32371 ts_tablet_manager.cc:1434] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:11:12.613668 32371 tablet_bootstrap.cc:492] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:12.615525 32148 raft_consensus.cc:2468] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:12.616835 32376 tablet_bootstrap.cc:654] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.617693 32394 leader_election.cc:290] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:12.621546 32372 tablet_bootstrap.cc:492] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:12.622162 32372 ts_tablet_manager.cc:1403] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.074s	user 0.016s	sys 0.005s
I20260709 13:11:12.622956 32371 tablet_bootstrap.cc:654] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.619088 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2150aff863544382a1801920674d76be" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:12.619607 32393 raft_consensus.cc:493] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.624727 32146 raft_consensus.cc:3060] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.624826 32393 raft_consensus.cc:515] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.625523 32372 raft_consensus.cc:359] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.626408 32372 raft_consensus.cc:385] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.626704 32372 raft_consensus.cc:740] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.620059 32071 raft_consensus.cc:2468] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:12.627521 32372 consensus_queue.cc:260] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.620317 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2150aff863544382a1801920674d76be" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:12.629024 32072 raft_consensus.cc:3060] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.629961 32372 ts_tablet_manager.cc:1434] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:12.631126 32372 tablet_bootstrap.cc:492] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:12.634168 32146 raft_consensus.cc:2468] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:12.620841 32390 leader_election.cc:290] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:12.639878 32188 leader_election.cc:304] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:12.641294 32393 leader_election.cc:290] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.641727 32187 leader_election.cc:304] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:12.642354 32390 raft_consensus.cc:2804] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:12.648130 32394 raft_consensus.cc:2804] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:12.646870 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26975e63d5a54a1399df116dc00be42a" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:12.649879 32219 raft_consensus.cc:2468] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:12.651491 32113 leader_election.cc:304] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:12.651933 32376 tablet_bootstrap.cc:492] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:12.652081 32072 raft_consensus.cc:2468] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:12.652527 32376 ts_tablet_manager.cc:1403] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.044s	user 0.005s	sys 0.010s
I20260709 13:11:12.657956 32393 raft_consensus.cc:2804] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:12.658466 32393 raft_consensus.cc:493] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.658811 32393 raft_consensus.cc:3060] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.648767 32390 raft_consensus.cc:697] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:12.662477 32376 raft_consensus.cc:359] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.662716 32390 consensus_queue.cc:237] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.663319 32376 raft_consensus.cc:385] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.663650 32376 raft_consensus.cc:740] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.664533 32376 consensus_queue.cc:260] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.662091 32394 raft_consensus.cc:697] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:12.667630 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26975e63d5a54a1399df116dc00be42a" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:12.668440 32072 raft_consensus.cc:2468] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:12.669463 32400 raft_consensus.cc:493] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.669952 32400 raft_consensus.cc:515] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.670933 32394 consensus_queue.cc:237] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.676115 31971 catalog_manager.cc:5697] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:12.674226 32372 tablet_bootstrap.cc:654] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.682063 32376 ts_tablet_manager.cc:1434] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:11:12.683264 32371 tablet_bootstrap.cc:492] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:12.683343 32400 leader_election.cc:290] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:12.683355 32376 tablet_bootstrap.cc:492] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:12.684192 32371 ts_tablet_manager.cc:1403] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.071s	user 0.023s	sys 0.028s
I20260709 13:11:12.689921 32371 raft_consensus.cc:359] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.690881 32371 raft_consensus.cc:385] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.691272 32371 raft_consensus.cc:740] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.684558 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:12.692910 32072 raft_consensus.cc:2468] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:12.685554 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:12.695459 32146 raft_consensus.cc:2468] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:12.696794 32187 leader_election.cc:304] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:12.697713 32376 tablet_bootstrap.cc:654] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.697916 32390 raft_consensus.cc:2804] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:12.698316 32390 raft_consensus.cc:493] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.698658 32390 raft_consensus.cc:3060] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.694264 32393 raft_consensus.cc:515] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.704566 32371 consensus_queue.cc:260] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.705170 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26975e63d5a54a1399df116dc00be42a" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:12.706102 32219 raft_consensus.cc:3060] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.707224 32371 ts_tablet_manager.cc:1434] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:11:12.708395 32371 tablet_bootstrap.cc:492] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:12.711899 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26975e63d5a54a1399df116dc00be42a" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:12.712910 32072 raft_consensus.cc:3060] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.711979 32393 leader_election.cc:290] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.723822 32376 tablet_bootstrap.cc:492] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:12.724074 32219 raft_consensus.cc:2468] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:12.724500 32376 ts_tablet_manager.cc:1403] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.018s
I20260709 13:11:12.725634 32113 leader_election.cc:304] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:12.726009 32371 tablet_bootstrap.cc:654] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.726585 32393 raft_consensus.cc:2804] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:12.727240 32393 raft_consensus.cc:697] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:12.727651 32072 raft_consensus.cc:2468] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:12.727989 32376 raft_consensus.cc:359] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.728157 32393 consensus_queue.cc:237] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.728852 32376 raft_consensus.cc:385] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.729215 32376 raft_consensus.cc:740] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.726683 31967 catalog_manager.cc:5697] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 1 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.730376 32376 consensus_queue.cc:260] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.730623 31967 catalog_manager.cc:5697] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 1 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.733867 32376 ts_tablet_manager.cc:1434] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:11:12.733721 32390 raft_consensus.cc:515] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.735431 32376 tablet_bootstrap.cc:492] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:12.739321 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:12.740212 32072 raft_consensus.cc:3060] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.743848 32390 leader_election.cc:290] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:12.745477 32376 tablet_bootstrap.cc:654] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.745442 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:12.746356 32146 raft_consensus.cc:3060] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.751605 32372 tablet_bootstrap.cc:492] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:12.752229 32372 ts_tablet_manager.cc:1403] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.121s	user 0.013s	sys 0.003s
I20260709 13:11:12.764428 32072 raft_consensus.cc:2468] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:12.766654 32188 leader_election.cc:304] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:12.767105 32372 raft_consensus.cc:359] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.767640 32390 raft_consensus.cc:2804] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:12.768355 32376 tablet_bootstrap.cc:492] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:12.767982 32372 raft_consensus.cc:385] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.771349 32390 raft_consensus.cc:697] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:12.771713 32372 raft_consensus.cc:740] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.771831 32376 ts_tablet_manager.cc:1403] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.037s	user 0.022s	sys 0.003s
I20260709 13:11:12.772576 32372 consensus_queue.cc:260] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.772475 32390 consensus_queue.cc:237] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.774984 32371 tablet_bootstrap.cc:492] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:12.775055 32372 ts_tablet_manager.cc:1434] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:11:12.774930 32376 raft_consensus.cc:359] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.775945 32371 ts_tablet_manager.cc:1403] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.068s	user 0.020s	sys 0.030s
I20260709 13:11:12.775952 32376 raft_consensus.cc:385] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.776525 32376 raft_consensus.cc:740] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.776659 32372 tablet_bootstrap.cc:492] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:12.777304 32376 consensus_queue.cc:260] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.779753 32376 ts_tablet_manager.cc:1434] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:12.779422 32371 raft_consensus.cc:359] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.780279 32371 raft_consensus.cc:385] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.780611 32371 raft_consensus.cc:740] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.781252 32376 tablet_bootstrap.cc:492] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b: Bootstrap starting.
I20260709 13:11:12.781418 32371 consensus_queue.cc:260] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.762507 32146 raft_consensus.cc:2468] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:12.783937 32371 ts_tablet_manager.cc:1434] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:12.784161 32372 tablet_bootstrap.cc:654] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.785169 32371 tablet_bootstrap.cc:492] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227: Bootstrap starting.
I20260709 13:11:12.787966 32376 tablet_bootstrap.cc:654] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.798787 32372 tablet_bootstrap.cc:492] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:12.799716 32372 ts_tablet_manager.cc:1403] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.003s
I20260709 13:11:12.803056 32371 tablet_bootstrap.cc:654] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.807729 32373 raft_consensus.cc:493] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.808279 32373 raft_consensus.cc:515] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.810384 32373 leader_election.cc:290] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.814041 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
W20260709 13:11:12.815572 32038 leader_election.cc:343] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:12.817984 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:12.818707 32146 raft_consensus.cc:2468] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 0.
I20260709 13:11:12.819979 32038 leader_election.cc:304] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c
I20260709 13:11:12.821135 32373 raft_consensus.cc:2804] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:12.821506 32373 raft_consensus.cc:493] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.822080 32373 raft_consensus.cc:3060] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.824400 32372 raft_consensus.cc:359] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.825176 32372 raft_consensus.cc:385] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.825446 32372 raft_consensus.cc:740] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.826131 32372 consensus_queue.cc:260] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.828234 32372 ts_tablet_manager.cc:1434] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.001s
I20260709 13:11:12.829211 32372 tablet_bootstrap.cc:492] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c: Bootstrap starting.
I20260709 13:11:12.830713 32373 raft_consensus.cc:515] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.833880 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:12.834486 32146 raft_consensus.cc:3060] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.835203 32372 tablet_bootstrap.cc:654] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:12.843808 32376 tablet_bootstrap.cc:492] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b: No bootstrap required, opened a new log
I20260709 13:11:12.844430 32376 ts_tablet_manager.cc:1403] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.020s
I20260709 13:11:12.848309 32372 tablet_bootstrap.cc:492] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c: No bootstrap required, opened a new log
I20260709 13:11:12.847740 32376 raft_consensus.cc:359] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.848935 32372 ts_tablet_manager.cc:1403] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.007s
I20260709 13:11:12.849186 32376 raft_consensus.cc:385] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.849881 32376 raft_consensus.cc:740] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.850217 31971 catalog_manager.cc:5697] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:12.850939 32376 consensus_queue.cc:260] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.852953 32372 raft_consensus.cc:359] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.868155 32372 raft_consensus.cc:385] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.868530 32372 raft_consensus.cc:740] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.869303 32372 consensus_queue.cc:260] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.870949 32372 ts_tablet_manager.cc:1434] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c: Time spent starting tablet: real 0.022s	user 0.013s	sys 0.004s
I20260709 13:11:12.872567 32376 ts_tablet_manager.cc:1434] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b: Time spent starting tablet: real 0.028s	user 0.015s	sys 0.000s
I20260709 13:11:12.875947 32371 tablet_bootstrap.cc:492] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227: No bootstrap required, opened a new log
I20260709 13:11:12.876577 32371 ts_tablet_manager.cc:1403] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent bootstrapping tablet: real 0.092s	user 0.024s	sys 0.035s
I20260709 13:11:12.879614 32371 raft_consensus.cc:359] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.880326 32371 raft_consensus.cc:385] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:12.880628 32371 raft_consensus.cc:740] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Initialized, Role: FOLLOWER
I20260709 13:11:12.880777 32373 leader_election.cc:290] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.879812 32146 raft_consensus.cc:2468] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 1.
I20260709 13:11:12.881349 32371 consensus_queue.cc:260] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.882161 32038 leader_election.cc:304] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:12.883508 32373 raft_consensus.cc:2804] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:12.883924 32371 ts_tablet_manager.cc:1434] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:11:12.883695 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:12.884183 32373 raft_consensus.cc:697] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:12.884474 32219 raft_consensus.cc:3060] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.885177 32373 consensus_queue.cc:237] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.887190 31967 catalog_manager.cc:5697] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 1 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.891647 32219 raft_consensus.cc:2468] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 1.
W20260709 13:11:12.898180 32019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:12.909406 31967 catalog_manager.cc:5697] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 1 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:12.924234 32393 raft_consensus.cc:493] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.924779 32393 raft_consensus.cc:515] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.926447 32393 leader_election.cc:290] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.927492 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebb8d66845146b1b2d45c442e803bd1" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:12.927431 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebb8d66845146b1b2d45c442e803bd1" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:12.928380 32219 raft_consensus.cc:2468] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:12.928414 32071 raft_consensus.cc:2468] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:12.929764 32114 leader_election.cc:304] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:12.930451 32393 raft_consensus.cc:2804] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:12.930778 32393 raft_consensus.cc:493] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.931061 32393 raft_consensus.cc:3060] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.936095 32393 raft_consensus.cc:515] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.937724 32393 leader_election.cc:290] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.938709 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebb8d66845146b1b2d45c442e803bd1" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:12.938885 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebb8d66845146b1b2d45c442e803bd1" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:12.939404 32071 raft_consensus.cc:3060] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.939544 32219 raft_consensus.cc:3060] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.945914 32219 raft_consensus.cc:2468] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:12.946014 32071 raft_consensus.cc:2468] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:12.947094 32113 leader_election.cc:304] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:12.948040 32393 raft_consensus.cc:2804] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:12.948483 32393 raft_consensus.cc:697] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:12.949117 32393 consensus_queue.cc:237] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.957602 31967 catalog_manager.cc:5697] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:12.963374 32415 raft_consensus.cc:493] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.963960 32415 raft_consensus.cc:515] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.966035 32415 leader_election.cc:290] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.967116 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981b55af3f1041619f509a52299dab01" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:12.967221 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981b55af3f1041619f509a52299dab01" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:12.968009 32219 raft_consensus.cc:2393] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee90f5b2a1e849739b82a66fdce52a8b in current term 1: Already voted for candidate 3bc3d543183d4ff5a6635ffb2e110227 in this term.
I20260709 13:11:12.968897 32038 leader_election.cc:304] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee90f5b2a1e849739b82a66fdce52a8b; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:12.969578 32415 raft_consensus.cc:3060] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.975654 32415 raft_consensus.cc:2749] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:12.976434 32390 raft_consensus.cc:493] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.976938 32390 raft_consensus.cc:515] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.978673 32390 leader_election.cc:290] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:12.979740 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a4bfa356274b21a59d280e9ac8b477" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:12.979910 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a4bfa356274b21a59d280e9ac8b477" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227" is_pre_election: true
I20260709 13:11:12.980492 32071 raft_consensus.cc:2468] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:12.980634 32146 raft_consensus.cc:2468] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 0.
I20260709 13:11:12.981669 32188 leader_election.cc:304] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:12.982466 32390 raft_consensus.cc:2804] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:12.982875 32390 raft_consensus.cc:493] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.983124 32390 raft_consensus.cc:3060] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.988672 32390 raft_consensus.cc:515] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.990473 32390 leader_election.cc:290] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:12.991469 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a4bfa356274b21a59d280e9ac8b477" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:12.991753 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a4bfa356274b21a59d280e9ac8b477" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:12.991768 32393 raft_consensus.cc:493] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.992318 32071 raft_consensus.cc:3060] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.992585 32146 raft_consensus.cc:3060] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:12.992626 32393 raft_consensus.cc:515] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:12.996371 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3f5770f9654ebcb524ef1028166cbc" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" is_pre_election: true
I20260709 13:11:12.996754 32393 leader_election.cc:290] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:12.997128 32072 raft_consensus.cc:2468] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:12.998423 32114 leader_election.cc:304] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:12.999362 32393 raft_consensus.cc:2804] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:12.999742 32393 raft_consensus.cc:493] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.999830 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3f5770f9654ebcb524ef1028166cbc" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" is_pre_election: true
I20260709 13:11:13.000126 32393 raft_consensus.cc:3060] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:13.000253 32146 raft_consensus.cc:2468] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:13.000411 32071 raft_consensus.cc:2468] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 1.
I20260709 13:11:13.001030 32219 raft_consensus.cc:2468] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 0.
I20260709 13:11:13.001776 32187 leader_election.cc:304] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:13.002753 32390 raft_consensus.cc:2804] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:13.003520 32390 raft_consensus.cc:697] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:13.004388 32390 consensus_queue.cc:237] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.006755 32393 raft_consensus.cc:515] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.009982 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3f5770f9654ebcb524ef1028166cbc" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:13.010658 32071 raft_consensus.cc:3060] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:13.012099 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3f5770f9654ebcb524ef1028166cbc" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:13.012877 32219 raft_consensus.cc:3060] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:13.013286 31967 catalog_manager.cc:5697] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 1 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:13.014587 32393 leader_election.cc:290] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:13.018776 32071 raft_consensus.cc:2468] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:13.020112 32114 leader_election.cc:304] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:13.020948 32393 raft_consensus.cc:2804] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:13.021533 32393 raft_consensus.cc:697] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:13.021682 32219 raft_consensus.cc:2468] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 1.
I20260709 13:11:13.022542 32393 consensus_queue.cc:237] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.026629 32393 consensus_queue.cc:1048] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.032696 31967 catalog_manager.cc:5697] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.045495 32377 consensus_queue.cc:1048] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:13.060858 32394 consensus_queue.cc:1048] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.072477 32400 consensus_queue.cc:1048] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.089462 32219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
mode: GRACEFUL
new_leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:40766
I20260709 13:11:13.089993 32219 raft_consensus.cc:606] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Received request to transfer leadership to TS ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:13.094916 32146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981b55af3f1041619f509a52299dab01"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
mode: GRACEFUL
new_leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:50352
I20260709 13:11:13.095459 32146 raft_consensus.cc:606] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Received request to transfer leadership to TS ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:13.100214 32219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e06665820de481db4d97fa058f082f3"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
mode: GRACEFUL
new_leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:40766
I20260709 13:11:13.100689 32219 raft_consensus.cc:606] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Received request to transfer leadership to TS ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:13.105587 32146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26975e63d5a54a1399df116dc00be42a"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
mode: GRACEFUL
new_leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:50352
I20260709 13:11:13.106027 32146 raft_consensus.cc:606] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Received request to transfer leadership to TS ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:13.110962 32219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2150aff863544382a1801920674d76be"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
mode: GRACEFUL
new_leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:40766
I20260709 13:11:13.111541 32219 raft_consensus.cc:606] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Received request to transfer leadership to TS 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:13.116621 32219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54a4bfa356274b21a59d280e9ac8b477"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
mode: GRACEFUL
new_leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:40766
I20260709 13:11:13.117120 32219 raft_consensus.cc:606] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Received request to transfer leadership to TS 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:13.125396 32071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
mode: GRACEFUL
new_leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:60832
I20260709 13:11:13.126003 32071 raft_consensus.cc:606] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Received request to transfer leadership to TS 3bc3d543183d4ff5a6635ffb2e110227
I20260709 13:11:13.131994 32146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ebb8d66845146b1b2d45c442e803bd1"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
mode: GRACEFUL
new_leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:50352
I20260709 13:11:13.132515 32146 raft_consensus.cc:606] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Received request to transfer leadership to TS 1b650608729b4ad6a28fd1c7e3d30d1c
I20260709 13:11:13.170274 32400 consensus_queue.cc:1048] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.181257 32390 consensus_queue.cc:1048] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.190188 32394 raft_consensus.cc:993] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c: : Instructing follower ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:13.190718 32390 raft_consensus.cc:1081] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Signalling peer ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:13.192433 32071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:60826
I20260709 13:11:13.193073 32071 raft_consensus.cc:493] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:13.193455 32071 raft_consensus.cc:3060] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.196606 32330 consensus_queue.cc:1048] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:13.201037 32071 raft_consensus.cc:515] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.203315 32071 leader_election.cc:290] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:13.205132 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:13.205408 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:13.205653 32390 consensus_queue.cc:1048] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.206077 32222 raft_consensus.cc:3055] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.206427 32222 raft_consensus.cc:740] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:13.205717 32146 raft_consensus.cc:3060] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.207188 32222 consensus_queue.cc:260] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.208559 32222 raft_consensus.cc:3060] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.215137 32146 raft_consensus.cc:2468] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:13.215435 32222 raft_consensus.cc:2468] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:13.216684 32038 leader_election.cc:304] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:13.217559 32415 raft_consensus.cc:2804] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:13.218066 32415 raft_consensus.cc:697] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:13.218889 32415 consensus_queue.cc:237] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.229758 32394 consensus_queue.cc:1048] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.235474 32330 consensus_queue.cc:1048] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.248697 32390 raft_consensus.cc:993] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c: : Instructing follower 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:13.249233 32390 raft_consensus.cc:1081] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Signalling peer 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:13.247320 31967 catalog_manager.cc:5697] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 1 to 2, leader changed from 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195) to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 2 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.252566 32146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2150aff863544382a1801920674d76be"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:50336
I20260709 13:11:13.253193 32146 raft_consensus.cc:493] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:13.253669 32146 raft_consensus.cc:3060] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.260830 32146 raft_consensus.cc:515] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.263685 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2150aff863544382a1801920674d76be" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:13.264341 32071 raft_consensus.cc:3060] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.264554 32146 leader_election.cc:290] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:13.266512 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2150aff863544382a1801920674d76be" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:13.267117 32222 raft_consensus.cc:3055] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.267931 32222 raft_consensus.cc:740] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:13.268718 32222 consensus_queue.cc:260] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.269943 32222 raft_consensus.cc:3060] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.274437 32071 raft_consensus.cc:2468] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 2.
I20260709 13:11:13.275107 32432 raft_consensus.cc:993] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227: : Instructing follower ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:13.275710 32432 raft_consensus.cc:1081] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Signalling peer ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:13.275759 32114 leader_election.cc:304] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:13.276785 32432 raft_consensus.cc:2804] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:13.277436 32432 raft_consensus.cc:697] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:13.277750 32071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26975e63d5a54a1399df116dc00be42a"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:60820
I20260709 13:11:13.278362 32071 raft_consensus.cc:493] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:13.278716 32222 raft_consensus.cc:2468] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 2.
I20260709 13:11:13.278228 32432 consensus_queue.cc:237] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.279424 32071 raft_consensus.cc:3060] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.287413 32071 raft_consensus.cc:515] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.289381 32071 leader_election.cc:290] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:13.290251 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26975e63d5a54a1399df116dc00be42a" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:13.290555 32222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26975e63d5a54a1399df116dc00be42a" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:13.290946 32146 raft_consensus.cc:3055] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.291378 32146 raft_consensus.cc:740] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:13.291194 32222 raft_consensus.cc:3060] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.290802 31970 catalog_manager.cc:5697] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 1 to 2, leader changed from 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195) to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.292341 32146 consensus_queue.cc:260] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.293563 32146 raft_consensus.cc:3060] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.298169 32222 raft_consensus.cc:2468] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:13.299791 32146 raft_consensus.cc:2468] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:13.299716 32038 leader_election.cc:304] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:13.300870 32433 raft_consensus.cc:2804] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:13.301337 32433 raft_consensus.cc:697] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:13.302084 32433 consensus_queue.cc:237] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.308959 31970 catalog_manager.cc:5697] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 1 to 2, leader changed from 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194) to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 2 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.350343 32432 consensus_queue.cc:1048] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.357136 32393 raft_consensus.cc:993] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227: : Instructing follower 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:13.357654 32330 raft_consensus.cc:1081] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Signalling peer 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:13.359807 32222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ebb8d66845146b1b2d45c442e803bd1"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:40734
I20260709 13:11:13.363575 32222 raft_consensus.cc:493] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:13.364029 32222 raft_consensus.cc:3060] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.363916 32330 consensus_queue.cc:1048] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:13.373725 32222 raft_consensus.cc:515] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.376297 32222 leader_election.cc:290] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 2 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:13.379024 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebb8d66845146b1b2d45c442e803bd1" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:13.379736 32146 raft_consensus.cc:3055] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.380055 32146 raft_consensus.cc:740] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:13.380756 32146 consensus_queue.cc:260] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.381973 32146 raft_consensus.cc:3060] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.384251 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebb8d66845146b1b2d45c442e803bd1" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:13.384908 32073 raft_consensus.cc:3060] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.389463 32146 raft_consensus.cc:2468] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 2.
I20260709 13:11:13.391185 32187 leader_election.cc:304] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:13.392198 32390 raft_consensus.cc:2804] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:13.392707 32390 raft_consensus.cc:697] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:13.393566 32433 consensus_queue.cc:1048] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.393594 32390 consensus_queue.cc:237] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.392707 32073 raft_consensus.cc:2468] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 2.
I20260709 13:11:13.402530 31970 catalog_manager.cc:5697] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 1 to 2, leader changed from 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194) to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 2 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:13.404652 32373 raft_consensus.cc:993] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b: : Instructing follower 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:13.405236 32415 raft_consensus.cc:1081] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Signalling peer 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:13.407573 32146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:50326
I20260709 13:11:13.408128 32146 raft_consensus.cc:493] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:13.408524 32146 raft_consensus.cc:3060] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.410367 32415 consensus_queue.cc:1048] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.415248 32146 raft_consensus.cc:515] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.416450 32148 raft_consensus.cc:1240] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Rejecting Update request from peer ee90f5b2a1e849739b82a66fdce52a8b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:13.417450 32146 leader_election.cc:290] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:13.419700 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:13.419168 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8ca9fffb23a4ec5820b34d42497b5dc" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:13.420356 32073 raft_consensus.cc:3055] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.421944 32073 raft_consensus.cc:740] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:13.422748 32073 consensus_queue.cc:260] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.423646 32433 consensus_queue.cc:1059] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.424813 32073 raft_consensus.cc:3060] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.430428 32073 raft_consensus.cc:2468] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 2.
I20260709 13:11:13.431443 32114 leader_election.cc:304] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 1b650608729b4ad6a28fd1c7e3d30d1c
I20260709 13:11:13.432170 32330 raft_consensus.cc:2804] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:13.432541 32330 raft_consensus.cc:697] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:13.433295 32330 consensus_queue.cc:237] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.439826 31967 catalog_manager.cc:5697] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 1 to 2, leader changed from ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193) to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.478583 32390 consensus_queue.cc:1048] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.486291 32400 raft_consensus.cc:993] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c: : Instructing follower 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:13.486845 32394 raft_consensus.cc:1081] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Signalling peer 3bc3d543183d4ff5a6635ffb2e110227 to start an election
I20260709 13:11:13.491044 32148 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54a4bfa356274b21a59d280e9ac8b477"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
 from {username='slave'} at 127.0.0.1:50336
I20260709 13:11:13.491765 32148 raft_consensus.cc:493] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:13.491698 32394 consensus_queue.cc:1048] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:13.492456 32148 raft_consensus.cc:3060] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.498150 32330 consensus_queue.cc:1048] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:13.500885 32148 raft_consensus.cc:515] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.503543 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a4bfa356274b21a59d280e9ac8b477" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:13.504134 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a4bfa356274b21a59d280e9ac8b477" candidate_uuid: "3bc3d543183d4ff5a6635ffb2e110227" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:13.505004 32219 raft_consensus.cc:3055] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.505400 32219 raft_consensus.cc:740] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:13.506206 32148 leader_election.cc:290] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:13.506489 32219 consensus_queue.cc:260] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.508054 32219 raft_consensus.cc:3060] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.517174 32219 raft_consensus.cc:2468] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bc3d543183d4ff5a6635ffb2e110227 in term 2.
I20260709 13:11:13.518354 32113 leader_election.cc:304] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: ee90f5b2a1e849739b82a66fdce52a8b
I20260709 13:11:13.519866 32377 raft_consensus.cc:2804] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:13.520328 32377 raft_consensus.cc:697] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Becoming Leader. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:13.521116 32377 consensus_queue.cc:237] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.522679 32393 consensus_queue.cc:1048] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:13.530368 32432 raft_consensus.cc:993] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227: : Instructing follower ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:13.530906 32432 raft_consensus.cc:1081] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Signalling peer ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:13.532989 32073 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "981b55af3f1041619f509a52299dab01"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:60820
I20260709 13:11:13.533563 32073 raft_consensus.cc:493] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:13.534008 32073 raft_consensus.cc:3060] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.541252 32073 raft_consensus.cc:515] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.543907 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981b55af3f1041619f509a52299dab01" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:13.544489 32148 raft_consensus.cc:3055] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.544927 32148 raft_consensus.cc:740] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:13.545730 32148 consensus_queue.cc:260] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.546478 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981b55af3f1041619f509a52299dab01" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:13.545846 31970 catalog_manager.cc:5697] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 reported cstate change: term changed from 1 to 2, leader changed from 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195) to 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "3bc3d543183d4ff5a6635ffb2e110227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.546861 32148 raft_consensus.cc:3060] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.547199 32219 raft_consensus.cc:3060] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.544467 32073 leader_election.cc:290] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:13.553818 32219 raft_consensus.cc:2468] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:13.555140 32038 leader_election.cc:304] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:13.555624 32148 raft_consensus.cc:2468] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:13.556053 32433 raft_consensus.cc:2804] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:13.556595 32433 raft_consensus.cc:697] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:13.557412 32433 consensus_queue.cc:237] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.563879 31970 catalog_manager.cc:5697] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 1 to 2, leader changed from 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194) to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 2 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.608614 32148 raft_consensus.cc:1275] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.610136 32433 consensus_queue.cc:1048] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.618398 32219 raft_consensus.cc:1275] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.620461 32373 consensus_queue.cc:1048] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.631521 32390 raft_consensus.cc:993] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c: : Instructing follower ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:13.632267 32394 raft_consensus.cc:1081] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Signalling peer ee90f5b2a1e849739b82a66fdce52a8b to start an election
I20260709 13:11:13.633991 32073 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e06665820de481db4d97fa058f082f3"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
 from {username='slave'} at 127.0.0.1:60826
I20260709 13:11:13.634524 32073 raft_consensus.cc:493] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:13.634905 32073 raft_consensus.cc:3060] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.641697 32073 raft_consensus.cc:515] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.643738 32073 leader_election.cc:290] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Requested vote from peers 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657), 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195:43043)
I20260709 13:11:13.644471 32148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e06665820de481db4d97fa058f082f3" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:13.644825 32219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e06665820de481db4d97fa058f082f3" candidate_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
I20260709 13:11:13.645202 32148 raft_consensus.cc:3060] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.645450 32219 raft_consensus.cc:3055] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.645838 32219 raft_consensus.cc:740] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:13.646621 32219 consensus_queue.cc:260] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.647886 32219 raft_consensus.cc:3060] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.652712 32148 raft_consensus.cc:2468] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:13.654206 32038 leader_election.cc:304] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc3d543183d4ff5a6635ffb2e110227, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:13.654477 32219 raft_consensus.cc:2468] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee90f5b2a1e849739b82a66fdce52a8b in term 2.
I20260709 13:11:13.655213 32415 raft_consensus.cc:2804] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:13.655787 32415 raft_consensus.cc:697] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Becoming Leader. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:13.656515 32415 consensus_queue.cc:237] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:13.663224 31969 catalog_manager.cc:5697] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b reported cstate change: term changed from 1 to 2, leader changed from 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195) to ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193). New cstate: current_term: 2 leader_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.745949 32073 raft_consensus.cc:1275] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.747356 32377 consensus_queue.cc:1048] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.755154 32219 raft_consensus.cc:1275] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.756726 32432 consensus_queue.cc:1048] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.767475 32219 raft_consensus.cc:1275] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.768790 32373 consensus_queue.cc:1048] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.776969 32148 raft_consensus.cc:1275] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.779186 32415 consensus_queue.cc:1048] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.811721 32219 raft_consensus.cc:3060] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.817116 32219 raft_consensus.cc:1275] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.818339 32432 consensus_queue.cc:1048] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.824002 32148 raft_consensus.cc:1275] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.827661 32073 raft_consensus.cc:1275] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.828765 32394 consensus_queue.cc:1048] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.829270 32432 consensus_queue.cc:1048] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.840401 32073 raft_consensus.cc:1275] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.841920 32390 consensus_queue.cc:1048] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.016288 32219 raft_consensus.cc:1275] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.017730 32415 consensus_queue.cc:1048] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.025707 32148 raft_consensus.cc:1275] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.028007 32373 consensus_queue.cc:1048] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.104300 32219 raft_consensus.cc:1275] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.105963 32330 consensus_queue.cc:1048] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.116369 32073 raft_consensus.cc:3060] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.124751 32073 raft_consensus.cc:1275] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Refusing update from remote peer 3bc3d543183d4ff5a6635ffb2e110227: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.126150 32432 consensus_queue.cc:1048] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.145864 32148 raft_consensus.cc:1275] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.147367 32373 consensus_queue.cc:1048] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.155064 32219 raft_consensus.cc:1275] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Refusing update from remote peer ee90f5b2a1e849739b82a66fdce52a8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.156533 32433 consensus_queue.cc:1048] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.135681 31969 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58186:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:11:16.156122 31969 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58186:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:11:16.357730 32479 tablet.cc:1724] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.374670 19967 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:11:16.425923 32486 tablet.cc:1724] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:16.441339 32038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:46657, user_credentials={real_user=slave}} blocked reactor thread for 67553.1us
I20260709 13:11:16.481662 32497 tablet.cc:1724] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:16.488083 32248 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:54a4bfa356274b21a59d280e9ac8b477)
W20260709 13:11:16.522291 32248 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:54a4bfa356274b21a59d280e9ac8b477)
W20260709 13:11:16.525100 32248 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:54a4bfa356274b21a59d280e9ac8b477)
I20260709 13:11:16.525063 32501 tablet.cc:1724] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.482158 32493 tablet.cc:1724] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.551729 32498 tablet.cc:1724] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:16.536283 32495 log.cc:927] Time spent T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 13:11:16.591398 32502 log.cc:927] Time spent T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:11:16.633140 32038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:46657, user_credentials={real_user=slave}} blocked reactor thread for 188087us
I20260709 13:11:16.661751 32493 tablet.cc:1724] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.684159 32501 tablet.cc:1724] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.718017 32509 tablet.cc:1724] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.686293 32510 tablet.cc:1724] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.730840 32479 tablet.cc:1724] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.693969 32512 tablet.cc:1724] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.711946 32497 tablet.cc:1724] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.752091 32521 tablet.cc:1724] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.764951 32498 tablet.cc:1724] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:16.782689 32038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.194:46657, user_credentials={real_user=slave}} blocked reactor thread for 110064us
I20260709 13:11:16.782207 32493 tablet.cc:1724] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.788673 32519 tablet.cc:1724] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.814357 32486 tablet.cc:1724] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.818074 32493 tablet.cc:1724] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.821712 32497 tablet.cc:1724] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.831523 32072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
mode: GRACEFUL
new_leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:43440
I20260709 13:11:16.832085 32519 tablet.cc:1724] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.834964 32072 raft_consensus.cc:606] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Received request to transfer leadership to TS 1b650608729b4ad6a28fd1c7e3d30d1c
I20260709 13:11:16.847374 32524 tablet.cc:1724] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.858166 32523 raft_consensus.cc:993] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b: : Instructing follower 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:16.859246 32520 raft_consensus.cc:1081] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Signalling peer 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:16.844130 32501 tablet.cc:1724] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.864553 32218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:40750
I20260709 13:11:16.865175 32218 raft_consensus.cc:493] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:16.865649 32218 raft_consensus.cc:3060] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:16.876873 32510 tablet.cc:1724] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.857290 32521 tablet.cc:1724] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.909632 32218 raft_consensus.cc:515] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:16.914350 32146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea3f5770f9654ebcb524ef1028166cbc"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
mode: GRACEFUL
new_leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:11:16.915016 32146 raft_consensus.cc:606] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Received request to transfer leadership to TS 1b650608729b4ad6a28fd1c7e3d30d1c
I20260709 13:11:16.917443 19967 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:11:16.918128 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:16.925555 32479 tablet.cc:1724] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.930668 32497 tablet.cc:1724] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:16.932679 32218 leader_election.cc:290] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 3 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:16.933254 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:16.940603 32071 raft_consensus.cc:3055] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:16.941008 32071 raft_consensus.cc:740] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:16.941864 32071 consensus_queue.cc:260] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:16.933712 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e291faba389e4a21b5bf01b3a0de914d" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:16.943773 32146 raft_consensus.cc:3060] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:16.943058 32071 raft_consensus.cc:3060] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:16.948352 32072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7"
dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
mode: GRACEFUL
new_leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:43440
I20260709 13:11:16.948997 32072 raft_consensus.cc:606] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Received request to transfer leadership to TS 1b650608729b4ad6a28fd1c7e3d30d1c
I20260709 13:11:16.953433 32146 raft_consensus.cc:2468] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 3.
I20260709 13:11:16.953328 32145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3"
dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
mode: GRACEFUL
new_leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:11:16.956372 32145 raft_consensus.cc:606] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Received request to transfer leadership to TS 1b650608729b4ad6a28fd1c7e3d30d1c
I20260709 13:11:16.958150 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:11:16.958767 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:16.960204 32071 raft_consensus.cc:2468] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 3.
I20260709 13:11:16.961594 32188 leader_election.cc:304] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:16.962862 32507 raft_consensus.cc:2804] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:16.963431 32507 raft_consensus.cc:697] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:16.964308 32507 consensus_queue.cc:237] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:16.974149 31969 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246: soft_deleted_table [id=84d8a11b226241df9af0356cb52751aa] alter complete (version 1)
I20260709 13:11:16.982357 32465 raft_consensus.cc:993] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227: : Instructing follower 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:16.983031 32483 raft_consensus.cc:1081] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Signalling peer 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:16.987023 32218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea3f5770f9654ebcb524ef1028166cbc"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:40734
I20260709 13:11:16.987690 32218 raft_consensus.cc:493] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:16.988081 32218 raft_consensus.cc:3060] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.989611 31971 catalog_manager.cc:5697] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 2 to 3, leader changed from ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193) to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 3 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:16.995417 32218 raft_consensus.cc:515] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:16.997784 32218 leader_election.cc:290] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 2 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:16.998557 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3f5770f9654ebcb524ef1028166cbc" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:16.998881 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3f5770f9654ebcb524ef1028166cbc" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:16.999261 32071 raft_consensus.cc:3060] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.999557 32146 raft_consensus.cc:3055] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:16.999930 32146 raft_consensus.cc:740] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:17.000595 32146 consensus_queue.cc:260] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:17.001487 32146 raft_consensus.cc:3060] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.004912 32071 raft_consensus.cc:2468] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 2.
I20260709 13:11:17.006150 32146 raft_consensus.cc:2468] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 2.
I20260709 13:11:17.006176 32188 leader_election.cc:304] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:17.007274 32507 raft_consensus.cc:2804] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:17.007792 32507 raft_consensus.cc:697] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:17.008567 32507 consensus_queue.cc:237] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:17.015635 31971 catalog_manager.cc:5697] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 1 to 2, leader changed from 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194) to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 2 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:17.127059 32483 raft_consensus.cc:993] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227: : Instructing follower 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:17.127584 32465 raft_consensus.cc:1081] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Signalling peer 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:17.129180 32218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:40734
I20260709 13:11:17.129680 32218 raft_consensus.cc:493] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:17.130012 32218 raft_consensus.cc:3060] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:17.136533 32218 raft_consensus.cc:515] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:17.138741 32218 leader_election.cc:290] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 3 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:17.139632 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:17.140115 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f604db35eda34a43997c049f9d49baa3" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:17.140444 32071 raft_consensus.cc:3060] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:17.140815 32146 raft_consensus.cc:3055] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:17.141242 32146 raft_consensus.cc:740] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bc3d543183d4ff5a6635ffb2e110227, State: Running, Role: LEADER
I20260709 13:11:17.142138 32146 consensus_queue.cc:260] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:17.143503 32146 raft_consensus.cc:3060] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:17.148859 32071 raft_consensus.cc:2468] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 3.
I20260709 13:11:17.150260 32188 leader_election.cc:304] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, ee90f5b2a1e849739b82a66fdce52a8b; no voters: 
I20260709 13:11:17.150939 32146 raft_consensus.cc:2468] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 3.
I20260709 13:11:17.151082 32507 raft_consensus.cc:2804] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:17.151969 32507 raft_consensus.cc:697] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:17.152875 32507 consensus_queue.cc:237] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:17.160682 31971 catalog_manager.cc:5697] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 2 to 3, leader changed from 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194) to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 3 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:17.171046 32525 raft_consensus.cc:993] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b: : Instructing follower 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:17.171663 32525 raft_consensus.cc:1081] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Signalling peer 1b650608729b4ad6a28fd1c7e3d30d1c to start an election
I20260709 13:11:17.173239 32222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7"
dest_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c"
 from {username='slave'} at 127.0.0.1:40750
I20260709 13:11:17.173691 32222 raft_consensus.cc:493] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:17.174002 32222 raft_consensus.cc:3060] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:17.178651 32222 raft_consensus.cc:515] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:17.180322 32222 leader_election.cc:290] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 3 election: Requested vote from peers ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055), 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657)
I20260709 13:11:17.181221 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ee90f5b2a1e849739b82a66fdce52a8b"
I20260709 13:11:17.181376 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc00c7f1fec24b569dd708c0b47e1ae7" candidate_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3bc3d543183d4ff5a6635ffb2e110227"
I20260709 13:11:17.181905 32146 raft_consensus.cc:3060] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:17.181908 32071 raft_consensus.cc:3055] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:17.182343 32071 raft_consensus.cc:740] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee90f5b2a1e849739b82a66fdce52a8b, State: Running, Role: LEADER
I20260709 13:11:17.183085 32071 consensus_queue.cc:260] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:17.184247 32071 raft_consensus.cc:3060] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:17.188725 32146 raft_consensus.cc:2468] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 3.
I20260709 13:11:17.190018 32187 leader_election.cc:304] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b650608729b4ad6a28fd1c7e3d30d1c, 3bc3d543183d4ff5a6635ffb2e110227; no voters: 
I20260709 13:11:17.190636 32071 raft_consensus.cc:2468] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b650608729b4ad6a28fd1c7e3d30d1c in term 3.
I20260709 13:11:17.190760 32507 raft_consensus.cc:2804] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:17.191443 32507 raft_consensus.cc:697] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 LEADER]: Becoming Leader. State: Replica: 1b650608729b4ad6a28fd1c7e3d30d1c, State: Running, Role: LEADER
I20260709 13:11:17.192318 32507 consensus_queue.cc:237] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } }
I20260709 13:11:17.200695 31971 catalog_manager.cc:5697] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c reported cstate change: term changed from 2 to 3, leader changed from ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193) to 1b650608729b4ad6a28fd1c7e3d30d1c (127.19.127.195). New cstate: current_term: 3 leader_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b650608729b4ad6a28fd1c7e3d30d1c" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:17.426412 32072 raft_consensus.cc:1275] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:17.427596 32507 consensus_queue.cc:1048] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:17.494131 32146 raft_consensus.cc:1275] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:11:17.496320 32188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:43055, user_credentials={real_user=slave}} blocked reactor thread for 58594.9us
I20260709 13:11:17.497354 32448 consensus_queue.cc:1048] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:11:17.504065 32072 raft_consensus.cc:1275] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:17.506217 32535 consensus_queue.cc:1048] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:17.515928 32145 raft_consensus.cc:1275] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:17.517517 32535 consensus_queue.cc:1048] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:17.583982 32145 raft_consensus.cc:1275] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:17.585238 32538 consensus_queue.cc:1048] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:17.594342 32072 raft_consensus.cc:1275] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:17.596086 32535 consensus_queue.cc:1048] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:17.661051 32072 raft_consensus.cc:1275] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:17.662297 32535 consensus_queue.cc:1048] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee90f5b2a1e849739b82a66fdce52a8b" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:17.672402 32145 raft_consensus.cc:1275] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Refusing update from remote peer 1b650608729b4ad6a28fd1c7e3d30d1c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:17.673852 32535 consensus_queue.cc:1048] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc3d543183d4ff5a6635ffb2e110227" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:18.023739 19967 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:11:18.030460 19967 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:11:18.030864 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:18.035543 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:11:18.035931 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:19.071273 19967 tablet_server.cc:179] TabletServer@127.19.127.193:0 shutting down...
I20260709 13:11:19.105595 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:19.106608 19967 tablet_replica.cc:333] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.108491 19967 raft_consensus.cc:2243] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.109438 19967 raft_consensus.cc:2272] T ea3f5770f9654ebcb524ef1028166cbc P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.112449 19967 tablet_replica.cc:333] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.113190 19967 raft_consensus.cc:2243] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.113821 19967 raft_consensus.cc:2272] T c8ca9fffb23a4ec5820b34d42497b5dc P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.116631 19967 tablet_replica.cc:333] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.117441 19967 raft_consensus.cc:2243] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.118753 19967 raft_consensus.cc:2272] T 981b55af3f1041619f509a52299dab01 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.121433 19967 tablet_replica.cc:333] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.122134 19967 raft_consensus.cc:2243] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.122735 19967 raft_consensus.cc:2272] T 2150aff863544382a1801920674d76be P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.125562 19967 tablet_replica.cc:333] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.126178 19967 raft_consensus.cc:2243] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.127135 19967 raft_consensus.cc:2272] T 4e06665820de481db4d97fa058f082f3 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.129698 19967 tablet_replica.cc:333] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.130338 19967 raft_consensus.cc:2243] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.130976 19967 raft_consensus.cc:2272] T 0ebb8d66845146b1b2d45c442e803bd1 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.133598 19967 tablet_replica.cc:333] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.134196 19967 raft_consensus.cc:2243] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.135110 19967 raft_consensus.cc:2272] T 9e05e529aa954954acd28cf6021b0d35 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.138914 19967 tablet_replica.cc:333] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.139552 19967 raft_consensus.cc:2243] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.140544 19967 raft_consensus.cc:2272] T 26975e63d5a54a1399df116dc00be42a P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.143072 19967 tablet_replica.cc:333] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.143757 19967 raft_consensus.cc:2243] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.144724 19967 raft_consensus.cc:2272] T b28632f2919847d2b35e28a7baff3fa0 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.147567 19967 tablet_replica.cc:333] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.148182 19967 raft_consensus.cc:2243] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.148883 19967 raft_consensus.cc:2272] T e291faba389e4a21b5bf01b3a0de914d P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.151736 19967 tablet_replica.cc:333] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.152390 19967 raft_consensus.cc:2243] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.153048 19967 raft_consensus.cc:2272] T 54a4bfa356274b21a59d280e9ac8b477 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:19.156355 32114 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
I20260709 13:11:19.156489 19967 tablet_replica.cc:333] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.157398 19967 raft_consensus.cc:2243] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.158125 19967 raft_consensus.cc:2272] T cc00c7f1fec24b569dd708c0b47e1ae7 P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:19.160223 32114 consensus_peers.cc:597] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 -> Peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Couldn't send request to peer ee90f5b2a1e849739b82a66fdce52a8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.161477 19967 tablet_replica.cc:333] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.162458 19967 raft_consensus.cc:2243] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.163928 19967 raft_consensus.cc:2272] T c2b786f061514123aca8f3d1750e39d9 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.167194 19967 tablet_replica.cc:333] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.167883 19967 raft_consensus.cc:2243] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.168545 19967 raft_consensus.cc:2272] T f27ad9f1bb0349b4bb39afe329cf59d8 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.172308 19967 tablet_replica.cc:333] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.173194 19967 raft_consensus.cc:2243] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.173949 19967 raft_consensus.cc:2272] T 93f3937eb75d421abe285280d24d430d P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.177130 19967 tablet_replica.cc:333] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.177903 19967 raft_consensus.cc:2243] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.178471 19967 raft_consensus.cc:2272] T fdef32d47736414d8b9a2f7d897ecee8 P ee90f5b2a1e849739b82a66fdce52a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.182351 19967 tablet_replica.cc:333] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
W20260709 13:11:19.182547 32188 consensus_peers.cc:597] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Couldn't send request to peer ee90f5b2a1e849739b82a66fdce52a8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.183097 19967 raft_consensus.cc:2243] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.183838 19967 raft_consensus.cc:2272] T f604db35eda34a43997c049f9d49baa3 P ee90f5b2a1e849739b82a66fdce52a8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.186770 19967 tablet_replica.cc:333] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b: stopping tablet replica
I20260709 13:11:19.187337 19967 raft_consensus.cc:2243] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.188257 19967 raft_consensus.cc:2272] T 3dda35f1227549ff933e504aadd37233 P ee90f5b2a1e849739b82a66fdce52a8b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:19.223425 32188 consensus_peers.cc:597] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Couldn't send request to peer ee90f5b2a1e849739b82a66fdce52a8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:19.233492 32114 consensus_peers.cc:597] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 -> Peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Couldn't send request to peer ee90f5b2a1e849739b82a66fdce52a8b. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:43055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.240161 19967 tablet_server.cc:196] TabletServer@127.19.127.193:0 shutdown complete.
I20260709 13:11:19.288582 19967 tablet_server.cc:179] TabletServer@127.19.127.194:0 shutting down...
W20260709 13:11:19.298014 32114 consensus_peers.cc:597] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 -> Peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Couldn't send request to peer ee90f5b2a1e849739b82a66fdce52a8b. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:43055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:19.328676 32188 consensus_peers.cc:597] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Couldn't send request to peer ee90f5b2a1e849739b82a66fdce52a8b. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:43055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.330205 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:19.331086 19967 tablet_replica.cc:333] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.332044 19967 raft_consensus.cc:2243] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.332721 19967 raft_consensus.cc:2272] T ea3f5770f9654ebcb524ef1028166cbc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.336053 19967 tablet_replica.cc:333] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.337074 19967 raft_consensus.cc:2243] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.338362 19967 raft_consensus.cc:2272] T 2150aff863544382a1801920674d76be P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:19.338871 32188 consensus_peers.cc:597] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Couldn't send request to peer ee90f5b2a1e849739b82a66fdce52a8b. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:43055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.341063 19967 tablet_replica.cc:333] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.341760 19967 raft_consensus.cc:2243] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.342250 19967 raft_consensus.cc:2272] T 4e06665820de481db4d97fa058f082f3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.344810 19967 tablet_replica.cc:333] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.345418 19967 raft_consensus.cc:2243] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.346307 19967 raft_consensus.cc:2272] T c8ca9fffb23a4ec5820b34d42497b5dc P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.349138 19967 tablet_replica.cc:333] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.349776 19967 raft_consensus.cc:2243] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.350414 19967 raft_consensus.cc:2272] T b28632f2919847d2b35e28a7baff3fa0 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.352869 19967 tablet_replica.cc:333] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.353454 19967 raft_consensus.cc:2243] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.354058 19967 raft_consensus.cc:2272] T e291faba389e4a21b5bf01b3a0de914d P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.357376 19967 tablet_replica.cc:333] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.358016 19967 raft_consensus.cc:2243] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:11:19.358494 32187 consensus_peers.cc:597] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657): Couldn't send request to peer 3bc3d543183d4ff5a6635ffb2e110227. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.359187 19967 raft_consensus.cc:2272] T fdef32d47736414d8b9a2f7d897ecee8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.361943 19967 tablet_replica.cc:333] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.362666 19967 raft_consensus.cc:2243] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.363202 19967 raft_consensus.cc:2272] T 0ebb8d66845146b1b2d45c442e803bd1 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.365525 19967 tablet_replica.cc:333] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.366142 19967 raft_consensus.cc:2243] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.366671 19967 raft_consensus.cc:2272] T 26975e63d5a54a1399df116dc00be42a P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.368988 19967 tablet_replica.cc:333] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.369616 19967 raft_consensus.cc:2243] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.370558 19967 raft_consensus.cc:2272] T 9e05e529aa954954acd28cf6021b0d35 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.372946 19967 tablet_replica.cc:333] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.373584 19967 raft_consensus.cc:2243] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.374204 19967 raft_consensus.cc:2272] T 981b55af3f1041619f509a52299dab01 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.376899 19967 tablet_replica.cc:333] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.377511 19967 raft_consensus.cc:2243] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.378041 19967 raft_consensus.cc:2272] T f604db35eda34a43997c049f9d49baa3 P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.380304 19967 tablet_replica.cc:333] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.380908 19967 raft_consensus.cc:2243] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.381418 19967 raft_consensus.cc:2272] T c2b786f061514123aca8f3d1750e39d9 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.383863 19967 tablet_replica.cc:333] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.384426 19967 raft_consensus.cc:2243] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.384945 19967 raft_consensus.cc:2272] T f27ad9f1bb0349b4bb39afe329cf59d8 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.387509 19967 tablet_replica.cc:333] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.388118 19967 raft_consensus.cc:2243] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.389137 19967 raft_consensus.cc:2272] T 93f3937eb75d421abe285280d24d430d P 3bc3d543183d4ff5a6635ffb2e110227 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.391603 19967 tablet_replica.cc:333] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.392243 19967 raft_consensus.cc:2243] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.393239 19967 raft_consensus.cc:2272] T 54a4bfa356274b21a59d280e9ac8b477 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.395675 19967 tablet_replica.cc:333] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.396199 19967 raft_consensus.cc:2243] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.396782 19967 raft_consensus.cc:2272] T cc00c7f1fec24b569dd708c0b47e1ae7 P 3bc3d543183d4ff5a6635ffb2e110227 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.399173 19967 tablet_replica.cc:333] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227: stopping tablet replica
I20260709 13:11:19.399745 19967 raft_consensus.cc:2243] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.400262 19967 raft_consensus.cc:2272] T 3dda35f1227549ff933e504aadd37233 P 3bc3d543183d4ff5a6635ffb2e110227 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:19.421272 32187 consensus_peers.cc:597] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657): Couldn't send request to peer 3bc3d543183d4ff5a6635ffb2e110227. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.447706 19967 tablet_server.cc:196] TabletServer@127.19.127.194:0 shutdown complete.
W20260709 13:11:19.473801 32188 consensus_peers.cc:597] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Couldn't send request to peer ee90f5b2a1e849739b82a66fdce52a8b. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:43055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.490881 19967 tablet_server.cc:179] TabletServer@127.19.127.195:0 shutting down...
W20260709 13:11:19.509516 32187 consensus_peers.cc:597] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657): Couldn't send request to peer 3bc3d543183d4ff5a6635ffb2e110227. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:46657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.526207 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:19.527065 19967 tablet_replica.cc:333] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.527902 19967 raft_consensus.cc:2243] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.528548 19967 raft_consensus.cc:2272] T c8ca9fffb23a4ec5820b34d42497b5dc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.531066 19967 tablet_replica.cc:333] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.531781 19967 raft_consensus.cc:2243] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.532720 19967 raft_consensus.cc:2272] T ea3f5770f9654ebcb524ef1028166cbc P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.535465 19967 tablet_replica.cc:333] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.536528 19967 raft_consensus.cc:2243] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.537140 19967 raft_consensus.cc:2272] T 2150aff863544382a1801920674d76be P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:19.539136 32187 consensus_peers.cc:597] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer 3bc3d543183d4ff5a6635ffb2e110227 (127.19.127.194:46657): Couldn't send request to peer 3bc3d543183d4ff5a6635ffb2e110227. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.194:46657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.540198 19967 tablet_replica.cc:333] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.540874 19967 raft_consensus.cc:2243] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.541416 19967 raft_consensus.cc:2272] T 4e06665820de481db4d97fa058f082f3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.543985 19967 tablet_replica.cc:333] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.544615 19967 raft_consensus.cc:2243] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.545114 19967 raft_consensus.cc:2272] T fdef32d47736414d8b9a2f7d897ecee8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.547470 19967 tablet_replica.cc:333] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.548108 19967 raft_consensus.cc:2243] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.548987 19967 raft_consensus.cc:2272] T 0ebb8d66845146b1b2d45c442e803bd1 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.551275 19967 tablet_replica.cc:333] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.551844 19967 raft_consensus.cc:2243] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.552397 19967 raft_consensus.cc:2272] T 26975e63d5a54a1399df116dc00be42a P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.554652 19967 tablet_replica.cc:333] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.555215 19967 raft_consensus.cc:2243] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.555797 19967 raft_consensus.cc:2272] T 9e05e529aa954954acd28cf6021b0d35 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.558108 19967 tablet_replica.cc:333] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.558660 19967 raft_consensus.cc:2243] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.559180 19967 raft_consensus.cc:2272] T 3dda35f1227549ff933e504aadd37233 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.562534 19967 tablet_replica.cc:333] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.563405 19967 raft_consensus.cc:2243] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.564108 19967 raft_consensus.cc:2272] T 54a4bfa356274b21a59d280e9ac8b477 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:19.565645 32188 consensus_peers.cc:597] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c -> Peer ee90f5b2a1e849739b82a66fdce52a8b (127.19.127.193:43055): Couldn't send request to peer ee90f5b2a1e849739b82a66fdce52a8b. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:43055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:19.566905 19967 tablet_replica.cc:333] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.567565 19967 raft_consensus.cc:2243] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.568429 19967 raft_consensus.cc:2272] T cc00c7f1fec24b569dd708c0b47e1ae7 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.570501 19967 tablet_replica.cc:333] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.570992 19967 raft_consensus.cc:2243] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.571513 19967 raft_consensus.cc:2272] T 981b55af3f1041619f509a52299dab01 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.573696 19967 tablet_replica.cc:333] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.574182 19967 raft_consensus.cc:2243] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.575065 19967 raft_consensus.cc:2272] T f604db35eda34a43997c049f9d49baa3 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.577421 19967 tablet_replica.cc:333] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.577958 19967 raft_consensus.cc:2243] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.578385 19967 raft_consensus.cc:2272] T 93f3937eb75d421abe285280d24d430d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.580658 19967 tablet_replica.cc:333] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.581132 19967 raft_consensus.cc:2243] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.581950 19967 raft_consensus.cc:2272] T f27ad9f1bb0349b4bb39afe329cf59d8 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.584344 19967 tablet_replica.cc:333] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.584800 19967 raft_consensus.cc:2243] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.585282 19967 raft_consensus.cc:2272] T c2b786f061514123aca8f3d1750e39d9 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.587749 19967 tablet_replica.cc:333] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.588210 19967 raft_consensus.cc:2243] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.588984 19967 raft_consensus.cc:2272] T e291faba389e4a21b5bf01b3a0de914d P 1b650608729b4ad6a28fd1c7e3d30d1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.591109 19967 tablet_replica.cc:333] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c: stopping tablet replica
I20260709 13:11:19.591635 19967 raft_consensus.cc:2243] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:19.592132 19967 raft_consensus.cc:2272] T b28632f2919847d2b35e28a7baff3fa0 P 1b650608729b4ad6a28fd1c7e3d30d1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.634009 19967 tablet_server.cc:196] TabletServer@127.19.127.195:0 shutdown complete.
I20260709 13:11:19.673465 19967 master.cc:562] Master@127.19.127.254:33803 shutting down...
I20260709 13:11:19.698212 19967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:19.698875 19967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:19.699225 19967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 663e888d5ca24232b5e5009a4481a246: stopping tablet replica
I20260709 13:11:19.719677 19967 master.cc:584] Master@127.19.127.254:33803 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13059 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:11:19.776332 19967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.127.254:35923
I20260709 13:11:19.777561 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:19.783665 32550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:19.783816 32551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:19.784613 32553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:19.786760 19967 server_base.cc:1061] running on GCE node
I20260709 13:11:19.787791 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:19.787971 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:19.788161 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602679788128 us; error 0 us; skew 500 ppm
I20260709 13:11:19.788666 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:19.791013 19967 webserver.cc:533] Webserver started at http://127.19.127.254:36995/ using document root <none> and password file <none>
I20260709 13:11:19.791535 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:19.791728 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:19.792007 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:19.793161 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/master-0-root/instance:
uuid: "9289fd0c13be4e8c93ccc54aea6c3918"
format_stamp: "Formatted at 2026-07-09 13:11:19 on dist-test-slave-s3n6"
I20260709 13:11:19.797845 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:11:19.801486 32558 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:19.802470 19967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:11:19.802867 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/master-0-root
uuid: "9289fd0c13be4e8c93ccc54aea6c3918"
format_stamp: "Formatted at 2026-07-09 13:11:19 on dist-test-slave-s3n6"
I20260709 13:11:19.803161 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:19.819798 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:19.821051 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:19.861138 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.254:35923
I20260709 13:11:19.861276 32609 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.254:35923 every 8 connection(s)
I20260709 13:11:19.865258 32610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.877187 32610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918: Bootstrap starting.
I20260709 13:11:19.883116 32610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.888120 32610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918: No bootstrap required, opened a new log
I20260709 13:11:19.890316 32610 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9289fd0c13be4e8c93ccc54aea6c3918" member_type: VOTER }
I20260709 13:11:19.890739 32610 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.890930 32610 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9289fd0c13be4e8c93ccc54aea6c3918, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.891518 32610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9289fd0c13be4e8c93ccc54aea6c3918" member_type: VOTER }
I20260709 13:11:19.891953 32610 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:11:19.892143 32610 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:11:19.892364 32610 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:19.897329 32610 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9289fd0c13be4e8c93ccc54aea6c3918" member_type: VOTER }
I20260709 13:11:19.897835 32610 leader_election.cc:304] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9289fd0c13be4e8c93ccc54aea6c3918; no voters: 
I20260709 13:11:19.898931 32610 leader_election.cc:290] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:11:19.899487 32613 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:19.901038 32613 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 1 LEADER]: Becoming Leader. State: Replica: 9289fd0c13be4e8c93ccc54aea6c3918, State: Running, Role: LEADER
I20260709 13:11:19.901741 32613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9289fd0c13be4e8c93ccc54aea6c3918" member_type: VOTER }
I20260709 13:11:19.901986 32610 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:11:19.907542 32614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9289fd0c13be4e8c93ccc54aea6c3918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9289fd0c13be4e8c93ccc54aea6c3918" member_type: VOTER } }
I20260709 13:11:19.907919 32615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9289fd0c13be4e8c93ccc54aea6c3918. Latest consensus state: current_term: 1 leader_uuid: "9289fd0c13be4e8c93ccc54aea6c3918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9289fd0c13be4e8c93ccc54aea6c3918" member_type: VOTER } }
I20260709 13:11:19.908838 32615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:19.908361 32614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:19.912804 32622 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:11:19.919900 32622 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:11:19.921034 19967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:11:19.929756 32622 catalog_manager.cc:1383] Generated new cluster ID: aa1994d93bad4c7195d41709fd1485ea
I20260709 13:11:19.930102 32622 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:11:19.945566 32622 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:11:19.946918 32622 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:11:19.961225 32622 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918: Generated new TSK 0
I20260709 13:11:19.961898 32622 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:11:19.987883 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:19.994643 32631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:19.995776 32632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:19.997977 32634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:19.998515 19967 server_base.cc:1061] running on GCE node
I20260709 13:11:19.999572 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:19.999843 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:20.000030 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602680000008 us; error 0 us; skew 500 ppm
I20260709 13:11:20.000576 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:20.003232 19967 webserver.cc:533] Webserver started at http://127.19.127.193:45057/ using document root <none> and password file <none>
I20260709 13:11:20.003852 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:20.004102 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:20.004325 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:20.005476 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-0-root/instance:
uuid: "6f399f3b38144952b3b85bfa93c7798d"
format_stamp: "Formatted at 2026-07-09 13:11:20 on dist-test-slave-s3n6"
I20260709 13:11:20.010124 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:11:20.014008 32639 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:20.014858 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:11:20.015131 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-0-root
uuid: "6f399f3b38144952b3b85bfa93c7798d"
format_stamp: "Formatted at 2026-07-09 13:11:20 on dist-test-slave-s3n6"
I20260709 13:11:20.015477 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:20.027757 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:20.028980 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:20.030433 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:20.032867 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:20.033046 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:20.033354 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:20.033535 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:20.077186 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.193:37431
I20260709 13:11:20.077304 32701 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.193:37431 every 8 connection(s)
I20260709 13:11:20.082356 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:20.090404 32706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:20.092456 32707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:20.094224 32709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:20.095906 19967 server_base.cc:1061] running on GCE node
I20260709 13:11:20.097503 32702 heartbeater.cc:344] Connected to a master server at 127.19.127.254:35923
I20260709 13:11:20.097880 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:11:20.097993 32702 heartbeater.cc:461] Registering TS with master...
W20260709 13:11:20.098242 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:20.098448 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602680098432 us; error 0 us; skew 500 ppm
I20260709 13:11:20.098953 32702 heartbeater.cc:507] Master 127.19.127.254:35923 requested a full tablet report, sending...
I20260709 13:11:20.099309 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:20.101583 32575 ts_manager.cc:194] Registered new tserver with Master: 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431)
I20260709 13:11:20.102519 19967 webserver.cc:533] Webserver started at http://127.19.127.194:43771/ using document root <none> and password file <none>
I20260709 13:11:20.103252 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:20.103572 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:20.104024 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:20.104319 32575 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33360
I20260709 13:11:20.105847 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-1-root/instance:
uuid: "a1b4f81a18fe44738b1e84db1a960118"
format_stamp: "Formatted at 2026-07-09 13:11:20 on dist-test-slave-s3n6"
I20260709 13:11:20.110981 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:11:20.114744 32714 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:20.115743 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:11:20.116081 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-1-root
uuid: "a1b4f81a18fe44738b1e84db1a960118"
format_stamp: "Formatted at 2026-07-09 13:11:20 on dist-test-slave-s3n6"
I20260709 13:11:20.116375 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:20.134649 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:20.135989 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:20.137514 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:20.139863 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:20.140086 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:20.140343 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:20.140518 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:20.181797 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.194:39419
I20260709 13:11:20.181924   308 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.194:39419 every 8 connection(s)
I20260709 13:11:20.186744 19967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:20.194252   312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:20.195425   313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:20.199249   309 heartbeater.cc:344] Connected to a master server at 127.19.127.254:35923
W20260709 13:11:20.199791   315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:20.199855 19967 server_base.cc:1061] running on GCE node
I20260709 13:11:20.199882   309 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:20.201247   309 heartbeater.cc:507] Master 127.19.127.254:35923 requested a full tablet report, sending...
I20260709 13:11:20.201349 19967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:20.201712 19967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:20.201906 19967 hybrid_clock.cc:648] HybridClock initialized: now 1783602680201884 us; error 0 us; skew 500 ppm
I20260709 13:11:20.202540 19967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:20.203785 32575 ts_manager.cc:194] Registered new tserver with Master: a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:20.205850 19967 webserver.cc:533] Webserver started at http://127.19.127.195:41753/ using document root <none> and password file <none>
I20260709 13:11:20.205983 32575 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33374
I20260709 13:11:20.206664 19967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:20.206950 19967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:20.207329 19967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:20.208622 19967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-2-root/instance:
uuid: "101b2cd1326246aea2ffd9866c9dbe42"
format_stamp: "Formatted at 2026-07-09 13:11:20 on dist-test-slave-s3n6"
I20260709 13:11:20.213567 19967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:11:20.217356   320 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:20.218315 19967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:11:20.218607 19967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-2-root
uuid: "101b2cd1326246aea2ffd9866c9dbe42"
format_stamp: "Formatted at 2026-07-09 13:11:20 on dist-test-slave-s3n6"
I20260709 13:11:20.218858 19967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task2exwQs/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602314247669-19967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:20.232657 19967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:20.233909 19967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:20.235472 19967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:20.237759 19967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:20.237952 19967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:20.238255 19967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:20.238396 19967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:20.280496 19967 rpc_server.cc:307] RPC server started. Bound to: 127.19.127.195:40123
I20260709 13:11:20.280611   382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.127.195:40123 every 8 connection(s)
I20260709 13:11:20.295245   383 heartbeater.cc:344] Connected to a master server at 127.19.127.254:35923
I20260709 13:11:20.295845   383 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:20.296706   383 heartbeater.cc:507] Master 127.19.127.254:35923 requested a full tablet report, sending...
I20260709 13:11:20.298986 32575 ts_manager.cc:194] Registered new tserver with Master: 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:20.300030 19967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016196161s
I20260709 13:11:20.300624 32575 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33386
I20260709 13:11:20.326746 32575 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33392:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:20.329622 32575 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:20.419775 32667 tablet_service.cc:1511] Processing CreateTablet for tablet 5e764d2123ed4dd38e5dabeb6cc4213e (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:20.421314 32667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e764d2123ed4dd38e5dabeb6cc4213e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.423085 32664 tablet_service.cc:1511] Processing CreateTablet for tablet fe2467e91cd54fa9b43a9fe739ecff76 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:20.424104 32666 tablet_service.cc:1511] Processing CreateTablet for tablet 0eb15a9d614142a288d7f82e7f7d470b (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:20.426090 32663 tablet_service.cc:1511] Processing CreateTablet for tablet 83a092096b9341099d87e2d7e187bcdf (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:20.427510 32663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a092096b9341099d87e2d7e187bcdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.421334 32665 tablet_service.cc:1511] Processing CreateTablet for tablet 8fcfc2016f2e4adbabdc926eba611c6f (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:20.430480 32665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fcfc2016f2e4adbabdc926eba611c6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.431066 32662 tablet_service.cc:1511] Processing CreateTablet for tablet 84e5964fd90e4a5aa0453e1d5e19b480 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:20.431660 32666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb15a9d614142a288d7f82e7f7d470b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.432353 32664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe2467e91cd54fa9b43a9fe739ecff76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.433547 32661 tablet_service.cc:1511] Processing CreateTablet for tablet 5ac46056ab9c41a88146e39bff533e4f (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:20.435051 32662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84e5964fd90e4a5aa0453e1d5e19b480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.437180   348 tablet_service.cc:1511] Processing CreateTablet for tablet 5e764d2123ed4dd38e5dabeb6cc4213e (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:20.438557   348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e764d2123ed4dd38e5dabeb6cc4213e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.440299   342 tablet_service.cc:1511] Processing CreateTablet for tablet 5ac46056ab9c41a88146e39bff533e4f (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:20.441744   342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac46056ab9c41a88146e39bff533e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.444559   347 tablet_service.cc:1511] Processing CreateTablet for tablet 0eb15a9d614142a288d7f82e7f7d470b (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:20.446089   347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb15a9d614142a288d7f82e7f7d470b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.446249 32660 tablet_service.cc:1511] Processing CreateTablet for tablet 6dbdaca625174743a36e9d8388b1cb90 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:20.448557   346 tablet_service.cc:1511] Processing CreateTablet for tablet 8fcfc2016f2e4adbabdc926eba611c6f (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:20.450064   346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fcfc2016f2e4adbabdc926eba611c6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.452099   345 tablet_service.cc:1511] Processing CreateTablet for tablet fe2467e91cd54fa9b43a9fe739ecff76 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:20.453688   345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe2467e91cd54fa9b43a9fe739ecff76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.454495 32659 tablet_service.cc:1511] Processing CreateTablet for tablet c02e9a4cbb294b84b80687bafc097470 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:20.456612   344 tablet_service.cc:1511] Processing CreateTablet for tablet 83a092096b9341099d87e2d7e187bcdf (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:20.458196   344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a092096b9341099d87e2d7e187bcdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.460855   340 tablet_service.cc:1511] Processing CreateTablet for tablet c02e9a4cbb294b84b80687bafc097470 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:20.460855   341 tablet_service.cc:1511] Processing CreateTablet for tablet 6dbdaca625174743a36e9d8388b1cb90 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:20.461958   343 tablet_service.cc:1511] Processing CreateTablet for tablet 84e5964fd90e4a5aa0453e1d5e19b480 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:20.462604   340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02e9a4cbb294b84b80687bafc097470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.463846   343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84e5964fd90e4a5aa0453e1d5e19b480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.464496   341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dbdaca625174743a36e9d8388b1cb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.482205 32661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac46056ab9c41a88146e39bff533e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.484761 32660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dbdaca625174743a36e9d8388b1cb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.487138 32659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02e9a4cbb294b84b80687bafc097470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.547528   399 tablet_bootstrap.cc:492] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:20.558254   399 tablet_bootstrap.cc:654] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.562162 32742 tablet_service.cc:1511] Processing CreateTablet for tablet 5e764d2123ed4dd38e5dabeb6cc4213e (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:20.567076 32738 tablet_service.cc:1511] Processing CreateTablet for tablet 83a092096b9341099d87e2d7e187bcdf (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:20.569516 32737 tablet_service.cc:1511] Processing CreateTablet for tablet 84e5964fd90e4a5aa0453e1d5e19b480 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:20.571727 32740 tablet_service.cc:1511] Processing CreateTablet for tablet 8fcfc2016f2e4adbabdc926eba611c6f (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:20.573990 32739 tablet_service.cc:1511] Processing CreateTablet for tablet fe2467e91cd54fa9b43a9fe739ecff76 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:20.564910 32741 tablet_service.cc:1511] Processing CreateTablet for tablet 0eb15a9d614142a288d7f82e7f7d470b (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:20.576611 32742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e764d2123ed4dd38e5dabeb6cc4213e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.582284 32737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84e5964fd90e4a5aa0453e1d5e19b480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.586612 32739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe2467e91cd54fa9b43a9fe739ecff76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.592257 32738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a092096b9341099d87e2d7e187bcdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.597034 32740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fcfc2016f2e4adbabdc926eba611c6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.604907 32741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb15a9d614142a288d7f82e7f7d470b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.609050 32736 tablet_service.cc:1511] Processing CreateTablet for tablet 5ac46056ab9c41a88146e39bff533e4f (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:20.610498 32736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac46056ab9c41a88146e39bff533e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.613150 32735 tablet_service.cc:1511] Processing CreateTablet for tablet 6dbdaca625174743a36e9d8388b1cb90 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:20.614670 32735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dbdaca625174743a36e9d8388b1cb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.617174 32734 tablet_service.cc:1511] Processing CreateTablet for tablet c02e9a4cbb294b84b80687bafc097470 (DEFAULT_TABLE table=test-workload [id=74b8d007763c48248302883fb2005221]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:20.618588 32734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02e9a4cbb294b84b80687bafc097470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.644541   401 tablet_bootstrap.cc:492] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:20.662549   401 tablet_bootstrap.cc:654] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.714911   399 tablet_bootstrap.cc:492] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:20.715464   399 ts_tablet_manager.cc:1403] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.168s	user 0.052s	sys 0.059s
I20260709 13:11:20.725628   399 raft_consensus.cc:359] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.726428   399 raft_consensus.cc:385] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.726725   399 raft_consensus.cc:740] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.727836   399 consensus_queue.cc:260] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.740790   398 tablet_bootstrap.cc:492] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:20.749085   399 ts_tablet_manager.cc:1434] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 13:11:20.749855   383 heartbeater.cc:499] Master 127.19.127.254:35923 was elected leader, sending a full tablet report...
I20260709 13:11:20.750291   401 tablet_bootstrap.cc:492] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:20.750339   399 tablet_bootstrap.cc:492] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:20.751042   401 ts_tablet_manager.cc:1403] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.107s	user 0.028s	sys 0.030s
I20260709 13:11:20.751178   398 tablet_bootstrap.cc:654] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.754768   401 raft_consensus.cc:359] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.755695   401 raft_consensus.cc:385] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.756042   401 raft_consensus.cc:740] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.757112   401 consensus_queue.cc:260] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.758652   399 tablet_bootstrap.cc:654] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.764783   398 tablet_bootstrap.cc:492] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:20.765159   401 ts_tablet_manager.cc:1434] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.006s
I20260709 13:11:20.765413   398 ts_tablet_manager.cc:1403] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 13:11:20.766135   309 heartbeater.cc:499] Master 127.19.127.254:35923 was elected leader, sending a full tablet report...
I20260709 13:11:20.766528   401 tablet_bootstrap.cc:492] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:20.769603   398 raft_consensus.cc:359] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.770632   398 raft_consensus.cc:385] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.771032   398 raft_consensus.cc:740] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.774978   401 tablet_bootstrap.cc:654] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.775468   398 consensus_queue.cc:260] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.780414   399 tablet_bootstrap.cc:492] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:20.781268   399 ts_tablet_manager.cc:1403] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.012s
I20260709 13:11:20.784987   399 raft_consensus.cc:359] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.785409   398 ts_tablet_manager.cc:1434] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.008s
I20260709 13:11:20.785771   399 raft_consensus.cc:385] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.786125   399 raft_consensus.cc:740] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.786535 32702 heartbeater.cc:499] Master 127.19.127.254:35923 was elected leader, sending a full tablet report...
I20260709 13:11:20.787137   399 consensus_queue.cc:260] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.791050   399 ts_tablet_manager.cc:1434] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:11:20.791870   398 tablet_bootstrap.cc:492] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:20.792173   399 tablet_bootstrap.cc:492] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:20.798344   401 tablet_bootstrap.cc:492] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:20.799016   401 ts_tablet_manager.cc:1403] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.033s	user 0.019s	sys 0.003s
I20260709 13:11:20.801035   398 tablet_bootstrap.cc:654] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.802412   399 tablet_bootstrap.cc:654] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.802393   401 raft_consensus.cc:359] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.803464   401 raft_consensus.cc:385] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.803807   401 raft_consensus.cc:740] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.804752   401 consensus_queue.cc:260] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.807142   401 ts_tablet_manager.cc:1434] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:11:20.808219   401 tablet_bootstrap.cc:492] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:20.815613   401 tablet_bootstrap.cc:654] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.816198   399 tablet_bootstrap.cc:492] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:20.816256   398 tablet_bootstrap.cc:492] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:20.817097   399 ts_tablet_manager.cc:1403] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.003s
I20260709 13:11:20.817256   398 ts_tablet_manager.cc:1403] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.009s
I20260709 13:11:20.820709   399 raft_consensus.cc:359] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.820983   398 raft_consensus.cc:359] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } }
I20260709 13:11:20.821683   399 raft_consensus.cc:385] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.821846   398 raft_consensus.cc:385] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.822060   399 raft_consensus.cc:740] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.822297   398 raft_consensus.cc:740] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.823041   399 consensus_queue.cc:260] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.823367   398 consensus_queue.cc:260] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } }
I20260709 13:11:20.826014   398 ts_tablet_manager.cc:1434] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:11:20.827072   398 tablet_bootstrap.cc:492] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:20.828696   399 ts_tablet_manager.cc:1434] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:11:20.829820   399 tablet_bootstrap.cc:492] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:20.835264   398 tablet_bootstrap.cc:654] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.836844   399 tablet_bootstrap.cc:654] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.839495   403 raft_consensus.cc:493] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.840247   403 raft_consensus.cc:515] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.853358   403 leader_election.cc:290] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:20.855218   401 tablet_bootstrap.cc:492] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:20.855934   401 ts_tablet_manager.cc:1403] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.048s	user 0.024s	sys 0.021s
I20260709 13:11:20.859654   401 raft_consensus.cc:359] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.860716   401 raft_consensus.cc:385] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.861085   401 raft_consensus.cc:740] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.862041   401 consensus_queue.cc:260] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.864580   401 ts_tablet_manager.cc:1434] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:11:20.859522   398 tablet_bootstrap.cc:492] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:20.865984   401 tablet_bootstrap.cc:492] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:20.866729   398 ts_tablet_manager.cc:1403] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.040s	user 0.005s	sys 0.019s
I20260709 13:11:20.870210   399 tablet_bootstrap.cc:492] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:20.870811   398 raft_consensus.cc:359] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.877374   398 raft_consensus.cc:385] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.877713   398 raft_consensus.cc:740] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.878587   398 consensus_queue.cc:260] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.870838   399 ts_tablet_manager.cc:1403] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.001s
I20260709 13:11:20.883358 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:20.884227 32752 raft_consensus.cc:2468] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:20.884367   399 raft_consensus.cc:359] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.885142   399 raft_consensus.cc:385] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.885453   399 raft_consensus.cc:740] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.885944   322 leader_election.cc:304] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:20.886260   399 consensus_queue.cc:260] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.886885   403 raft_consensus.cc:2804] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.887814   403 raft_consensus.cc:493] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.888270   403 raft_consensus.cc:3060] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.889644   399 ts_tablet_manager.cc:1434] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:11:20.890808   399 tablet_bootstrap.cc:492] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:20.894383 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:20.889819   398 ts_tablet_manager.cc:1434] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:11:20.894891   407 raft_consensus.cc:493] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.895648   407 raft_consensus.cc:515] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } }
W20260709 13:11:20.896472   324 leader_election.cc:343] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.897061   398 tablet_bootstrap.cc:492] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:20.897912   401 tablet_bootstrap.cc:654] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.900229   407 leader_election.cc:290] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:20.894342   403 raft_consensus.cc:515] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.908607   398 tablet_bootstrap.cc:654] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.908607   399 tablet_bootstrap.cc:654] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:20.914760 32623 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:20.919669   401 tablet_bootstrap.cc:492] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:20.920336   401 ts_tablet_manager.cc:1403] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.055s	user 0.007s	sys 0.010s
I20260709 13:11:20.921813 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
W20260709 13:11:20.923516   324 leader_election.cc:343] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.925420   403 leader_election.cc:290] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:20.925778 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:20.923517   401 raft_consensus.cc:359] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.926757 32752 raft_consensus.cc:3060] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.926887   401 raft_consensus.cc:385] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.930981   401 raft_consensus.cc:740] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.931972   401 consensus_queue.cc:260] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.934885 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e764d2123ed4dd38e5dabeb6cc4213e" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
W20260709 13:11:20.943914 32641 leader_election.cc:343] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.947571   401 ts_tablet_manager.cc:1434] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.003s
I20260709 13:11:20.945799 32752 raft_consensus.cc:2468] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:20.948632   398 tablet_bootstrap.cc:492] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:20.949314   398 ts_tablet_manager.cc:1403] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.024s
I20260709 13:11:20.948652   399 tablet_bootstrap.cc:492] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:20.949978   399 ts_tablet_manager.cc:1403] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.059s	user 0.025s	sys 0.005s
I20260709 13:11:20.950659   322 leader_election.cc:304] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:20.952188   403 raft_consensus.cc:2804] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:20.952339   416 raft_consensus.cc:493] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.952428   398 raft_consensus.cc:359] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.953189   416 raft_consensus.cc:515] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.953613   398 raft_consensus.cc:385] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.954102   398 raft_consensus.cc:740] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.955672   416 leader_election.cc:290] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:20.955338   398 consensus_queue.cc:260] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.956918 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:20.956817   401 tablet_bootstrap.cc:492] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
W20260709 13:11:20.958353   322 leader_election.cc:343] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.958280   398 ts_tablet_manager.cc:1434] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:11:20.959620 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:20.960355   398 tablet_bootstrap.cc:492] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
W20260709 13:11:20.962221   324 leader_election.cc:343] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.962813   324 leader_election.cc:304] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42; no voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:20.966616   401 tablet_bootstrap.cc:654] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.966827   416 raft_consensus.cc:2749] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:20.968017   403 raft_consensus.cc:697] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:20.968464   398 tablet_bootstrap.cc:654] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.969070   403 consensus_queue.cc:237] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.975575   407 raft_consensus.cc:493] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.976320   407 raft_consensus.cc:515] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:20.980156   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e764d2123ed4dd38e5dabeb6cc4213e" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:20.980230 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
W20260709 13:11:20.983410 32641 leader_election.cc:343] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.985301   407 leader_election.cc:290] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:20.974821   399 raft_consensus.cc:359] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } }
I20260709 13:11:20.986490   399 raft_consensus.cc:385] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.986927   399 raft_consensus.cc:740] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.987196   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:20.988252   399 consensus_queue.cc:260] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } }
W20260709 13:11:20.989586 32640 leader_election.cc:343] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.989755   358 raft_consensus.cc:2468] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:20.990378 32640 leader_election.cc:304] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d; no voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:20.992187   407 raft_consensus.cc:2749] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:20.994918 32640 leader_election.cc:304] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:20.996464   407 raft_consensus.cc:2804] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.996840   407 raft_consensus.cc:493] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.997152   407 raft_consensus.cc:3060] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.999123   399 ts_tablet_manager.cc:1434] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.003s
I20260709 13:11:21.000783   399 tablet_bootstrap.cc:492] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:21.006560   401 tablet_bootstrap.cc:492] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:21.007380   401 ts_tablet_manager.cc:1403] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.051s	user 0.021s	sys 0.024s
I20260709 13:11:21.009758   399 tablet_bootstrap.cc:654] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.010823   401 raft_consensus.cc:359] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } }
I20260709 13:11:21.011828   401 raft_consensus.cc:385] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.012216   401 raft_consensus.cc:740] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.012925 32573 catalog_manager.cc:5697] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:21.013445   401 consensus_queue.cc:260] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } }
I20260709 13:11:21.017197   401 ts_tablet_manager.cc:1434] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:11:21.030184   401 tablet_bootstrap.cc:492] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:21.033684   407 raft_consensus.cc:515] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } }
I20260709 13:11:21.034355   398 tablet_bootstrap.cc:492] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:21.038039   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e764d2123ed4dd38e5dabeb6cc4213e" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.038403   398 ts_tablet_manager.cc:1403] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.078s	user 0.016s	sys 0.057s
I20260709 13:11:21.038734   358 raft_consensus.cc:3060] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.041944   398 raft_consensus.cc:359] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.042804   398 raft_consensus.cc:385] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.043124   398 raft_consensus.cc:740] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.044835   398 consensus_queue.cc:260] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.045955   401 tablet_bootstrap.cc:654] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.043004   403 raft_consensus.cc:493] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.046775   403 raft_consensus.cc:515] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.049270   358 raft_consensus.cc:2468] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:21.049338 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e764d2123ed4dd38e5dabeb6cc4213e" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:21.051470 32752 raft_consensus.cc:3060] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.052372   398 ts_tablet_manager.cc:1434] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.002s
I20260709 13:11:21.054034 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9a4cbb294b84b80687bafc097470" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:21.055627 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9a4cbb294b84b80687bafc097470" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
W20260709 13:11:21.055970   324 leader_election.cc:343] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:21.056420 32751 raft_consensus.cc:2468] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:21.057155   403 leader_election.cc:290] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:21.057673   322 leader_election.cc:304] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:21.050036   407 leader_election.cc:290] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Requested vote from peers 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:21.059037   403 raft_consensus.cc:2804] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:21.059644   407 raft_consensus.cc:493] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.059778   398 tablet_bootstrap.cc:492] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:21.062996 32752 raft_consensus.cc:2468] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:21.064703 32641 leader_election.cc:304] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.065088   407 raft_consensus.cc:515] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.059887   403 raft_consensus.cc:493] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.066058   403 raft_consensus.cc:3060] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.066110   428 raft_consensus.cc:2804] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:21.066888   428 raft_consensus.cc:697] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:21.067329   398 tablet_bootstrap.cc:654] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.067726   407 leader_election.cc:290] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.068889   428 consensus_queue.cc:237] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } }
I20260709 13:11:21.064751   399 tablet_bootstrap.cc:492] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:21.071063   399 ts_tablet_manager.cc:1403] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.071s	user 0.025s	sys 0.018s
I20260709 13:11:21.070204 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e5964fd90e4a5aa0453e1d5e19b480" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:21.074952 32752 raft_consensus.cc:2468] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:21.077669 32641 leader_election.cc:304] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.075071   399 raft_consensus.cc:359] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.078871   399 raft_consensus.cc:385] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.079314   399 raft_consensus.cc:740] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.079927   407 raft_consensus.cc:2804] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:21.080330   399 consensus_queue.cc:260] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.083066   399 ts_tablet_manager.cc:1434] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:11:21.084153   399 tablet_bootstrap.cc:492] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:21.084635   403 raft_consensus.cc:515] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.080366   407 raft_consensus.cc:493] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.087877   407 raft_consensus.cc:3060] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.087927   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e5964fd90e4a5aa0453e1d5e19b480" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:21.088631   358 raft_consensus.cc:2468] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:21.088511 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9a4cbb294b84b80687bafc097470" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
W20260709 13:11:21.090615   324 leader_election.cc:343] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:21.091802   403 leader_election.cc:290] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:21.093142 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9a4cbb294b84b80687bafc097470" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:21.094081 32751 raft_consensus.cc:3060] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.096068   399 tablet_bootstrap.cc:654] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.102422   401 tablet_bootstrap.cc:492] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:21.103163   401 ts_tablet_manager.cc:1403] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.073s	user 0.015s	sys 0.002s
I20260709 13:11:21.111708   430 raft_consensus.cc:493] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.112741   430 raft_consensus.cc:515] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.115530   398 tablet_bootstrap.cc:492] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:21.116351   398 ts_tablet_manager.cc:1403] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.057s	user 0.029s	sys 0.004s
I20260709 13:11:21.117031 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a092096b9341099d87e2d7e187bcdf" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
W20260709 13:11:21.119488 32641 leader_election.cc:343] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:21.121366   398 raft_consensus.cc:359] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.122509   398 raft_consensus.cc:385] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.122937   398 raft_consensus.cc:740] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.125910   401 raft_consensus.cc:359] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.126551   401 raft_consensus.cc:385] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.126821   401 raft_consensus.cc:740] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.127803   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a092096b9341099d87e2d7e187bcdf" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:21.127589   401 consensus_queue.cc:260] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.129329 32751 raft_consensus.cc:2468] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
W20260709 13:11:21.129329 32640 leader_election.cc:343] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:21.129431   430 leader_election.cc:290] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.131875   322 leader_election.cc:304] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:21.132822   401 ts_tablet_manager.cc:1434] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:11:21.130107 32640 leader_election.cc:304] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d; no voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:21.133191   403 raft_consensus.cc:2804] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:21.133589   430 raft_consensus.cc:2749] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:21.133764   403 raft_consensus.cc:697] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:21.133929   401 tablet_bootstrap.cc:492] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:21.134093   407 raft_consensus.cc:515] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.135149   403 consensus_queue.cc:237] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.138172   398 consensus_queue.cc:260] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.138749   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e5964fd90e4a5aa0453e1d5e19b480" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.139432   358 raft_consensus.cc:3060] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.143235 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e5964fd90e4a5aa0453e1d5e19b480" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:21.144024 32751 raft_consensus.cc:3060] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.146173   401 tablet_bootstrap.cc:654] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.141777   399 tablet_bootstrap.cc:492] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:21.149852   399 ts_tablet_manager.cc:1403] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.013s
I20260709 13:11:21.141968   407 leader_election.cc:290] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.148916 32574 catalog_manager.cc:5697] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 1 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.155126   358 raft_consensus.cc:2468] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:21.157081 32640 leader_election.cc:304] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:21.158216   407 raft_consensus.cc:2804] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:21.159143   407 raft_consensus.cc:697] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:21.158380 32573 catalog_manager.cc:5697] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:21.159722   398 ts_tablet_manager.cc:1434] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 13:11:21.161072   398 tablet_bootstrap.cc:492] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:21.162261   407 consensus_queue.cc:237] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.154636   399 raft_consensus.cc:359] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.167510   398 tablet_bootstrap.cc:654] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.167155 32751 raft_consensus.cc:2468] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:21.170766   399 raft_consensus.cc:385] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.171169   399 raft_consensus.cc:740] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.172266   399 consensus_queue.cc:260] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.175928   399 ts_tablet_manager.cc:1434] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:11:21.177448   399 tablet_bootstrap.cc:492] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:21.186537   401 tablet_bootstrap.cc:492] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:21.186745   399 tablet_bootstrap.cc:654] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.187251   401 ts_tablet_manager.cc:1403] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.004s
I20260709 13:11:21.196614   401 raft_consensus.cc:359] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.198012   401 raft_consensus.cc:385] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.198483   401 raft_consensus.cc:740] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.199163   399 tablet_bootstrap.cc:492] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:21.199478   398 tablet_bootstrap.cc:492] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:21.199855   399 ts_tablet_manager.cc:1403] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 13:11:21.200038   398 ts_tablet_manager.cc:1403] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.015s
I20260709 13:11:21.199795   401 consensus_queue.cc:260] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.203437   399 raft_consensus.cc:359] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.203634   398 raft_consensus.cc:359] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.204264   399 raft_consensus.cc:385] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.204420   398 raft_consensus.cc:385] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.204640   399 raft_consensus.cc:740] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.204821   398 raft_consensus.cc:740] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.205613   399 consensus_queue.cc:260] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.205613   398 consensus_queue.cc:260] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.208118   398 ts_tablet_manager.cc:1434] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:11:21.208523   401 ts_tablet_manager.cc:1434] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.021s	user 0.002s	sys 0.006s
I20260709 13:11:21.209141   398 tablet_bootstrap.cc:492] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:21.209668   401 tablet_bootstrap.cc:492] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:21.209967 32573 catalog_manager.cc:5697] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 1 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.215613   399 ts_tablet_manager.cc:1434] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:11:21.216439   398 tablet_bootstrap.cc:654] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.216964   401 tablet_bootstrap.cc:654] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.227866   398 tablet_bootstrap.cc:492] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:21.228376   398 ts_tablet_manager.cc:1403] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:11:21.228683   399 tablet_bootstrap.cc:492] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:21.230986   398 raft_consensus.cc:359] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.231736   398 raft_consensus.cc:385] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.232023   398 raft_consensus.cc:740] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.232200   407 raft_consensus.cc:493] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.232784   407 raft_consensus.cc:515] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.232939   398 consensus_queue.cc:260] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.233566   401 tablet_bootstrap.cc:492] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:21.234324   401 ts_tablet_manager.cc:1403] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
I20260709 13:11:21.235071   407 leader_election.cc:290] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.235169   399 tablet_bootstrap.cc:654] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.238025 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:21.237185   401 raft_consensus.cc:359] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.241714   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:21.247140   401 raft_consensus.cc:385] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.235318   398 ts_tablet_manager.cc:1434] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:21.237633   403 raft_consensus.cc:493] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.247493   401 raft_consensus.cc:740] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.248077   403 raft_consensus.cc:515] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.248644   401 consensus_queue.cc:260] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.250458   403 leader_election.cc:290] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:21.251688   398 tablet_bootstrap.cc:492] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:21.251590 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac46056ab9c41a88146e39bff533e4f" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:21.252252 32677 raft_consensus.cc:2468] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:21.254285   401 ts_tablet_manager.cc:1434] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.009s
I20260709 13:11:21.255448   401 tablet_bootstrap.cc:492] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:21.257622 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac46056ab9c41a88146e39bff533e4f" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:21.258309   398 tablet_bootstrap.cc:654] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.258423 32752 raft_consensus.cc:2468] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:21.261554   401 tablet_bootstrap.cc:654] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:21.264514   322 leader_election.cc:304] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.247400 32751 raft_consensus.cc:2393] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6f399f3b38144952b3b85bfa93c7798d in current term 1: Already voted for candidate 101b2cd1326246aea2ffd9866c9dbe42 in this term.
I20260709 13:11:21.265555   403 raft_consensus.cc:2804] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:21.265976   403 raft_consensus.cc:493] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.266417 32641 leader_election.cc:304] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d; no voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:21.267694   407 raft_consensus.cc:3060] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.266463   403 raft_consensus.cc:3060] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.274973   399 tablet_bootstrap.cc:492] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:21.275629   399 ts_tablet_manager.cc:1403] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.007s
I20260709 13:11:21.275700   407 raft_consensus.cc:2749] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:21.275720   403 raft_consensus.cc:515] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.278731   398 tablet_bootstrap.cc:492] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:21.279021 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac46056ab9c41a88146e39bff533e4f" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:21.279383   398 ts_tablet_manager.cc:1403] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.003s
I20260709 13:11:21.279711 32677 raft_consensus.cc:3060] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.279840   399 raft_consensus.cc:359] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.280664   399 raft_consensus.cc:385] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.280951   399 raft_consensus.cc:740] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.281807   399 consensus_queue.cc:260] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.282789   401 tablet_bootstrap.cc:492] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:21.282501   398 raft_consensus.cc:359] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.283332   401 ts_tablet_manager.cc:1403] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.004s
I20260709 13:11:21.283274   398 raft_consensus.cc:385] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.284382   398 raft_consensus.cc:740] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.285295   398 consensus_queue.cc:260] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.286278   401 raft_consensus.cc:359] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.287036   401 raft_consensus.cc:385] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:21.287467   401 raft_consensus.cc:740] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:21.288534 32677 raft_consensus.cc:2468] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:21.290097   324 leader_election.cc:304] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:21.291095   416 raft_consensus.cc:2804] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:21.291622   416 raft_consensus.cc:697] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:21.292341   398 ts_tablet_manager.cc:1434] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:11:21.292217   401 consensus_queue.cc:260] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.292724   416 consensus_queue.cc:237] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.294499   401 ts_tablet_manager.cc:1434] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:11:21.295594   399 ts_tablet_manager.cc:1434] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:11:21.299525   403 leader_election.cc:290] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:21.302616 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac46056ab9c41a88146e39bff533e4f" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:21.303409 32751 raft_consensus.cc:3060] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.303211 32572 catalog_manager.cc:5697] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:21.307899   405 raft_consensus.cc:493] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.308526   405 raft_consensus.cc:515] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.310633 32751 raft_consensus.cc:2468] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:21.311424   405 leader_election.cc:290] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.325500 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:21.326329 32677 raft_consensus.cc:2468] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 0.
I20260709 13:11:21.327514 32718 leader_election.cc:304] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.328251   405 raft_consensus.cc:2804] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:21.328645   405 raft_consensus.cc:493] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.328996   405 raft_consensus.cc:3060] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.331391   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:21.332154   358 raft_consensus.cc:2468] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 0.
I20260709 13:11:21.335919   405 raft_consensus.cc:515] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.337747   405 leader_election.cc:290] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.338644 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:21.338757   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.339511 32677 raft_consensus.cc:3060] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.339577   358 raft_consensus.cc:3060] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.346961   358 raft_consensus.cc:2468] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 1.
I20260709 13:11:21.346961 32677 raft_consensus.cc:2468] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 1.
I20260709 13:11:21.348605 32715 leader_election.cc:304] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.349392   405 raft_consensus.cc:2804] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:21.350283   405 raft_consensus.cc:697] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:21.350586   403 raft_consensus.cc:493] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.351166   403 raft_consensus.cc:515] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.351186   405 consensus_queue.cc:237] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.354082   403 leader_election.cc:290] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:21.354449 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:21.354663 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:21.355190 32677 raft_consensus.cc:2468] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:21.355233 32751 raft_consensus.cc:2468] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:21.356586   324 leader_election.cc:304] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:21.357735   403 raft_consensus.cc:2804] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:21.358162   403 raft_consensus.cc:493] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.358467   403 raft_consensus.cc:3060] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.361974 32573 catalog_manager.cc:5697] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.363090   416 consensus_queue.cc:1048] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:21.365180   403 raft_consensus.cc:515] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.367118   403 leader_election.cc:290] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:21.367980 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:21.368673 32677 raft_consensus.cc:3060] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.368587 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:21.369527 32752 raft_consensus.cc:3060] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.377229 32677 raft_consensus.cc:2468] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:21.377892   405 raft_consensus.cc:493] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.378571   324 leader_election.cc:304] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:21.378472   405 raft_consensus.cc:515] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.380488   422 raft_consensus.cc:2804] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:21.380990   422 raft_consensus.cc:697] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:21.381851   422 consensus_queue.cc:237] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.383015 32676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2467e91cd54fa9b43a9fe739ecff76" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:21.383849   403 consensus_queue.cc:1048] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:21.384035 32676 raft_consensus.cc:2468] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 0.
I20260709 13:11:21.385162   405 leader_election.cc:290] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.385802 32718 leader_election.cc:304] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.385749 32752 raft_consensus.cc:2468] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:21.387375   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2467e91cd54fa9b43a9fe739ecff76" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:21.388350   358 raft_consensus.cc:2468] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 0.
I20260709 13:11:21.387693   405 raft_consensus.cc:2804] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:21.389658   405 raft_consensus.cc:493] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.390138   405 raft_consensus.cc:3060] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.398181   405 raft_consensus.cc:515] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.402145 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2467e91cd54fa9b43a9fe739ecff76" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:21.401602 32573 catalog_manager.cc:5697] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:21.403117 32677 raft_consensus.cc:3060] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.412930   450 raft_consensus.cc:493] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.413789   450 raft_consensus.cc:515] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.416476   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2467e91cd54fa9b43a9fe739ecff76" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.417500   358 raft_consensus.cc:3060] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.426019 32677 raft_consensus.cc:2468] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 1.
I20260709 13:11:21.427454 32718 leader_election.cc:304] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.427863   358 raft_consensus.cc:2468] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 1.
I20260709 13:11:21.429780   405 leader_election.cc:290] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.430608   405 raft_consensus.cc:2804] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:21.431443   405 raft_consensus.cc:697] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:21.433041   450 leader_election.cc:290] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.435124   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a092096b9341099d87e2d7e187bcdf" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:21.436614   357 raft_consensus.cc:2468] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 0.
I20260709 13:11:21.432674   405 consensus_queue.cc:237] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.439178 32715 leader_election.cc:304] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.440261   458 raft_consensus.cc:2804] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:21.443310 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a092096b9341099d87e2d7e187bcdf" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:21.445089   458 raft_consensus.cc:493] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.445636 32677 raft_consensus.cc:2468] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 0.
I20260709 13:11:21.445721   458 raft_consensus.cc:3060] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.454197 32573 catalog_manager.cc:5697] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.457060   458 raft_consensus.cc:515] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.459574   458 leader_election.cc:290] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.461206   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a092096b9341099d87e2d7e187bcdf" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.462328   357 raft_consensus.cc:3060] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.464254 32676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a092096b9341099d87e2d7e187bcdf" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:21.464963 32676 raft_consensus.cc:3060] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.470980   357 raft_consensus.cc:2468] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 1.
I20260709 13:11:21.472060 32676 raft_consensus.cc:2468] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 1.
I20260709 13:11:21.472448 32715 leader_election.cc:304] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.473888   458 raft_consensus.cc:2804] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:21.474305   458 raft_consensus.cc:697] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:21.475128   458 consensus_queue.cc:237] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.482276 32573 catalog_manager.cc:5697] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 1 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.522881   407 consensus_queue.cc:1048] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:21.535562   428 consensus_queue.cc:1048] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:21.567557   357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b"
dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
mode: GRACEFUL
new_leader_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:37786
I20260709 13:11:21.568225   357 raft_consensus.cc:606] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Received request to transfer leadership to TS 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:21.582288 32751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
mode: GRACEFUL
new_leader_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:38806
I20260709 13:11:21.582891 32751 raft_consensus.cc:606] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Received request to transfer leadership to TS 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:21.589025 32751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe2467e91cd54fa9b43a9fe739ecff76"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
mode: GRACEFUL
new_leader_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:38806
I20260709 13:11:21.589612 32751 raft_consensus.cc:606] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Received request to transfer leadership to TS 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:21.607251 32676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84e5964fd90e4a5aa0453e1d5e19b480"
dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
mode: GRACEFUL
new_leader_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:47600
I20260709 13:11:21.607909 32676 raft_consensus.cc:606] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Received request to transfer leadership to TS a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:21.608657   428 consensus_queue.cc:1048] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:21.613883   357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ac46056ab9c41a88146e39bff533e4f"
dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
mode: GRACEFUL
new_leader_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:37786
I20260709 13:11:21.614446   357 raft_consensus.cc:606] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Received request to transfer leadership to TS a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:21.616506   407 raft_consensus.cc:993] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d: : Instructing follower a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:21.616958   407 raft_consensus.cc:1081] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Signalling peer a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:21.619642 32752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84e5964fd90e4a5aa0453e1d5e19b480"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:38794
I20260709 13:11:21.622097 32752 raft_consensus.cc:493] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:21.622530 32752 raft_consensus.cc:3060] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.628237   357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90"
dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
mode: GRACEFUL
new_leader_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:37786
I20260709 13:11:21.628077   428 consensus_queue.cc:1048] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:21.629101   357 raft_consensus.cc:606] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Received request to transfer leadership to TS a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:21.628986 32752 raft_consensus.cc:515] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.634219 32676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e5964fd90e4a5aa0453e1d5e19b480" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:21.635074 32676 raft_consensus.cc:3055] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:21.635703 32676 raft_consensus.cc:740] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:21.636854 32676 consensus_queue.cc:260] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.638207 32676 raft_consensus.cc:3060] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.642329   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e5964fd90e4a5aa0453e1d5e19b480" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.644291 32752 leader_election.cc:290] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.649932 32676 raft_consensus.cc:2468] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:21.651341 32718 leader_election.cc:304] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 101b2cd1326246aea2ffd9866c9dbe42
I20260709 13:11:21.652339   458 raft_consensus.cc:2804] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:21.652866   458 raft_consensus.cc:697] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:21.653688   458 consensus_queue.cc:237] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.660596 32573 catalog_manager.cc:5697] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 1 to 2, leader changed from 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193) to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.664821   422 consensus_queue.cc:1048] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:21.670529   428 raft_consensus.cc:493] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.671134   428 raft_consensus.cc:515] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.674499   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9a4cbb294b84b80687bafc097470" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:21.676468 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9a4cbb294b84b80687bafc097470" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:21.678013 32641 leader_election.cc:304] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d; no voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:21.678678   428 leader_election.cc:290] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.679435   407 raft_consensus.cc:2749] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:21.680248 32676 raft_consensus.cc:3060] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.684361   422 consensus_queue.cc:1048] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:21.690104   422 consensus_queue.cc:1048] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:21.692445   422 raft_consensus.cc:993] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42: : Instructing follower a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:21.693449   403 raft_consensus.cc:1081] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Signalling peer a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:21.699945   454 consensus_queue.cc:1048] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:21.700400 32752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ac46056ab9c41a88146e39bff533e4f"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:38788
I20260709 13:11:21.701198 32752 raft_consensus.cc:493] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:21.701689 32752 raft_consensus.cc:3060] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.709329 32752 raft_consensus.cc:515] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.713119 32752 leader_election.cc:290] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.713898 32676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac46056ab9c41a88146e39bff533e4f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:21.714591 32676 raft_consensus.cc:3060] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.714949   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac46056ab9c41a88146e39bff533e4f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.715684   357 raft_consensus.cc:3055] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:21.716058   357 raft_consensus.cc:740] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:21.716979   357 consensus_queue.cc:260] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.718873   357 raft_consensus.cc:3060] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.721837 32676 raft_consensus.cc:2468] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:21.723104 32718 leader_election.cc:304] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.723994   458 raft_consensus.cc:2804] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:21.724557   458 raft_consensus.cc:697] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:21.725615   357 raft_consensus.cc:2468] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:21.725479   458 consensus_queue.cc:237] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.733438 32573 catalog_manager.cc:5697] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 1 to 2, leader changed from 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195) to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.738849   458 consensus_queue.cc:1048] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:21.746295   405 raft_consensus.cc:993] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118: : Instructing follower 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:21.746817   405 raft_consensus.cc:1081] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Signalling peer 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:21.748692 32677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f"
dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:47586
I20260709 13:11:21.749461 32677 raft_consensus.cc:493] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:21.749882 32677 raft_consensus.cc:3060] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.754428   405 consensus_queue.cc:1048] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:21.758567 32677 raft_consensus.cc:515] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.764086 32677 leader_election.cc:290] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.764222 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:21.765003 32752 raft_consensus.cc:3055] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:21.765364 32752 raft_consensus.cc:740] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:21.765785   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.766433   357 raft_consensus.cc:3060] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.766218 32752 consensus_queue.cc:260] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.768039 32752 raft_consensus.cc:3060] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.774609   357 raft_consensus.cc:2468] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 2.
I20260709 13:11:21.775417 32752 raft_consensus.cc:2468] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 2.
I20260709 13:11:21.776106 32640 leader_election.cc:304] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:21.777032   407 raft_consensus.cc:2804] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:21.777545   407 raft_consensus.cc:697] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:21.778362   407 consensus_queue.cc:237] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.785221 32574 catalog_manager.cc:5697] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 1 to 2, leader changed from a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194) to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 2 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.856283   458 consensus_queue.cc:1048] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:21.867595   405 consensus_queue.cc:1048] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:21.875609   450 raft_consensus.cc:993] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118: : Instructing follower 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:21.876127   405 raft_consensus.cc:1081] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Signalling peer 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:21.876168   416 consensus_queue.cc:1048] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:21.877888 32676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe2467e91cd54fa9b43a9fe739ecff76"
dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:47586
I20260709 13:11:21.878443 32676 raft_consensus.cc:493] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:21.878757 32676 raft_consensus.cc:3060] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.883586   422 raft_consensus.cc:993] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42: : Instructing follower 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:21.884119   454 raft_consensus.cc:1081] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Signalling peer 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:21.886196 32676 raft_consensus.cc:515] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.886209 32675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b"
dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:47584
I20260709 13:11:21.887943 32675 raft_consensus.cc:493] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:21.888346 32675 raft_consensus.cc:3060] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.888379 32676 leader_election.cc:290] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.891943 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2467e91cd54fa9b43a9fe739ecff76" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:21.892545 32751 raft_consensus.cc:3055] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:21.892884 32751 raft_consensus.cc:740] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:21.893707 32751 consensus_queue.cc:260] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.893059   416 consensus_queue.cc:1048] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:21.895084 32751 raft_consensus.cc:3060] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.896121 32675 raft_consensus.cc:515] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.897327 32677 raft_consensus.cc:1240] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Rejecting Update request from peer 101b2cd1326246aea2ffd9866c9dbe42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:21.898437 32675 leader_election.cc:290] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.899312 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:21.901309   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb15a9d614142a288d7f82e7f7d470b" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.901674   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe2467e91cd54fa9b43a9fe739ecff76" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.902226   357 raft_consensus.cc:3060] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.901561   416 consensus_queue.cc:1059] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:11:21.904657   358 raft_consensus.cc:3055] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:21.905014   358 raft_consensus.cc:740] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:21.905627 32751 raft_consensus.cc:2468] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 2.
I20260709 13:11:21.905861   358 consensus_queue.cc:260] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.906796 32641 leader_election.cc:304] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.907704   407 raft_consensus.cc:2804] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:21.908195   407 raft_consensus.cc:697] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:21.909021   407 consensus_queue.cc:237] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.906985   358 raft_consensus.cc:3060] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.910606   357 raft_consensus.cc:2468] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 2.
I20260709 13:11:21.917649   358 raft_consensus.cc:2468] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 2.
I20260709 13:11:21.919489 32640 leader_election.cc:304] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:21.920680   407 raft_consensus.cc:2804] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:21.921391   407 raft_consensus.cc:697] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:21.922231   407 consensus_queue.cc:237] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.922027 32574 catalog_manager.cc:5697] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 1 to 2, leader changed from a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194) to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 2 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.931639   422 raft_consensus.cc:993] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42: : Instructing follower a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:21.932260   422 raft_consensus.cc:1081] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Signalling peer a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:21.934080 32750 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:38788
I20260709 13:11:21.934701 32750 raft_consensus.cc:493] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:21.935096 32750 raft_consensus.cc:3060] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.940701 32750 raft_consensus.cc:515] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.942107 32574 catalog_manager.cc:5697] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 1 to 2, leader changed from 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195) to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 2 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.943346 32750 leader_election.cc:290] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:21.943907 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:21.943943   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:21.944725 32677 raft_consensus.cc:3060] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.945041   358 raft_consensus.cc:3055] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:21.945384   358 raft_consensus.cc:740] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:21.946233   358 consensus_queue.cc:260] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.947582   358 raft_consensus.cc:3060] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:21.952616 32677 raft_consensus.cc:2468] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:21.954063 32718 leader_election.cc:304] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:21.954794   405 raft_consensus.cc:2804] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:21.955219   405 raft_consensus.cc:697] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:21.955400   358 raft_consensus.cc:2468] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:21.956094   405 consensus_queue.cc:237] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:21.962628 32574 catalog_manager.cc:5697] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 1 to 2, leader changed from 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195) to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:22.034992   405 consensus_queue.cc:1048] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:22.037850 32675 raft_consensus.cc:1275] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.040365   450 consensus_queue.cc:1048] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.049705   458 consensus_queue.cc:1048] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:22.055595   357 raft_consensus.cc:3060] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.067581   357 raft_consensus.cc:1275] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.069502   405 consensus_queue.cc:1048] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.138574 32677 raft_consensus.cc:1275] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.140024   405 consensus_queue.cc:1048] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.149077   357 raft_consensus.cc:1275] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.150885   473 consensus_queue.cc:1048] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.341104 32750 raft_consensus.cc:3060] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.346714 32750 raft_consensus.cc:1275] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.348176   481 consensus_queue.cc:1048] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.358294   357 raft_consensus.cc:1275] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.359688   482 consensus_queue.cc:1048] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.396420 32750 raft_consensus.cc:1275] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.397873   482 consensus_queue.cc:1048] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.400544   357 raft_consensus.cc:1275] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.401836   407 consensus_queue.cc:1048] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.420818 32677 raft_consensus.cc:1275] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.422452   473 consensus_queue.cc:1048] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.430826   357 raft_consensus.cc:1275] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.432439   473 consensus_queue.cc:1048] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.441001 32750 raft_consensus.cc:1275] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.442432   481 consensus_queue.cc:1048] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:22.456566   357 raft_consensus.cc:1275] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.457890   481 consensus_queue.cc:1048] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:24.669412 32573 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33046:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:24.671599 32573 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:24.720554 32666 tablet_service.cc:1511] Processing CreateTablet for tablet 251194ffdb254820a2e7e2074f5db6a6 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:24.721954 32666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251194ffdb254820a2e7e2074f5db6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.722239 32738 tablet_service.cc:1511] Processing CreateTablet for tablet 251194ffdb254820a2e7e2074f5db6a6 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:24.723639 32738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251194ffdb254820a2e7e2074f5db6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.727936 32740 tablet_service.cc:1511] Processing CreateTablet for tablet 32b61db997cb491ca829f0e77b0e8c14 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:24.728242 32736 tablet_service.cc:1511] Processing CreateTablet for tablet 3c6d66fe66a54a34946a3a966f1ed7b1 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:24.729445 32740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b61db997cb491ca829f0e77b0e8c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.730067 32736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c6d66fe66a54a34946a3a966f1ed7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.730906 32741 tablet_service.cc:1511] Processing CreateTablet for tablet 830dd51789be46d8a64f1e1bdf682a4a (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:24.731388   344 tablet_service.cc:1511] Processing CreateTablet for tablet 251194ffdb254820a2e7e2074f5db6a6 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:24.732319 32741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 830dd51789be46d8a64f1e1bdf682a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.732649   344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251194ffdb254820a2e7e2074f5db6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.735456 32742 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfec1932b8046b0be6e73f558feb46d (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:24.736881 32742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfec1932b8046b0be6e73f558feb46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.740460 32664 tablet_service.cc:1511] Processing CreateTablet for tablet 3c6d66fe66a54a34946a3a966f1ed7b1 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:24.741549 32665 tablet_service.cc:1511] Processing CreateTablet for tablet 32b61db997cb491ca829f0e77b0e8c14 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:24.742106 32664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c6d66fe66a54a34946a3a966f1ed7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.742974 32665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b61db997cb491ca829f0e77b0e8c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.744222 32660 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfec1932b8046b0be6e73f558feb46d (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:24.745294 32739 tablet_service.cc:1511] Processing CreateTablet for tablet 77ce7157862640f9a0078fdb6162c50f (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:24.746675 32739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ce7157862640f9a0078fdb6162c50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.748333   342 tablet_service.cc:1511] Processing CreateTablet for tablet 3c6d66fe66a54a34946a3a966f1ed7b1 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:24.749042 32734 tablet_service.cc:1511] Processing CreateTablet for tablet 01d473370c3a4b8c92694cf50174d4b6 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:24.749786   342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c6d66fe66a54a34946a3a966f1ed7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.750383 32734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d473370c3a4b8c92694cf50174d4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.744222 32659 tablet_service.cc:1511] Processing CreateTablet for tablet 830dd51789be46d8a64f1e1bdf682a4a (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:24.754582 32737 tablet_service.cc:1511] Processing CreateTablet for tablet 548b08bd3a09478eabeb9dbcf4a453f1 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:24.756109 32737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 548b08bd3a09478eabeb9dbcf4a453f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.759171 32735 tablet_service.cc:1511] Processing CreateTablet for tablet 8f31f60abc1e491780a4452665f85feb (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:24.759508   345 tablet_service.cc:1511] Processing CreateTablet for tablet 32b61db997cb491ca829f0e77b0e8c14 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:24.760757 32735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f31f60abc1e491780a4452665f85feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.761339   345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b61db997cb491ca829f0e77b0e8c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.762882   496 tablet_bootstrap.cc:492] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:24.765342 32661 tablet_service.cc:1511] Processing CreateTablet for tablet 77ce7157862640f9a0078fdb6162c50f (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:24.766911 32661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ce7157862640f9a0078fdb6162c50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.768440 32662 tablet_service.cc:1511] Processing CreateTablet for tablet 01d473370c3a4b8c92694cf50174d4b6 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:24.770886 32663 tablet_service.cc:1511] Processing CreateTablet for tablet 548b08bd3a09478eabeb9dbcf4a453f1 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:24.773032 32659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 830dd51789be46d8a64f1e1bdf682a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.773767 32663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 548b08bd3a09478eabeb9dbcf4a453f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.775331 32662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d473370c3a4b8c92694cf50174d4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.778203   496 tablet_bootstrap.cc:654] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:24.787817 32660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfec1932b8046b0be6e73f558feb46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.789389 32667 tablet_service.cc:1511] Processing CreateTablet for tablet 8f31f60abc1e491780a4452665f85feb (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:24.791205 32667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f31f60abc1e491780a4452665f85feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.801952   498 tablet_bootstrap.cc:492] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:24.808136   346 tablet_service.cc:1511] Processing CreateTablet for tablet 548b08bd3a09478eabeb9dbcf4a453f1 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:24.808369   347 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfec1932b8046b0be6e73f558feb46d (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:24.809665   346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 548b08bd3a09478eabeb9dbcf4a453f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.810163   343 tablet_service.cc:1511] Processing CreateTablet for tablet 01d473370c3a4b8c92694cf50174d4b6 (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:24.811214   348 tablet_service.cc:1511] Processing CreateTablet for tablet 77ce7157862640f9a0078fdb6162c50f (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:24.813020   340 tablet_service.cc:1511] Processing CreateTablet for tablet 8f31f60abc1e491780a4452665f85feb (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:24.814401   343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d473370c3a4b8c92694cf50174d4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.819171   347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfec1932b8046b0be6e73f558feb46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.823830   340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f31f60abc1e491780a4452665f85feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.827731   348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ce7157862640f9a0078fdb6162c50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.831949   345 tablet_service.cc:1511] Processing CreateTablet for tablet 830dd51789be46d8a64f1e1bdf682a4a (DEFAULT_TABLE table=soft_deleted_table [id=450652246e13439483f2fdf7f90276d1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:24.833700   345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 830dd51789be46d8a64f1e1bdf682a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.870249   498 tablet_bootstrap.cc:654] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:24.902225   496 tablet_bootstrap.cc:492] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:24.902899   496 ts_tablet_manager.cc:1403] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.140s	user 0.021s	sys 0.028s
I20260709 13:11:24.909488   496 raft_consensus.cc:359] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:24.910234   499 tablet_bootstrap.cc:492] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:24.910396   496 raft_consensus.cc:385] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:24.911094   496 raft_consensus.cc:740] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:24.912176   496 consensus_queue.cc:260] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:24.917989   499 tablet_bootstrap.cc:654] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:24.929734   496 ts_tablet_manager.cc:1434] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:11:24.931330   499 tablet_bootstrap.cc:492] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:24.932003   499 ts_tablet_manager.cc:1403] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:11:24.935206   499 raft_consensus.cc:359] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:24.938988   496 tablet_bootstrap.cc:492] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:24.943480   499 raft_consensus.cc:385] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:24.943964   499 raft_consensus.cc:740] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:24.944950   499 consensus_queue.cc:260] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:24.946350   496 tablet_bootstrap.cc:654] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:24.958969 32623 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:24.960371   499 ts_tablet_manager.cc:1434] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:11:24.962576   496 tablet_bootstrap.cc:492] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:24.962069   498 tablet_bootstrap.cc:492] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:24.963137   496 ts_tablet_manager.cc:1403] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.007s
I20260709 13:11:24.963213   498 ts_tablet_manager.cc:1403] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.162s	user 0.020s	sys 0.016s
I20260709 13:11:24.966434   496 raft_consensus.cc:359] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:24.967180   496 raft_consensus.cc:385] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:24.967525   496 raft_consensus.cc:740] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:24.968411   496 consensus_queue.cc:260] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:24.970032   499 tablet_bootstrap.cc:492] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:24.971442   498 raft_consensus.cc:359] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:24.972299   498 raft_consensus.cc:385] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:24.972623   498 raft_consensus.cc:740] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:24.973434   498 consensus_queue.cc:260] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:24.974120   496 ts_tablet_manager.cc:1434] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:11:24.975219   496 tablet_bootstrap.cc:492] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:24.976235   498 ts_tablet_manager.cc:1434] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:11:24.977449   498 tablet_bootstrap.cc:492] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:24.978973   499 tablet_bootstrap.cc:654] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:24.983242   496 tablet_bootstrap.cc:654] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:24.985157   498 tablet_bootstrap.cc:654] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:24.995450   473 raft_consensus.cc:493] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.996078   473 raft_consensus.cc:515] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:24.998831   473 leader_election.cc:290] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.000941   499 tablet_bootstrap.cc:492] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:25.001163 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c6d66fe66a54a34946a3a966f1ed7b1" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:25.001657   499 ts_tablet_manager.cc:1403] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.012s
W20260709 13:11:25.003750 32718 leader_election.cc:343] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.001782   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c6d66fe66a54a34946a3a966f1ed7b1" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.005393   499 raft_consensus.cc:359] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.006417   499 raft_consensus.cc:385] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.006717   499 raft_consensus.cc:740] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
W20260709 13:11:25.007175 32715 leader_election.cc:343] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.007790 32715 leader_election.cc:304] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1b4f81a18fe44738b1e84db1a960118; no voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.007727   499 consensus_queue.cc:260] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.008533   496 tablet_bootstrap.cc:492] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:25.008934   498 tablet_bootstrap.cc:492] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:25.009135   496 ts_tablet_manager.cc:1403] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.003s
I20260709 13:11:25.009102   473 raft_consensus.cc:2749] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:25.009470   498 ts_tablet_manager.cc:1403] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.005s
I20260709 13:11:25.012228   499 ts_tablet_manager.cc:1434] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:11:25.013303   499 tablet_bootstrap.cc:492] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:25.013109   496 raft_consensus.cc:359] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.013921   496 raft_consensus.cc:385] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.014240   496 raft_consensus.cc:740] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.014163   498 raft_consensus.cc:359] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.015381   498 raft_consensus.cc:385] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.016331   498 raft_consensus.cc:740] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.015219   496 consensus_queue.cc:260] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.017328   498 consensus_queue.cc:260] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.020366   498 ts_tablet_manager.cc:1434] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:11:25.021130   496 ts_tablet_manager.cc:1434] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:11:25.021549   498 tablet_bootstrap.cc:492] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:25.022507   496 tablet_bootstrap.cc:492] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:25.027423   499 tablet_bootstrap.cc:654] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.052556   496 tablet_bootstrap.cc:654] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.052805   498 tablet_bootstrap.cc:654] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.056298   499 tablet_bootstrap.cc:492] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:25.057018   499 ts_tablet_manager.cc:1403] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.023s
I20260709 13:11:25.061452   499 raft_consensus.cc:359] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.062404   499 raft_consensus.cc:385] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.062825   499 raft_consensus.cc:740] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.063541   496 tablet_bootstrap.cc:492] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:25.064147   496 ts_tablet_manager.cc:1403] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.003s
I20260709 13:11:25.063728   499 consensus_queue.cc:260] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.064612   498 tablet_bootstrap.cc:492] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:25.065182   498 ts_tablet_manager.cc:1403] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.029s
I20260709 13:11:25.066756   499 ts_tablet_manager.cc:1434] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:11:25.067231   496 raft_consensus.cc:359] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.068013   496 raft_consensus.cc:385] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.068333   496 raft_consensus.cc:740] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.068007   499 tablet_bootstrap.cc:492] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:25.069082   496 consensus_queue.cc:260] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.069309   498 raft_consensus.cc:359] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.071599   496 ts_tablet_manager.cc:1434] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:25.074347   498 raft_consensus.cc:385] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.074735   498 raft_consensus.cc:740] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.075502   496 tablet_bootstrap.cc:492] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:25.075659   498 consensus_queue.cc:260] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.078348   498 ts_tablet_manager.cc:1434] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:11:25.079404   498 tablet_bootstrap.cc:492] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:25.085011   496 tablet_bootstrap.cc:654] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.085901   498 tablet_bootstrap.cc:654] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.084926   499 tablet_bootstrap.cc:654] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.093185   496 tablet_bootstrap.cc:492] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:25.093343   498 tablet_bootstrap.cc:492] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:25.093700   496 ts_tablet_manager.cc:1403] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.000s
I20260709 13:11:25.093945   498 ts_tablet_manager.cc:1403] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:11:25.096546   496 raft_consensus.cc:359] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.097333   496 raft_consensus.cc:385] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.097285   498 raft_consensus.cc:359] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.097921   496 raft_consensus.cc:740] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.098325   498 raft_consensus.cc:385] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.098697   498 raft_consensus.cc:740] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.098948   496 consensus_queue.cc:260] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.099689   498 consensus_queue.cc:260] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.102358   498 ts_tablet_manager.cc:1434] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:11:25.102929   499 tablet_bootstrap.cc:492] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:25.103641   499 ts_tablet_manager.cc:1403] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.000s
I20260709 13:11:25.103636   498 tablet_bootstrap.cc:492] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:25.105970   496 ts_tablet_manager.cc:1434] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:11:25.107121   496 tablet_bootstrap.cc:492] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:25.107043   499 raft_consensus.cc:359] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.108114   499 raft_consensus.cc:385] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.108470   499 raft_consensus.cc:740] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.109279   499 consensus_queue.cc:260] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.111840   499 ts_tablet_manager.cc:1434] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:11:25.112394   498 tablet_bootstrap.cc:654] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.113057   499 tablet_bootstrap.cc:492] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:25.120541   499 tablet_bootstrap.cc:654] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.121016   496 tablet_bootstrap.cc:654] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.124172   428 raft_consensus.cc:493] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.124769   428 raft_consensus.cc:515] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.127081   428 leader_election.cc:290] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.128610   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.129022 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
W20260709 13:11:25.131129 32640 leader_election.cc:343] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:25.132022 32641 leader_election.cc:343] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.132620 32641 leader_election.cc:304] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d; no voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:25.134545   496 tablet_bootstrap.cc:492] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:25.135376   496 ts_tablet_manager.cc:1403] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.004s
I20260709 13:11:25.139503   428 raft_consensus.cc:2749] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:25.140969   473 raft_consensus.cc:493] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.141698   473 raft_consensus.cc:515] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.150826 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
W20260709 13:11:25.152913 32718 leader_election.cc:343] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.155920   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
W20260709 13:11:25.157972 32715 leader_election.cc:343] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.158615 32715 leader_election.cc:304] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1b4f81a18fe44738b1e84db1a960118; no voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.160259   473 leader_election.cc:290] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.160991   473 raft_consensus.cc:2749] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:25.140214   496 raft_consensus.cc:359] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.168155   498 tablet_bootstrap.cc:492] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:25.171720   496 raft_consensus.cc:385] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.172149   496 raft_consensus.cc:740] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.172190   498 ts_tablet_manager.cc:1403] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.069s	user 0.018s	sys 0.043s
I20260709 13:11:25.172996   496 consensus_queue.cc:260] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.173491   499 tablet_bootstrap.cc:492] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:25.174139   499 ts_tablet_manager.cc:1403] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.061s	user 0.034s	sys 0.024s
I20260709 13:11:25.175727   498 raft_consensus.cc:359] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.176486   498 raft_consensus.cc:385] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.176744   498 raft_consensus.cc:740] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.177515   498 consensus_queue.cc:260] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.177624   499 raft_consensus.cc:359] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.178546   496 ts_tablet_manager.cc:1434] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:11:25.178599   499 raft_consensus.cc:385] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.179232   499 raft_consensus.cc:740] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.179859   496 tablet_bootstrap.cc:492] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:25.179929   498 ts_tablet_manager.cc:1434] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:25.180162   499 consensus_queue.cc:260] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.182732   499 ts_tablet_manager.cc:1434] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:11:25.183866   499 tablet_bootstrap.cc:492] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:25.185633   498 tablet_bootstrap.cc:492] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:25.187163   496 tablet_bootstrap.cc:654] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.193118   498 tablet_bootstrap.cc:654] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.193471   516 raft_consensus.cc:493] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.194056   516 raft_consensus.cc:515] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.195509   499 tablet_bootstrap.cc:654] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.198637 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
W20260709 13:11:25.200971 32641 leader_election.cc:343] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.207947   496 tablet_bootstrap.cc:492] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:25.214323   496 ts_tablet_manager.cc:1403] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.003s
I20260709 13:11:25.222069   496 raft_consensus.cc:359] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.222569   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.226584   357 raft_consensus.cc:2468] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:25.222265   516 leader_election.cc:290] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.227633   496 raft_consensus.cc:385] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.227684   498 tablet_bootstrap.cc:492] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:25.227893   499 tablet_bootstrap.cc:492] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:25.227998   496 raft_consensus.cc:740] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.228325   498 ts_tablet_manager.cc:1403] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.024s
I20260709 13:11:25.228078 32640 leader_election.cc:304] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:25.228577   499 ts_tablet_manager.cc:1403] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.028s
I20260709 13:11:25.229184   516 raft_consensus.cc:2804] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:25.229562   516 raft_consensus.cc:493] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.229969   516 raft_consensus.cc:3060] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.231182   498 raft_consensus.cc:359] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.231989   496 consensus_queue.cc:260] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.231875   498 raft_consensus.cc:385] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.233049   498 raft_consensus.cc:740] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.233042   485 raft_consensus.cc:493] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.233713   485 raft_consensus.cc:515] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.233913   498 consensus_queue.cc:260] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.231843   499 raft_consensus.cc:359] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.234861   499 raft_consensus.cc:385] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.235162   499 raft_consensus.cc:740] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.235932   499 consensus_queue.cc:260] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.237039 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:25.238440   499 ts_tablet_manager.cc:1434] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:11:25.234467   496 ts_tablet_manager.cc:1434] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
W20260709 13:11:25.238900   324 leader_election.cc:343] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.240767   496 tablet_bootstrap.cc:492] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:25.240583   516 raft_consensus.cc:515] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.242972 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:25.243345   473 raft_consensus.cc:493] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.243724 32750 raft_consensus.cc:2468] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:25.244082   473 raft_consensus.cc:515] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.245229   322 leader_election.cc:304] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.246244   473 leader_election.cc:290] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.246341   485 leader_election.cc:290] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.248013   485 raft_consensus.cc:2804] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:25.248194   496 tablet_bootstrap.cc:654] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.249130   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.255698   485 raft_consensus.cc:493] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.256168   357 raft_consensus.cc:2468] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 0.
I20260709 13:11:25.250130 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:25.257927 32675 raft_consensus.cc:2393] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a1b4f81a18fe44738b1e84db1a960118 in current term 1: Already voted for candidate 6f399f3b38144952b3b85bfa93c7798d in this term.
I20260709 13:11:25.260356 32715 leader_election.cc:304] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.262562   473 raft_consensus.cc:2804] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:25.263172   473 raft_consensus.cc:493] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.263664   473 raft_consensus.cc:3060] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.251397   499 tablet_bootstrap.cc:492] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:25.252626   516 leader_election.cc:290] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.252661 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:25.254252   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:25.271603   358 raft_consensus.cc:3060] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.275033   521 raft_consensus.cc:493] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.256196   485 raft_consensus.cc:3060] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.278398   499 tablet_bootstrap.cc:654] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.275673   521 raft_consensus.cc:515] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.281553   358 raft_consensus.cc:2468] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:25.281911   515 raft_consensus.cc:493] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.284099   515 raft_consensus.cc:515] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.284878 32640 leader_election.cc:304] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:25.282546 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f31f60abc1e491780a4452665f85feb" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:25.288039 32675 raft_consensus.cc:2468] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:25.286586   516 raft_consensus.cc:2804] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:25.290064   324 leader_election.cc:304] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:25.290405 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f31f60abc1e491780a4452665f85feb" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:25.291342   515 leader_election.cc:290] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.291564   516 raft_consensus.cc:697] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
W20260709 13:11:25.292220   322 leader_election.cc:343] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.293467   516 consensus_queue.cc:237] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.295785   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251194ffdb254820a2e7e2074f5db6a6" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.297758   520 raft_consensus.cc:2804] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:11:25.298252 32715 leader_election.cc:343] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.299011   498 ts_tablet_manager.cc:1434] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.070s	user 0.019s	sys 0.041s
I20260709 13:11:25.294723   521 leader_election.cc:290] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.299253 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251194ffdb254820a2e7e2074f5db6a6" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:25.301541   498 tablet_bootstrap.cc:492] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
W20260709 13:11:25.302137 32718 leader_election.cc:343] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.304584 32718 leader_election.cc:304] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1b4f81a18fe44738b1e84db1a960118; no voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.305595   523 raft_consensus.cc:2749] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:25.307528   473 raft_consensus.cc:515] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.298292   520 raft_consensus.cc:493] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.310439 32750 raft_consensus.cc:2393] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f399f3b38144952b3b85bfa93c7798d in current term 1: Already voted for candidate a1b4f81a18fe44738b1e84db1a960118 in this term.
I20260709 13:11:25.314070   473 leader_election.cc:290] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.314971 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:25.315430   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:25.316463   358 raft_consensus.cc:2393] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a1b4f81a18fe44738b1e84db1a960118 in current term 1: Already voted for candidate 6f399f3b38144952b3b85bfa93c7798d in this term.
I20260709 13:11:25.310683   520 raft_consensus.cc:3060] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.318382   498 tablet_bootstrap.cc:654] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.320281 32718 leader_election.cc:304] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1b4f81a18fe44738b1e84db1a960118; no voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.325076   496 tablet_bootstrap.cc:492] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:25.321214   473 raft_consensus.cc:2749] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:25.327008   496 ts_tablet_manager.cc:1403] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.004s
I20260709 13:11:25.326155   520 raft_consensus.cc:515] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.328646   485 raft_consensus.cc:515] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.330722 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f31f60abc1e491780a4452665f85feb" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:25.331641 32675 raft_consensus.cc:3060] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.331736   520 leader_election.cc:290] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.334290 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f31f60abc1e491780a4452665f85feb" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:25.335177   498 tablet_bootstrap.cc:492] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
W20260709 13:11:25.335750   322 leader_election.cc:343] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.335759   498 ts_tablet_manager.cc:1403] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.018s
I20260709 13:11:25.336943   520 raft_consensus.cc:493] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.337213   496 raft_consensus.cc:359] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.337939   496 raft_consensus.cc:385] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.337548   520 raft_consensus.cc:515] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.338236   496 raft_consensus.cc:740] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.339179   498 raft_consensus.cc:359] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.339967   498 raft_consensus.cc:385] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.340292   498 raft_consensus.cc:740] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.340457 32675 raft_consensus.cc:2468] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:25.340915 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:25.341051   498 consensus_queue.cc:260] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.341775   324 leader_election.cc:304] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:25.343009   521 raft_consensus.cc:2804] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:25.344025   498 ts_tablet_manager.cc:1434] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:11:25.343866 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
W20260709 13:11:25.344661   324 leader_election.cc:343] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.344771 32750 raft_consensus.cc:2468] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:25.345219   498 tablet_bootstrap.cc:492] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:25.346251   322 leader_election.cc:304] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.351603   520 leader_election.cc:290] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.352998   520 raft_consensus.cc:2804] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:25.354131   520 raft_consensus.cc:493] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.322817   499 tablet_bootstrap.cc:492] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:25.357515   520 raft_consensus.cc:3060] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.357669   499 ts_tablet_manager.cc:1403] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.107s	user 0.013s	sys 0.000s
I20260709 13:11:25.338982   496 consensus_queue.cc:260] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.357923   521 raft_consensus.cc:697] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:25.359231   521 consensus_queue.cc:237] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.361452   498 tablet_bootstrap.cc:654] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.363791   496 ts_tablet_manager.cc:1434] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:11:25.361634   499 raft_consensus.cc:359] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.364557   499 raft_consensus.cc:385] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.364876   499 raft_consensus.cc:740] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.366380 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
W20260709 13:11:25.369297   324 leader_election.cc:343] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.371256   485 leader_election.cc:290] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.374379   496 tablet_bootstrap.cc:492] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118: Bootstrap starting.
I20260709 13:11:25.375399   499 consensus_queue.cc:260] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.373327 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:25.383001 32750 raft_consensus.cc:3060] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.388155 32573 catalog_manager.cc:5697] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 1 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:25.390708 32750 raft_consensus.cc:2468] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:25.392709   516 raft_consensus.cc:493] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.392910   499 ts_tablet_manager.cc:1434] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:11:25.394094   498 tablet_bootstrap.cc:492] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:25.406457   498 ts_tablet_manager.cc:1403] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.061s	user 0.016s	sys 0.025s
I20260709 13:11:25.406060   516 raft_consensus.cc:515] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.406570   322 leader_election.cc:304] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.406949   499 tablet_bootstrap.cc:492] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:25.408375   528 raft_consensus.cc:2804] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:25.409430   498 raft_consensus.cc:359] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.410261   498 raft_consensus.cc:385] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.409720   485 raft_consensus.cc:493] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.410823   498 raft_consensus.cc:740] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.411056   485 raft_consensus.cc:515] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.411618   498 consensus_queue.cc:260] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.412734   520 raft_consensus.cc:515] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.415474 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfec1932b8046b0be6e73f558feb46d" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:25.396773   496 tablet_bootstrap.cc:654] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.416003   527 raft_consensus.cc:493] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.416216 32751 raft_consensus.cc:2468] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:25.416805   527 raft_consensus.cc:515] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.417848   322 leader_election.cc:304] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:25.413625   485 leader_election.cc:290] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.409713   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d473370c3a4b8c92694cf50174d4b6" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.413889   498 ts_tablet_manager.cc:1434] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:25.417865 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d473370c3a4b8c92694cf50174d4b6" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:25.419060   358 raft_consensus.cc:2468] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:25.419226 32750 raft_consensus.cc:2468] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:25.419745   498 tablet_bootstrap.cc:492] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42: Bootstrap starting.
I20260709 13:11:25.419832 32675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfec1932b8046b0be6e73f558feb46d" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:25.421108   485 raft_consensus.cc:2804] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:25.421543   485 raft_consensus.cc:493] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.421954   485 raft_consensus.cc:3060] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.420511 32640 leader_election.cc:304] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:25.423465 32676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:25.420567 32675 raft_consensus.cc:2468] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:25.424299 32676 raft_consensus.cc:2468] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:25.424485   428 raft_consensus.cc:2804] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:25.424944   428 raft_consensus.cc:493] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.425365   428 raft_consensus.cc:3060] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.426802   520 leader_election.cc:290] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.408406   516 leader_election.cc:290] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.432412 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:25.433095 32750 raft_consensus.cc:3060] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.428432   324 leader_election.cc:304] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:25.423154 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:25.436177   498 tablet_bootstrap.cc:654] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.437003   520 raft_consensus.cc:2804] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:25.437482   520 raft_consensus.cc:493] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.437827   520 raft_consensus.cc:3060] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:25.437930   324 leader_election.cc:343] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:25.438401   527 leader_election.cc:290] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.439720 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:25.443256 32751 raft_consensus.cc:2468] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 0.
I20260709 13:11:25.448474   498 tablet_bootstrap.cc:492] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42: No bootstrap required, opened a new log
I20260709 13:11:25.449060   498 ts_tablet_manager.cc:1403] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.001s
I20260709 13:11:25.450843   496 tablet_bootstrap.cc:492] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118: No bootstrap required, opened a new log
I20260709 13:11:25.451345   496 ts_tablet_manager.cc:1403] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118: Time spent bootstrapping tablet: real 0.077s	user 0.014s	sys 0.000s
I20260709 13:11:25.451023   498 raft_consensus.cc:359] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.451926   498 raft_consensus.cc:385] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.452296   498 raft_consensus.cc:740] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.453193   498 consensus_queue.cc:260] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.454206   496 raft_consensus.cc:359] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.450660 32572 catalog_manager.cc:5697] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:25.454906   496 raft_consensus.cc:385] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.455982   498 ts_tablet_manager.cc:1434] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:25.441071   428 raft_consensus.cc:515] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.441825   516 raft_consensus.cc:493] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.458407   516 raft_consensus.cc:515] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.461256   516 leader_election.cc:290] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.461997   516 raft_consensus.cc:493] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.465853   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251194ffdb254820a2e7e2074f5db6a6" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.466843   358 raft_consensus.cc:2468] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:25.468598 32640 leader_election.cc:304] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:25.455250   496 raft_consensus.cc:740] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.470005   534 raft_consensus.cc:2804] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:25.470588   534 raft_consensus.cc:493] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.470312   496 consensus_queue.cc:260] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.462563   516 raft_consensus.cc:515] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.474031   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c6d66fe66a54a34946a3a966f1ed7b1" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.474769   358 raft_consensus.cc:2468] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:25.476152 32640 leader_election.cc:304] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:25.476658 32750 raft_consensus.cc:2468] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:25.477025   516 leader_election.cc:290] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.477738   516 raft_consensus.cc:2804] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:25.477913   322 leader_election.cc:304] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.478215   516 raft_consensus.cc:493] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.478751   516 raft_consensus.cc:3060] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.471172   534 raft_consensus.cc:3060] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.479220   521 raft_consensus.cc:2804] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:25.480113   521 raft_consensus.cc:697] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:25.481243   521 consensus_queue.cc:237] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.476296   496 ts_tablet_manager.cc:1434] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:11:25.488538   520 raft_consensus.cc:515] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.490154   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d473370c3a4b8c92694cf50174d4b6" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:25.490839   357 raft_consensus.cc:3060] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.492364 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:25.493291 32677 raft_consensus.cc:3060] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.494879   520 leader_election.cc:290] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.496838 32750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:25.497572 32750 raft_consensus.cc:3060] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.501538   357 raft_consensus.cc:2468] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:25.503127 32640 leader_election.cc:304] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:25.505887   528 raft_consensus.cc:697] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:25.505692 32750 raft_consensus.cc:2468] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:25.507051   536 raft_consensus.cc:2804] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:25.506949   528 consensus_queue.cc:237] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.505749 32677 raft_consensus.cc:2468] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:25.509477 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c6d66fe66a54a34946a3a966f1ed7b1" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:25.505748   485 raft_consensus.cc:515] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.523228 32752 raft_consensus.cc:2468] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:25.509318   428 leader_election.cc:290] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.524837   485 leader_election.cc:290] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:25.507556   322 leader_election.cc:304] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:25.525454   536 raft_consensus.cc:697] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:25.509886 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251194ffdb254820a2e7e2074f5db6a6" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:25.526723 32751 raft_consensus.cc:2468] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 0.
I20260709 13:11:25.512766   499 tablet_bootstrap.cc:654] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.510838 32749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d473370c3a4b8c92694cf50174d4b6" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:25.526850   536 consensus_queue.cc:237] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.527824 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfec1932b8046b0be6e73f558feb46d" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:25.536684 32677 raft_consensus.cc:3060] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.528409 32749 raft_consensus.cc:3060] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.540191   473 raft_consensus.cc:493] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.540836   473 raft_consensus.cc:515] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.529325 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfec1932b8046b0be6e73f558feb46d" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:25.544086 32751 raft_consensus.cc:3060] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.545423 32676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f31f60abc1e491780a4452665f85feb" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f399f3b38144952b3b85bfa93c7798d" is_pre_election: true
I20260709 13:11:25.546597 32676 raft_consensus.cc:2393] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a1b4f81a18fe44738b1e84db1a960118 in current term 1: Already voted for candidate 101b2cd1326246aea2ffd9866c9dbe42 in this term.
I20260709 13:11:25.550531   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f31f60abc1e491780a4452665f85feb" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.551272 32751 raft_consensus.cc:2468] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:25.528017   485 raft_consensus.cc:2804] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:25.552977   322 leader_election.cc:304] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:25.553831   473 leader_election.cc:290] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.552840 32715 leader_election.cc:304] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1b4f81a18fe44738b1e84db1a960118; no voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.554361   532 raft_consensus.cc:2804] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:25.555553   532 raft_consensus.cc:697] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:25.556476   532 consensus_queue.cc:237] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.560580 32677 raft_consensus.cc:2468] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 1.
I20260709 13:11:25.561233   485 raft_consensus.cc:697] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:25.560839   534 raft_consensus.cc:515] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.560918   473 raft_consensus.cc:3060] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.565249 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251194ffdb254820a2e7e2074f5db6a6" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:25.566195 32752 raft_consensus.cc:3060] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.540760   516 raft_consensus.cc:515] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.562698   485 consensus_queue.cc:237] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.552992 32749 raft_consensus.cc:2468] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:25.572751   516 leader_election.cc:290] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.573961   534 leader_election.cc:290] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.575356   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c6d66fe66a54a34946a3a966f1ed7b1" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:25.574157 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c6d66fe66a54a34946a3a966f1ed7b1" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:25.576023   357 raft_consensus.cc:3060] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.576113 32751 raft_consensus.cc:3060] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.578459   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251194ffdb254820a2e7e2074f5db6a6" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:25.579120   358 raft_consensus.cc:3060] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.580684 32573 catalog_manager.cc:5697] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:25.583271 32573 catalog_manager.cc:5697] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:25.585361 32752 raft_consensus.cc:2468] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:25.587921   473 raft_consensus.cc:2749] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:25.589587 32641 leader_election.cc:304] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:25.590581   536 raft_consensus.cc:2804] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:25.593256   536 raft_consensus.cc:697] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:25.593979 32751 raft_consensus.cc:2468] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:25.594116   536 consensus_queue.cc:237] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.595674 32641 leader_election.cc:304] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:25.596560   534 raft_consensus.cc:2804] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:25.597055   534 raft_consensus.cc:697] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:25.597847   534 consensus_queue.cc:237] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.600540 32571 catalog_manager.cc:5697] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 1 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:25.594933   357 raft_consensus.cc:2468] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:25.605360   358 raft_consensus.cc:2468] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 1.
I20260709 13:11:25.606870 32573 catalog_manager.cc:5697] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:25.609352   499 tablet_bootstrap.cc:492] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:25.609002 32573 catalog_manager.cc:5697] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 1 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:25.609934   499 ts_tablet_manager.cc:1403] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.203s	user 0.017s	sys 0.000s
I20260709 13:11:25.616771   499 raft_consensus.cc:359] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.619091   499 raft_consensus.cc:385] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.619419   499 raft_consensus.cc:740] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.620085   499 consensus_queue.cc:260] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.622442   499 ts_tablet_manager.cc:1434] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:11:25.623447   499 tablet_bootstrap.cc:492] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d: Bootstrap starting.
I20260709 13:11:25.633589   499 tablet_bootstrap.cc:654] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:25.634995 32573 catalog_manager.cc:5697] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 1 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:25.637132 32573 catalog_manager.cc:5697] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 1 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:25.638547   499 tablet_bootstrap.cc:492] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d: No bootstrap required, opened a new log
I20260709 13:11:25.639081   499 ts_tablet_manager.cc:1403] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:11:25.641801   499 raft_consensus.cc:359] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.642475   499 raft_consensus.cc:385] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:25.642681   499 raft_consensus.cc:740] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Initialized, Role: FOLLOWER
I20260709 13:11:25.643319   499 consensus_queue.cc:260] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.644999   499 ts_tablet_manager.cc:1434] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:25.673966   358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14"
dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
mode: GRACEFUL
new_leader_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:37786
I20260709 13:11:25.674543   358 raft_consensus.cc:606] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Received request to transfer leadership to TS 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.679809   358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a"
dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
mode: GRACEFUL
new_leader_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:37786
I20260709 13:11:25.680335   358 raft_consensus.cc:606] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Received request to transfer leadership to TS 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:25.685065   358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfec1932b8046b0be6e73f558feb46d"
dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
mode: GRACEFUL
new_leader_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:37786
I20260709 13:11:25.685601   358 raft_consensus.cc:606] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Received request to transfer leadership to TS a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:25.690586 32677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f"
dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
mode: GRACEFUL
new_leader_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:47600
I20260709 13:11:25.691130 32677 raft_consensus.cc:606] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Received request to transfer leadership to TS a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:25.696336 32677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01d473370c3a4b8c92694cf50174d4b6"
dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
mode: GRACEFUL
new_leader_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:47600
I20260709 13:11:25.696895 32677 raft_consensus.cc:606] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Received request to transfer leadership to TS a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:25.701725   358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1"
dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
mode: GRACEFUL
new_leader_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:37786
I20260709 13:11:25.702283   358 raft_consensus.cc:606] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Received request to transfer leadership to TS a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:25.736187   534 consensus_queue.cc:1048] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:25.748482   485 consensus_queue.cc:1048] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:25.750046   428 consensus_queue.cc:1048] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:25.765332   485 consensus_queue.cc:1048] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:25.779553   534 raft_consensus.cc:993] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d: : Instructing follower a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:25.780068   534 raft_consensus.cc:1081] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Signalling peer a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:25.782143 32751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:38794
I20260709 13:11:25.782735 32751 raft_consensus.cc:493] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:25.783113 32751 raft_consensus.cc:3060] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:25.790975 32751 raft_consensus.cc:515] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.793751 32751 leader_election.cc:290] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.793866 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:25.794885 32677 raft_consensus.cc:3055] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:25.794909   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ce7157862640f9a0078fdb6162c50f" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:25.795310 32677 raft_consensus.cc:740] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:25.795668   358 raft_consensus.cc:3060] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:25.796229 32677 consensus_queue.cc:260] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.797552 32677 raft_consensus.cc:3060] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:25.801465   358 raft_consensus.cc:2468] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:25.802652 32715 leader_election.cc:304] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:25.802999 32677 raft_consensus.cc:2468] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:25.803411   473 raft_consensus.cc:2804] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:25.803972   473 raft_consensus.cc:697] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:25.804888   473 consensus_queue.cc:237] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.812078 32571 catalog_manager.cc:5697] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 1 to 2, leader changed from 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193) to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:25.890215   536 raft_consensus.cc:493] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.890635   536 raft_consensus.cc:515] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.892513   536 leader_election.cc:290] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.893253 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:25.893854   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.894111 32751 raft_consensus.cc:2393] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6f399f3b38144952b3b85bfa93c7798d in current term 1: Already voted for candidate 101b2cd1326246aea2ffd9866c9dbe42 in this term.
I20260709 13:11:25.895429 32640 leader_election.cc:304] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d; no voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:25.896077   536 raft_consensus.cc:3060] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.901458   536 raft_consensus.cc:2749] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:25.950186   520 consensus_queue.cc:1048] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:25.954984   428 raft_consensus.cc:493] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.955646   428 raft_consensus.cc:515] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:25.961901 32752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b4f81a18fe44738b1e84db1a960118" is_pre_election: true
I20260709 13:11:25.962728 32752 raft_consensus.cc:2393] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6f399f3b38144952b3b85bfa93c7798d in current term 1: Already voted for candidate 101b2cd1326246aea2ffd9866c9dbe42 in this term.
I20260709 13:11:25.963825   428 leader_election.cc:290] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:25.967519   521 consensus_queue.cc:1048] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:25.966163   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42" is_pre_election: true
I20260709 13:11:25.970772 32640 leader_election.cc:304] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d; no voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118
I20260709 13:11:25.972093   428 raft_consensus.cc:3060] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:25.978875   521 consensus_queue.cc:1048] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:25.980472   428 raft_consensus.cc:2749] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:25.988848   532 raft_consensus.cc:993] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42: : Instructing follower 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:25.989326   532 raft_consensus.cc:1081] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Signalling peer 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:25.991262 32676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a"
dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:47584
I20260709 13:11:25.991928 32676 raft_consensus.cc:493] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:25.992278 32676 raft_consensus.cc:3060] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.001701   521 consensus_queue.cc:1048] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:26.004204 32676 raft_consensus.cc:515] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.007367 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:26.007988 32751 raft_consensus.cc:3060] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.008261 32676 leader_election.cc:290] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:26.008893   358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830dd51789be46d8a64f1e1bdf682a4a" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:26.009557   358 raft_consensus.cc:3055] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.009866   358 raft_consensus.cc:740] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:26.010754   358 consensus_queue.cc:260] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.012107   358 raft_consensus.cc:3060] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.014510   521 raft_consensus.cc:993] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42: : Instructing follower 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:26.015064   485 raft_consensus.cc:1081] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Signalling peer 6f399f3b38144952b3b85bfa93c7798d to start an election
I20260709 13:11:26.016438 32751 raft_consensus.cc:2468] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 2.
I20260709 13:11:26.016932 32677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14"
dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
 from {username='slave'} at 127.0.0.1:47584
I20260709 13:11:26.017576 32677 raft_consensus.cc:493] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.017794 32641 leader_election.cc:304] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:26.017925 32677 raft_consensus.cc:3060] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.018822   428 raft_consensus.cc:2804] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.019395   428 raft_consensus.cc:697] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:26.020226   428 consensus_queue.cc:237] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.020946   358 raft_consensus.cc:2468] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 2.
I20260709 13:11:26.025015 32677 raft_consensus.cc:515] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.028498 32677 leader_election.cc:290] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Requested vote from peers a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:26.030808 32751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:26.031059   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b61db997cb491ca829f0e77b0e8c14" candidate_uuid: "6f399f3b38144952b3b85bfa93c7798d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:26.031518 32751 raft_consensus.cc:3060] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.031711   357 raft_consensus.cc:3055] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.032107   357 raft_consensus.cc:740] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:26.032850   357 consensus_queue.cc:260] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.033152 32573 catalog_manager.cc:5697] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 1 to 2, leader changed from 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195) to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 2 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.034047   357 raft_consensus.cc:3060] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.037731   540 consensus_queue.cc:1048] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:26.039386 32751 raft_consensus.cc:2468] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 2.
I20260709 13:11:26.044684 32641 leader_election.cc:304] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:26.045578   357 raft_consensus.cc:2468] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f399f3b38144952b3b85bfa93c7798d in term 2.
I20260709 13:11:26.046022   536 raft_consensus.cc:2804] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.046814   428 consensus_queue.cc:1048] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:26.046810   536 raft_consensus.cc:697] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Becoming Leader. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:26.048358   536 consensus_queue.cc:237] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.056272   540 raft_consensus.cc:993] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d: : Instructing follower a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:26.056835   540 raft_consensus.cc:1081] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Signalling peer a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:26.061708   485 consensus_queue.cc:1048] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:26.062222 32573 catalog_manager.cc:5697] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d reported cstate change: term changed from 1 to 2, leader changed from 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195) to 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193). New cstate: current_term: 2 leader_uuid: "6f399f3b38144952b3b85bfa93c7798d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.068432 32751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01d473370c3a4b8c92694cf50174d4b6"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:38794
I20260709 13:11:26.068862   534 consensus_queue.cc:1048] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:26.069116 32751 raft_consensus.cc:493] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.069585 32751 raft_consensus.cc:3060] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.070052   540 consensus_queue.cc:1048] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:26.079223 32751 raft_consensus.cc:515] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.083420 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d473370c3a4b8c92694cf50174d4b6" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:26.084017 32677 raft_consensus.cc:3055] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.084122 32751 leader_election.cc:290] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:26.084396 32677 raft_consensus.cc:740] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:26.084043   357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d473370c3a4b8c92694cf50174d4b6" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:26.085170 32677 consensus_queue.cc:260] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.086082 32677 raft_consensus.cc:3060] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.093436 32677 raft_consensus.cc:2468] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:26.094455 32718 leader_election.cc:304] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 101b2cd1326246aea2ffd9866c9dbe42
I20260709 13:11:26.095345   520 raft_consensus.cc:993] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42: : Instructing follower a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:26.095675   523 raft_consensus.cc:2804] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.095849   520 raft_consensus.cc:1081] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Signalling peer a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:26.096282   523 raft_consensus.cc:697] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:26.097337   523 consensus_queue.cc:237] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.098783 32751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bfec1932b8046b0be6e73f558feb46d"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:38788
I20260709 13:11:26.099498 32751 raft_consensus.cc:493] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.099893 32751 raft_consensus.cc:3060] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.105312   520 consensus_queue.cc:1048] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:26.107650 32751 raft_consensus.cc:515] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.110263 32751 leader_election.cc:290] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:26.111068 32676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfec1932b8046b0be6e73f558feb46d" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:26.112821   356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfec1932b8046b0be6e73f558feb46d" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:26.112977 32750 raft_consensus.cc:1240] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Rejecting Update request from peer 101b2cd1326246aea2ffd9866c9dbe42 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:26.113582   356 raft_consensus.cc:3055] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.114188   356 raft_consensus.cc:740] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:26.115223   356 consensus_queue.cc:260] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.116358   356 raft_consensus.cc:3060] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.120014 32573 catalog_manager.cc:5697] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 1 to 2, leader changed from 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193) to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.125289   356 raft_consensus.cc:2468] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:26.126708 32715 leader_election.cc:304] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, a1b4f81a18fe44738b1e84db1a960118; no voters: 6f399f3b38144952b3b85bfa93c7798d
I20260709 13:11:26.127966   523 raft_consensus.cc:2804] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.128487   523 raft_consensus.cc:697] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:26.129360   523 consensus_queue.cc:237] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.133033   536 consensus_queue.cc:1048] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:26.141487 32572 catalog_manager.cc:5697] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 1 to 2, leader changed from 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195) to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.144968   540 consensus_queue.cc:1048] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:26.160081   520 consensus_queue.cc:1048] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:26.175071   521 consensus_queue.cc:1048] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:26.183175   520 raft_consensus.cc:993] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42: : Instructing follower a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:26.183739   521 raft_consensus.cc:1081] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Signalling peer a1b4f81a18fe44738b1e84db1a960118 to start an election
I20260709 13:11:26.185411 32751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
 from {username='slave'} at 127.0.0.1:38788
I20260709 13:11:26.185889 32751 raft_consensus.cc:493] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.186184 32751 raft_consensus.cc:3060] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.191023 32751 raft_consensus.cc:515] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.192693 32751 leader_election.cc:290] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195:40123)
I20260709 13:11:26.193691 32677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:26.193946   356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b08bd3a09478eabeb9dbcf4a453f1" candidate_uuid: "a1b4f81a18fe44738b1e84db1a960118" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
I20260709 13:11:26.194396 32677 raft_consensus.cc:3060] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.194628   356 raft_consensus.cc:3055] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.195076   356 raft_consensus.cc:740] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:26.195923   356 consensus_queue.cc:260] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.197283   356 raft_consensus.cc:3060] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.200169 32677 raft_consensus.cc:2468] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:26.201258 32718 leader_election.cc:304] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f399f3b38144952b3b85bfa93c7798d, a1b4f81a18fe44738b1e84db1a960118; no voters: 
I20260709 13:11:26.201978   523 raft_consensus.cc:2804] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.202478   523 raft_consensus.cc:697] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Becoming Leader. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:26.203322   523 consensus_queue.cc:237] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:26.204084   356 raft_consensus.cc:2468] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1b4f81a18fe44738b1e84db1a960118 in term 2.
I20260709 13:11:26.210114 32573 catalog_manager.cc:5697] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 reported cstate change: term changed from 1 to 2, leader changed from 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195) to a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194). New cstate: current_term: 2 leader_uuid: "a1b4f81a18fe44738b1e84db1a960118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.215195 32677 raft_consensus.cc:1275] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.216692   523 consensus_queue.cc:1048] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.226071   356 raft_consensus.cc:1275] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.227451   572 consensus_queue.cc:1048] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.424059 32751 raft_consensus.cc:1275] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.425556   540 consensus_queue.cc:1048] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.438478   356 raft_consensus.cc:1275] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.439971   540 consensus_queue.cc:1048] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.604857 32677 raft_consensus.cc:1275] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.606317   473 consensus_queue.cc:1048] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.607856 32677 raft_consensus.cc:3060] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.614711 32677 raft_consensus.cc:1275] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.617817   357 raft_consensus.cc:3060] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.617596   523 consensus_queue.cc:1048] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:26.621398   355 raft_consensus.cc:1275] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.624254   540 consensus_queue.cc:1048] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.625334   357 raft_consensus.cc:1275] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.627079   572 consensus_queue.cc:1048] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.639603   355 raft_consensus.cc:1275] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.641368 32751 raft_consensus.cc:1275] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Refusing update from remote peer 6f399f3b38144952b3b85bfa93c7798d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.644747   516 consensus_queue.cc:1048] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:26.647625   573 consensus_queue.cc:1048] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:26.677785 32676 raft_consensus.cc:1275] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.679109   573 consensus_queue.cc:1048] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.687182   357 raft_consensus.cc:1275] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Refusing update from remote peer a1b4f81a18fe44738b1e84db1a960118: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.692631   572 consensus_queue.cc:1048] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:29.010895 32571 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33046:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
W20260709 13:11:29.061038 32718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.127.193:37431, user_credentials={real_user=slave}} blocked reactor thread for 47395.5us
I20260709 13:11:29.077521 32571 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33046:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:11:29.101627 19967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:29.236405   600 log.cc:927] Time spent T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:11:29.284842   610 log.cc:927] Time spent T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:11:29.363859   625 tablet.cc:1724] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.364328   621 tablet.cc:1724] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.368466   624 tablet.cc:1724] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.397735   617 tablet.cc:1724] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:29.428333   619 log.cc:927] Time spent T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 13:11:29.429354   618 log.cc:927] Time spent T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:11:29.421626   632 tablet.cc:1724] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.446053   637 tablet.cc:1724] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.446534   630 tablet.cc:1724] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:29.455734   620 log.cc:927] Time spent T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118: Append to log took a long time: real 0.102s	user 0.002s	sys 0.000s
W20260709 13:11:29.455739   622 log.cc:927] Time spent T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118: Append to log took a long time: real 0.103s	user 0.002s	sys 0.000s
I20260709 13:11:29.512712   639 tablet.cc:1724] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.513094   642 tablet.cc:1724] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:29.521780   628 log.cc:927] Time spent T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:11:29.528575   633 log.cc:927] Time spent T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:11:29.530198   635 tablet.cc:1724] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.543936   629 tablet.cc:1724] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.549489   632 tablet.cc:1724] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.556093   617 tablet.cc:1724] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.560786   625 tablet.cc:1724] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.562886   643 tablet.cc:1724] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.564304   637 tablet.cc:1724] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.564296   642 tablet.cc:1724] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.582533   621 tablet.cc:1724] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.572810   632 tablet.cc:1724] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.570502   630 tablet.cc:1724] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.610038   642 tablet.cc:1724] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.616945   624 tablet.cc:1724] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.618476 32748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90"
dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
mode: GRACEFUL
new_leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
 from {username='slave'} at 127.0.0.1:38820
I20260709 13:11:29.644261 32748 raft_consensus.cc:606] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Received request to transfer leadership to TS 101b2cd1326246aea2ffd9866c9dbe42
I20260709 13:11:29.620747   635 tablet.cc:1724] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.628523   630 tablet.cc:1724] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.633455   617 tablet.cc:1724] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.637254   621 tablet.cc:1724] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.639930   625 tablet.cc:1724] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:29.670722 32674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f"
dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
mode: GRACEFUL
new_leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
 from {username='slave'} at 127.0.0.1:47612
I20260709 13:11:29.672618 32674 raft_consensus.cc:606] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Received request to transfer leadership to TS 101b2cd1326246aea2ffd9866c9dbe42
I20260709 13:11:29.676596 19967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:11:29.677287 19967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:29.695647 32575 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918: soft_deleted_table [id=450652246e13439483f2fdf7f90276d1] alter complete (version 1)
W20260709 13:11:29.974524   379 debug-util.cc:398] Leaking SignalData structure 0x7b0800160ba0 after lost signal to thread 19970
W20260709 13:11:29.975860   379 debug-util.cc:398] Leaking SignalData structure 0x7b0800204640 after lost signal to thread 32609
W20260709 13:11:29.978772   379 debug-util.cc:398] Leaking SignalData structure 0x7b08001199c0 after lost signal to thread 382
I20260709 13:11:29.994336   580 raft_consensus.cc:993] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d: : Instructing follower 101b2cd1326246aea2ffd9866c9dbe42 to start an election
I20260709 13:11:29.995167   581 raft_consensus.cc:1081] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Signalling peer 101b2cd1326246aea2ffd9866c9dbe42 to start an election
I20260709 13:11:29.997239   355 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f"
dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
 from {username='slave'} at 127.0.0.1:37776
I20260709 13:11:29.997839   355 raft_consensus.cc:493] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.998194   355 raft_consensus.cc:3060] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:30.004940   355 raft_consensus.cc:515] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:30.006851   355 leader_election.cc:290] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 3 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:30.008050 32748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:30.008159 32674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcfc2016f2e4adbabdc926eba611c6f" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:30.008980 32748 raft_consensus.cc:3060] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:30.009092 32674 raft_consensus.cc:3055] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:30.009730 32674 raft_consensus.cc:740] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f399f3b38144952b3b85bfa93c7798d, State: Running, Role: LEADER
I20260709 13:11:30.011184 32674 consensus_queue.cc:260] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:30.012949 32674 raft_consensus.cc:3060] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:30.020438   584 raft_consensus.cc:993] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118: : Instructing follower 101b2cd1326246aea2ffd9866c9dbe42 to start an election
I20260709 13:11:30.021059   584 raft_consensus.cc:1081] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Signalling peer 101b2cd1326246aea2ffd9866c9dbe42 to start an election
I20260709 13:11:30.023663 32674 raft_consensus.cc:2468] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 3.
I20260709 13:11:30.024503   356 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90"
dest_uuid: "101b2cd1326246aea2ffd9866c9dbe42"
 from {username='slave'} at 127.0.0.1:37780
I20260709 13:11:30.025314   356 raft_consensus.cc:493] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:30.025781   356 raft_consensus.cc:3060] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:30.025667   324 leader_election.cc:304] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:30.027235   582 raft_consensus.cc:2804] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:30.027252 32748 raft_consensus.cc:2468] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 3.
I20260709 13:11:30.028996   582 raft_consensus.cc:697] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:30.030051   582 consensus_queue.cc:237] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:30.039199   356 raft_consensus.cc:515] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:30.042533 32674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f399f3b38144952b3b85bfa93c7798d"
I20260709 13:11:30.043218 32674 raft_consensus.cc:3060] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:30.042616 32575 catalog_manager.cc:5697] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 2 to 3, leader changed from 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193) to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:30.044471 32748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbdaca625174743a36e9d8388b1cb90" candidate_uuid: "101b2cd1326246aea2ffd9866c9dbe42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a1b4f81a18fe44738b1e84db1a960118"
I20260709 13:11:30.045114 32748 raft_consensus.cc:3055] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:30.045408 32748 raft_consensus.cc:740] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a1b4f81a18fe44738b1e84db1a960118, State: Running, Role: LEADER
I20260709 13:11:30.046262 32748 consensus_queue.cc:260] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:30.047497 32748 raft_consensus.cc:3060] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:30.050936 32674 raft_consensus.cc:2468] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 3.
I20260709 13:11:30.052235   356 leader_election.cc:290] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 3 election: Requested vote from peers 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431), a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419)
I20260709 13:11:30.053987   324 leader_election.cc:304] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101b2cd1326246aea2ffd9866c9dbe42, 6f399f3b38144952b3b85bfa93c7798d; no voters: 
I20260709 13:11:30.055403   582 raft_consensus.cc:2804] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:30.056147   582 raft_consensus.cc:697] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 LEADER]: Becoming Leader. State: Replica: 101b2cd1326246aea2ffd9866c9dbe42, State: Running, Role: LEADER
I20260709 13:11:30.057108   582 consensus_queue.cc:237] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } }
I20260709 13:11:30.058190 32748 raft_consensus.cc:2468] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101b2cd1326246aea2ffd9866c9dbe42 in term 3.
I20260709 13:11:30.068101 32575 catalog_manager.cc:5697] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 reported cstate change: term changed from 2 to 3, leader changed from a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194) to 101b2cd1326246aea2ffd9866c9dbe42 (127.19.127.195). New cstate: current_term: 3 leader_uuid: "101b2cd1326246aea2ffd9866c9dbe42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101b2cd1326246aea2ffd9866c9dbe42" member_type: VOTER last_known_addr { host: "127.19.127.195" port: 40123 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:30.452020 32748 raft_consensus.cc:1275] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 3 FOLLOWER]: Refusing update from remote peer 101b2cd1326246aea2ffd9866c9dbe42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:30.453575   582 consensus_queue.cc:1048] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:30.472170 32674 raft_consensus.cc:1275] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 3 FOLLOWER]: Refusing update from remote peer 101b2cd1326246aea2ffd9866c9dbe42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:30.475917 32674 raft_consensus.cc:1275] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 3 FOLLOWER]: Refusing update from remote peer 101b2cd1326246aea2ffd9866c9dbe42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:30.480250   649 consensus_queue.cc:1048] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:30.476006   582 consensus_queue.cc:1048] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f399f3b38144952b3b85bfa93c7798d" member_type: VOTER last_known_addr { host: "127.19.127.193" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:30.514197 32748 raft_consensus.cc:1275] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 3 FOLLOWER]: Refusing update from remote peer 101b2cd1326246aea2ffd9866c9dbe42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:30.520063   582 consensus_queue.cc:1048] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b4f81a18fe44738b1e84db1a960118" member_type: VOTER last_known_addr { host: "127.19.127.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:30.744525 19967 tablet_server.cc:179] TabletServer@127.19.127.193:0 shutting down...
W20260709 13:11:30.778417 32718 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 12 similar messages]
W20260709 13:11:30.784389 32718 consensus_peers.cc:597] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 -> Peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Couldn't send request to peer 6f399f3b38144952b3b85bfa93c7798d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:30.796564 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:30.799633 19967 tablet_replica.cc:333] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.801031 19967 raft_consensus.cc:2243] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.802248 19967 raft_consensus.cc:2272] T 32b61db997cb491ca829f0e77b0e8c14 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.805861 19967 tablet_replica.cc:333] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.806883 19967 raft_consensus.cc:2243] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.808089 19967 raft_consensus.cc:2272] T 830dd51789be46d8a64f1e1bdf682a4a P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.811424 19967 tablet_replica.cc:333] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.812358 19967 raft_consensus.cc:2243] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.813431 19967 raft_consensus.cc:2272] T 251194ffdb254820a2e7e2074f5db6a6 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.817077 19967 tablet_replica.cc:333] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.818256 19967 raft_consensus.cc:2243] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.819034 19967 raft_consensus.cc:2272] T 01d473370c3a4b8c92694cf50174d4b6 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.822609 19967 tablet_replica.cc:333] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.823733 19967 raft_consensus.cc:2243] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.824477 19967 raft_consensus.cc:2272] T 9bfec1932b8046b0be6e73f558feb46d P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.828102 19967 tablet_replica.cc:333] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.829265 19967 raft_consensus.cc:2243] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.829962 19967 raft_consensus.cc:2272] T 548b08bd3a09478eabeb9dbcf4a453f1 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.833539 19967 tablet_replica.cc:333] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.834652 19967 raft_consensus.cc:2243] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.835337 19967 raft_consensus.cc:2272] T 8fcfc2016f2e4adbabdc926eba611c6f P 6f399f3b38144952b3b85bfa93c7798d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.839393 19967 tablet_replica.cc:333] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.840513 19967 raft_consensus.cc:2243] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.841749 19967 raft_consensus.cc:2272] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.845283 19967 tablet_replica.cc:333] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.846217 19967 raft_consensus.cc:2243] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.846902 19967 raft_consensus.cc:2272] T 8f31f60abc1e491780a4452665f85feb P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.850216 19967 tablet_replica.cc:333] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.851164 19967 raft_consensus.cc:2243] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.851889 19967 raft_consensus.cc:2272] T 77ce7157862640f9a0078fdb6162c50f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.855118 19967 tablet_replica.cc:333] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.856107 19967 raft_consensus.cc:2243] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.857419 19967 raft_consensus.cc:2272] T 0eb15a9d614142a288d7f82e7f7d470b P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.860998 19967 tablet_replica.cc:333] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.862025 19967 raft_consensus.cc:2243] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.862686 19967 raft_consensus.cc:2272] T c02e9a4cbb294b84b80687bafc097470 P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.866410 19967 tablet_replica.cc:333] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.867381 19967 raft_consensus.cc:2243] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.868032 19967 raft_consensus.cc:2272] T 6dbdaca625174743a36e9d8388b1cb90 P 6f399f3b38144952b3b85bfa93c7798d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.873214 19967 tablet_replica.cc:333] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.874182 19967 raft_consensus.cc:2243] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.874929 19967 raft_consensus.cc:2272] T 5ac46056ab9c41a88146e39bff533e4f P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.878916 19967 tablet_replica.cc:333] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.879853 19967 raft_consensus.cc:2243] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.880589 19967 raft_consensus.cc:2272] T 84e5964fd90e4a5aa0453e1d5e19b480 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:30.881491 32718 consensus_peers.cc:597] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 -> Peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Couldn't send request to peer 6f399f3b38144952b3b85bfa93c7798d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:30.884500 19967 tablet_replica.cc:333] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.885574 19967 raft_consensus.cc:2243] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.886842 19967 raft_consensus.cc:2272] T fe2467e91cd54fa9b43a9fe739ecff76 P 6f399f3b38144952b3b85bfa93c7798d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.890600 19967 tablet_replica.cc:333] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.891587 19967 raft_consensus.cc:2243] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.892323 19967 raft_consensus.cc:2272] T 83a092096b9341099d87e2d7e187bcdf P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.896006 19967 tablet_replica.cc:333] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d: stopping tablet replica
I20260709 13:11:30.896845 19967 raft_consensus.cc:2243] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.898126 19967 raft_consensus.cc:2272] T 5e764d2123ed4dd38e5dabeb6cc4213e P 6f399f3b38144952b3b85bfa93c7798d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:30.937860   324 consensus_peers.cc:597] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 -> Peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Couldn't send request to peer 6f399f3b38144952b3b85bfa93c7798d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:30.956686   324 consensus_peers.cc:597] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 -> Peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Couldn't send request to peer 6f399f3b38144952b3b85bfa93c7798d. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:37431: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:30.957231 19967 tablet_server.cc:196] TabletServer@127.19.127.193:0 shutdown complete.
I20260709 13:11:31.001293 19967 tablet_server.cc:179] TabletServer@127.19.127.194:0 shutting down...
W20260709 13:11:31.040678   324 consensus_peers.cc:597] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 -> Peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Couldn't send request to peer 6f399f3b38144952b3b85bfa93c7798d. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:37431: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.046025 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:31.046977 19967 tablet_replica.cc:333] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.047786 19967 raft_consensus.cc:2243] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.048365 19967 raft_consensus.cc:2272] T 8f31f60abc1e491780a4452665f85feb P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.050715 19967 tablet_replica.cc:333] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.051556 19967 raft_consensus.cc:2243] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.052386 19967 raft_consensus.cc:2272] T 548b08bd3a09478eabeb9dbcf4a453f1 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.054888 19967 tablet_replica.cc:333] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.055665 19967 raft_consensus.cc:2243] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.056188 19967 raft_consensus.cc:2272] T 32b61db997cb491ca829f0e77b0e8c14 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.058559 19967 tablet_replica.cc:333] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.059345 19967 raft_consensus.cc:2243] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.060288 19967 raft_consensus.cc:2272] T 9bfec1932b8046b0be6e73f558feb46d P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.063966 19967 tablet_replica.cc:333] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.065282 19967 raft_consensus.cc:2243] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.066864 19967 raft_consensus.cc:2272] T 01d473370c3a4b8c92694cf50174d4b6 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.071399 19967 tablet_replica.cc:333] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.072220 19967 raft_consensus.cc:2243] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.072904 19967 raft_consensus.cc:2272] T 830dd51789be46d8a64f1e1bdf682a4a P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.075409 32718 consensus_peers.cc:597] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 -> Peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Couldn't send request to peer 6f399f3b38144952b3b85bfa93c7798d. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:37431: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.076203 19967 tablet_replica.cc:333] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.077081 19967 raft_consensus.cc:2243] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.077586 19967 raft_consensus.cc:2272] T 3c6d66fe66a54a34946a3a966f1ed7b1 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.079972 19967 tablet_replica.cc:333] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.080600 19967 raft_consensus.cc:2243] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.081120 19967 raft_consensus.cc:2272] T 8fcfc2016f2e4adbabdc926eba611c6f P a1b4f81a18fe44738b1e84db1a960118 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.084069 19967 tablet_replica.cc:333] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.084656 19967 raft_consensus.cc:2243] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.085597 19967 raft_consensus.cc:2272] T 5ac46056ab9c41a88146e39bff533e4f P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.087984 19967 tablet_replica.cc:333] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.088570 19967 raft_consensus.cc:2243] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.089015 19967 raft_consensus.cc:2272] T 6dbdaca625174743a36e9d8388b1cb90 P a1b4f81a18fe44738b1e84db1a960118 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.091813 19967 tablet_replica.cc:333] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.092396 19967 raft_consensus.cc:2243] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.093470 19967 raft_consensus.cc:2272] T 77ce7157862640f9a0078fdb6162c50f P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.096588 19967 tablet_replica.cc:333] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.097340 19967 raft_consensus.cc:2243] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:31.099450   322 consensus_peers.cc:597] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 -> Peer a1b4f81a18fe44738b1e84db1a960118 (127.19.127.194:39419): Couldn't send request to peer a1b4f81a18fe44738b1e84db1a960118. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.100031 19967 raft_consensus.cc:2272] T 0eb15a9d614142a288d7f82e7f7d470b P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.103847 19967 tablet_replica.cc:333] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.104465 19967 raft_consensus.cc:2243] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.104945 19967 raft_consensus.cc:2272] T c02e9a4cbb294b84b80687bafc097470 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.107650 19967 tablet_replica.cc:333] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.108230 19967 raft_consensus.cc:2243] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.109092 19967 raft_consensus.cc:2272] T 84e5964fd90e4a5aa0453e1d5e19b480 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.111410 19967 tablet_replica.cc:333] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.111956 19967 raft_consensus.cc:2243] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.112495 19967 raft_consensus.cc:2272] T 251194ffdb254820a2e7e2074f5db6a6 P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.114907 19967 tablet_replica.cc:333] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.115481 19967 raft_consensus.cc:2243] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.116029 19967 raft_consensus.cc:2272] T 5e764d2123ed4dd38e5dabeb6cc4213e P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.118512 19967 tablet_replica.cc:333] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.119059 19967 raft_consensus.cc:2243] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.119951 19967 raft_consensus.cc:2272] T 83a092096b9341099d87e2d7e187bcdf P a1b4f81a18fe44738b1e84db1a960118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.122287 19967 tablet_replica.cc:333] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118: stopping tablet replica
I20260709 13:11:31.122809 19967 raft_consensus.cc:2243] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.123483 19967 raft_consensus.cc:2272] T fe2467e91cd54fa9b43a9fe739ecff76 P a1b4f81a18fe44738b1e84db1a960118 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.168062 19967 tablet_server.cc:196] TabletServer@127.19.127.194:0 shutdown complete.
I20260709 13:11:31.210913 19967 tablet_server.cc:179] TabletServer@127.19.127.195:0 shutting down...
W20260709 13:11:31.245833   324 consensus_peers.cc:597] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 -> Peer 6f399f3b38144952b3b85bfa93c7798d (127.19.127.193:37431): Couldn't send request to peer 6f399f3b38144952b3b85bfa93c7798d. Status: Network error: Client connection negotiation failed: client connection to 127.19.127.193:37431: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.249313 19967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:31.250195 19967 tablet_replica.cc:333] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.251206 19967 raft_consensus.cc:2243] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.251888 19967 raft_consensus.cc:2272] T 251194ffdb254820a2e7e2074f5db6a6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.254073 19967 tablet_replica.cc:333] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.254678 19967 raft_consensus.cc:2243] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.255178 19967 raft_consensus.cc:2272] T 548b08bd3a09478eabeb9dbcf4a453f1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.257252 19967 tablet_replica.cc:333] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.257881 19967 raft_consensus.cc:2243] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.258365 19967 raft_consensus.cc:2272] T 830dd51789be46d8a64f1e1bdf682a4a P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.260491 19967 tablet_replica.cc:333] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.261059 19967 raft_consensus.cc:2243] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.261549 19967 raft_consensus.cc:2272] T 9bfec1932b8046b0be6e73f558feb46d P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.263846 19967 tablet_replica.cc:333] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.264415 19967 raft_consensus.cc:2243] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.265273 19967 raft_consensus.cc:2272] T 8f31f60abc1e491780a4452665f85feb P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.267486 19967 tablet_replica.cc:333] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.268090 19967 raft_consensus.cc:2243] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.268573 19967 raft_consensus.cc:2272] T 32b61db997cb491ca829f0e77b0e8c14 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.270720 19967 tablet_replica.cc:333] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.271359 19967 raft_consensus.cc:2243] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.271824 19967 raft_consensus.cc:2272] T 83a092096b9341099d87e2d7e187bcdf P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.274511 19967 tablet_replica.cc:333] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.275245 19967 raft_consensus.cc:2243] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.275839 19967 raft_consensus.cc:2272] T fe2467e91cd54fa9b43a9fe739ecff76 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.278324 19967 tablet_replica.cc:333] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.278946 19967 raft_consensus.cc:2243] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.279460 19967 raft_consensus.cc:2272] T 77ce7157862640f9a0078fdb6162c50f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.281750 19967 tablet_replica.cc:333] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.282348 19967 raft_consensus.cc:2243] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.282894 19967 raft_consensus.cc:2272] T 0eb15a9d614142a288d7f82e7f7d470b P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.285437 19967 tablet_replica.cc:333] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.286027 19967 raft_consensus.cc:2243] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.286932 19967 raft_consensus.cc:2272] T c02e9a4cbb294b84b80687bafc097470 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.289415 19967 tablet_replica.cc:333] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.289981 19967 raft_consensus.cc:2243] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.290479 19967 raft_consensus.cc:2272] T 5ac46056ab9c41a88146e39bff533e4f P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.293026 19967 tablet_replica.cc:333] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.293601 19967 raft_consensus.cc:2243] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.294523 19967 raft_consensus.cc:2272] T 6dbdaca625174743a36e9d8388b1cb90 P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.297729 19967 tablet_replica.cc:333] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.298286 19967 raft_consensus.cc:2243] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.298779 19967 raft_consensus.cc:2272] T 3c6d66fe66a54a34946a3a966f1ed7b1 P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.301154 19967 tablet_replica.cc:333] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.301743 19967 raft_consensus.cc:2243] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.302567 19967 raft_consensus.cc:2272] T 8fcfc2016f2e4adbabdc926eba611c6f P 101b2cd1326246aea2ffd9866c9dbe42 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.305331 19967 tablet_replica.cc:333] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.305975 19967 raft_consensus.cc:2243] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.306416 19967 raft_consensus.cc:2272] T 01d473370c3a4b8c92694cf50174d4b6 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.308543 19967 tablet_replica.cc:333] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.309077 19967 raft_consensus.cc:2243] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.309548 19967 raft_consensus.cc:2272] T 5e764d2123ed4dd38e5dabeb6cc4213e P 101b2cd1326246aea2ffd9866c9dbe42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.311897 19967 tablet_replica.cc:333] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42: stopping tablet replica
I20260709 13:11:31.312376 19967 raft_consensus.cc:2243] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.312853 19967 raft_consensus.cc:2272] T 84e5964fd90e4a5aa0453e1d5e19b480 P 101b2cd1326246aea2ffd9866c9dbe42 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:32.352392 19967 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 379)
W20260709 13:11:33.352900 19967 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 379)
I20260709 13:11:33.622124 19967 tablet_server.cc:196] TabletServer@127.19.127.195:0 shutdown complete.
I20260709 13:11:33.659677 19967 master.cc:562] Master@127.19.127.254:35923 shutting down...
I20260709 13:11:33.690215 19967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:33.690897 19967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:33.691262 19967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9289fd0c13be4e8c93ccc54aea6c3918: stopping tablet replica
I20260709 13:11:33.712428 19967 master.cc:584] Master@127.19.127.254:35923 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (13990 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (27050 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (379344 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:11:33.769968 19967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 425 messages since previous log ~17 seconds ago
I20260709 13:11:33.770179 19967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 424 messages since previous log ~17 seconds ago
I20260709 13:11:33.770316 19967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 417 messages since previous log ~17 seconds ago
I20260709 13:11:33.770465 19967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:11:33.770610 19967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:11:33.770778 19967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 7 messages since previous log ~164 seconds ago
I20260709 13:11:33.770987 19967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 7 messages since previous log ~2 seconds ago
I20260709 13:11:33.771313 19967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 122 messages since previous log ~35 seconds ago
